Intelligent dynamic tracking method and device based on infrared thermal imaging recognition technology

Abstract

The application provides an intelligent dynamic tracking method and device based on infrared thermal imaging recognition technology, and relates to the field of intelligent tracking. The method collects thermal infrared video streams of a target area, applies image processing algorithms to segment the images, extracts high-temperature areas, sets dynamic temperature thresholds, and preliminarily screens out possible high-temperature target areas. Then, an artificial intelligence and deep learning model is used to classify suspected high-temperature areas to distinguish fire source and non-fire source areas. Finally, based on the real-time position and motion trail of the fire source, the best moving path of the infrared thermal imaging camera is calculated, and the infrared thermal imaging camera is controlled to continuously track the fire source. In this way, the time sequence correlation of thermal image video streams can be utilized to adapt to the overlapping challenge of high-temperature equipment and initial fire in industrial scenes, and the accuracy and efficiency of fire source recognition and positioning can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent tracking, and more particularly to an intelligent dynamic tracking method and device based on infrared thermal imaging recognition technology in the embodiments of the present application. BACKGROUND

[0002] With the rapid development of infrared thermal imaging technology, it shows unique advantages in the field of fire monitoring. Traditional fire detection systems are mostly based on visible light image analysis or smoke sensors, which have problems such as night monitoring failure and smoke diffusion lag. Infrared thermal imaging can realize all-weather temperature field monitoring by capturing the surface radiation heat of objects, and is particularly suitable for early fire warning. However, in actual industrial scenes, the thermal radiation generated by high-temperature equipment operation (such as metallurgical furnaces, reaction kettles, etc.) has temperature field overlap with initial fire, which brings great challenges to the accurate identification of the ignition point.

[0003] The prior art generally uses a static temperature threshold method to judge the fire, but research shows that when the environmental temperature fluctuates too much, the false positive rate of the fixed threshold will increase sharply. Although adaptive image processing algorithms such as threshold segmentation can effectively extract high-temperature regions, there are still the following technical bottlenecks in complex industrial scenes: 1. Single-frame thermal image lacks temperature evolution information, making it difficult to distinguish between transient high-temperature equipment and continuously spreading fire sources; 2. Traditional morphological processing is not sensitive enough to the dynamic flame edge features; 3. Temperature field distortion caused by thermal convection will interfere with static feature analysis. These factors will affect the judgment and tracking of the ignition point.

[0004] Therefore, an intelligent dynamic tracking scheme based on infrared thermal imaging recognition technology is expected. SUMMARY

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an intelligent dynamic tracking method and device based on infrared thermal imaging recognition technology, which acquires thermal infrared video stream of a target area, applies image processing algorithms to segment the image, extracts high-temperature regions, and sets a dynamic temperature threshold to preliminarily screen out possible high-temperature target regions. Then, an artificial intelligence and deep learning model is used to classify suspected high-temperature regions to distinguish between fire and non-fire regions. Finally, based on the real-time position and motion trail of the fire source, the best moving path of the infrared thermal imaging camera is calculated to control the infrared thermal imaging camera to continuously track the fire source. In this way, the time sequence correlation of thermal image video stream can be utilized to adapt to the overlap challenge of high-temperature equipment and initial fire in industrial scenes, and the accuracy and efficiency of fire source identification and positioning are improved.

[0006] According to one aspect of the present application, an intelligent dynamic tracking method based on infrared thermal imaging recognition technology is provided, which comprises:

[0007] acquire a thermal infrared tracking video stream of a target region captured by an infrared thermal imaging camera;

[0008] perform key frame discrete sampling on the thermal infrared tracking video stream, and then perform preprocessing on a time queue of thermal infrared tracking key frames obtained by the key frame discrete sampling based on image segmentation and data classification to obtain a set of time queues of suspected high temperature regions;

[0009] classify each time queue of suspected high temperature regions in the set of time queues of suspected high temperature regions using a deep learning model to determine whether each suspected high temperature region is a fire source region or a non-fire source region, including: performing temperature distribution space feature extraction and key clue feature filtering on the set of time queues of suspected high temperature regions to obtain a time queue of suspected high temperature region temperature distribution features; and performing high temperature region classification based on temperature time sequence fluctuation feature analysis on the time queue of suspected high temperature region temperature distribution features to determine whether the suspected high temperature region is a fire source region or a non-fire source region;

[0010] based on the time queue of fire source regions, calculate an optimal movement path of the infrared thermal imaging camera to track a fire starting point through the infrared thermal imaging camera.

[0011] According to another aspect of the present application, an intelligent dynamic tracking device based on infrared thermal imaging recognition technology is provided, which comprises:

[0012] a thermal infrared tracking video stream acquisition module configured to acquire a thermal infrared tracking video stream of a target region captured by an infrared thermal imaging camera;

[0013] a thermal infrared tracking video stream preprocessing module configured to perform key frame discrete sampling on the thermal infrared tracking video stream, and then perform preprocessing on a time queue of thermal infrared tracking key frames obtained by the key frame discrete sampling based on image segmentation and data classification to obtain a set of time queues of suspected high temperature regions;

[0014] a fire source region determination module configured to classify each time queue of suspected high temperature regions in the set of time queues of suspected high temperature regions using a deep learning model to determine whether each suspected high temperature region is a fire source region or a non-fire source region;

[0015] a fire starting point tracking module configured to calculate an optimal movement path of the infrared thermal imaging camera based on the time queue of fire source regions to track a fire starting point through the infrared thermal imaging camera.

[0016] Compared with the prior art, the intelligent dynamic tracking method and device based on infrared thermal imaging recognition technology provided by the application can collect thermal infrared video streams of a target area, apply image processing algorithms to segment the images, extract high-temperature areas, and set a dynamic temperature threshold to preliminarily screen out possible high-temperature target areas. Then, an artificial intelligence and deep learning model is used to classify suspected high-temperature areas to distinguish fire source areas from non-fire source areas. Finally, based on the real-time position and motion trail of the fire source, the optimal moving path of the infrared thermal imaging camera is calculated to control the infrared thermal imaging camera to continuously track the fire source. In this way, the time sequence correlation of thermal image video streams can be used to adapt to the overlapping challenge of high-temperature equipment and initial fire in an industrial scene, and the accuracy and efficiency of fire source recognition and positioning are improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, when taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to explain constituents of the present application and to provide in conjunction with the specification a complete disclosure of the application, but do not in any way limit the present application. In the drawings, like reference numerals refer to like parts or steps throughout the several views.

[0018] Figure 1 A flowchart of the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application.

[0019] Figure 2 A data flow diagram of the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application.

[0020] Figure 3 A flowchart of the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application, in which a deep learning model is used to classify each time queue of suspected high-temperature areas in the set of time queues of suspected high-temperature areas to determine whether each suspected high-temperature area is a fire source area or a non-fire source area.

[0021] Figure 4 A flowchart of the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application, in which the set of time queues of suspected high-temperature areas is subjected to temperature distribution space feature extraction and key clue feature filtering to obtain a time queue of suspected high-temperature area temperature distribution features.

[0022] Figure 5A flowchart of heat anomaly area temperature distribution key feature filtering on a time queue of suspected high-temperature area temperature distribution feature coding vectors in an intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application to obtain a filtered time queue of suspected high-temperature area temperature distribution feature coding vectors as a time queue of suspected high-temperature area temperature distribution features.

[0023] Figure 6 A block diagram of an intelligent dynamic tracking device based on infrared thermal imaging recognition technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings represent functionally the same or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0025] In recent years, deep learning has made breakthrough progress in thermal imaging analysis, and through the fusion of regional thermal infrared temperature distribution spatiotemporal features, it can more effectively identify flames. However, existing researches mostly focus on single-frame image classification and fail to effectively utilize the temporal correlation of thermal image video streams. Meanwhile, dynamic tracking still relies on preset scanning paths and lacks intelligent servo control mechanisms based on fire evolution.

[0026] To address the above technical problems, in the technical solution of the present application, an intelligent dynamic tracking method based on infrared thermal imaging recognition technology is proposed. Specifically, the tracking method applies image processing algorithms to segment images, extract high-temperature areas, and set dynamic temperature thresholds to preliminarily screen out possible high-temperature target areas. Then, artificial intelligence and deep learning models are used to classify suspected high-temperature areas to distinguish between fire sources and non-fire source areas. Based on the real-time location and motion trajectory of the fire source, the best moving path of the infrared thermal imaging camera is calculated to control the infrared thermal imaging camera to continuously track the fire source.

[0027] Figure 1 A flowchart of an intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application. Figure 2 A data flow diagram of an intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application. As shown in Figure 1 and Figure 2As shown, the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to the embodiment of the present application comprises: S110, acquiring a thermal infrared tracking video stream of a target region collected by an infrared thermal imaging camera; S120, after key frame discrete sampling is performed on the thermal infrared tracking video stream, then performing preprocessing on the time queue of the thermal infrared tracking key frame obtained based on image segmentation and data classification to obtain a set of time queues of suspected high-temperature regions; S130, using a deep learning model to classify each time queue of suspected high-temperature regions in the set of time queues of suspected high-temperature regions to determine each suspected high-temperature region as a fire source region or a non-fire source region; S140, based on the time queue of the fire source region, calculating an optimal moving path of the infrared thermal imaging camera to track the ignition point through the infrared thermal imaging camera.

[0028] In the above intelligent dynamic tracking method based on infrared thermal imaging recognition technology, the step S110 acquires a thermal infrared tracking video stream of a target region collected by an infrared thermal imaging camera. It can be understood that infrared thermal imaging is not limited by lighting conditions (such as at night, in a smoky environment), and is suitable for places such as factories that operate 24 hours a day. At the same time, the thermal infrared tracking video stream can record the temperature change process in real time and capture the continuity characteristics of the spread of the fire source, while a single image cannot reflect the trend of the spread of the fire. Therefore, in the technical solution of the present application, a thermal infrared tracking video stream of a target region collected by an infrared thermal imaging camera is acquired. Specifically, the working principle of the infrared thermal imaging camera is to image based on the infrared energy of the surface radiation of an object. The infrared thermal imaging camera can perceive and record the temperature distribution information in the target region and generate image data based on thermal energy. These data not only include the static temperature distribution, but also include dynamic time series information. In the technical solution of the present application, the camera is installed at a position that can cover the target monitoring area, and the focal length and the angle of view range are adjusted according to the specific scene to ensure that the possible high-temperature region can be fully captured. In addition, in order to adapt to complex industrial environments, the infrared thermal imaging camera also has the characteristics of waterproof and dustproof. Next, in the data acquisition process, the infrared thermal imaging camera will continuously generate a thermal infrared video stream. This video stream is composed of a series of continuous thermal infrared image frames, and each frame reflects the temperature distribution of the target region at a specific time point. Then, the collected thermal infrared tracking video stream will be transmitted to a central processing unit or a cloud server for further analysis. This transmission process can be realized through wired or wireless networks. For some occasions with high real-time requirements, such as chemical plants or metallurgical workshops, high-speed wired network connection is adopted to ensure the stability and low delay of data transmission; and for some more dispersed deployment scenarios, wireless communication technology (such as Wi-Fi or 5G) can be used to realize flexible data transmission.

[0029] In the intelligent dynamic tracking method based on the infrared thermal imaging recognition technology, after the step S120, the key frame discrete sampling is performed on the thermal infrared tracking video stream, and then the time queue of the obtained thermal infrared tracking key frames is preprocessed based on image segmentation and data classification to obtain a set of time queues of suspected high temperature areas, including: S121, performing key frame discrete sampling on the thermal infrared tracking video stream to obtain a time queue of thermal infrared tracking key frames; S122, performing image segmentation on each thermal infrared tracking key frame in the time queue of thermal infrared tracking key frames based on an Otsu threshold segmentation algorithm to obtain a time queue of a set of suspected high temperature areas; S123, based on the position of the suspected high temperature area in the thermal infrared tracking key frame, performing data classification on the time queue of the set of suspected high temperature areas to obtain a set of time queues of the suspected high temperature areas.

[0030] Specifically, in the step S121, the thermal infrared tracking video stream is key frame discrete sampled to obtain a time queue of thermal infrared tracking key frames. It should be understood that, considering the large amount of video stream data in industrial scenes, the thermal infrared tracking video stream is further key frame discrete sampled to obtain a time queue of thermal infrared tracking key frames. Discrete sampling can reduce the consumption of computing resources in subsequent processing while retaining key information of fire evolution. Specifically, by key frame discrete sampling, representative image frames can be screened from the video stream to form a time queue, and it is ensured that these key frames can fully reflect the temperature change trend of the target region, while avoiding the loss of important information due to too large sampling interval. In the specific embodiments of the present application, the selection of key frames is determined by time interval sampling and content-aware sampling. Among them, time interval sampling extracts image frames from the video stream according to a fixed time interval. This method can achieve simple and easy-to-control sampling frequency, but cannot adapt to the temperature change rate in different scenes, that is, for fast-spreading fire sources, a fixed time interval may lead to the omission of key information; while for slow-changing high-temperature equipment, too many key frames will increase unnecessary computing burden. In this case, the technical solution of the present application samples through content awareness, that is, by analyzing the content features of each frame of image, it is judged which frames contain important temperature change information, and they are marked as key frames. If a frame has a significant temperature rise or flame expansion compared with the previous frame, it can be regarded as a key frame. In this way, the sampling strategy can be dynamically adjusted according to the actual scene, so as to minimize redundant data while ensuring information integrity. Next, after the selection of key frames is completed, the key frames are arranged in chronological order to form a time queue of thermal infrared tracking key frames. The construction of the time queue not only involves the simple sorting of key frames, but also involves the management of their timestamp information. The timestamp of each frame of image records the specific time of its collection, which provides an important reference for subsequent time series analysis.

[0031] Specifically, in the step S122, each thermal infrared tracking key frame in the time queue of the thermal infrared tracking key frames is subjected to image segmentation based on an Otsu threshold segmentation algorithm to obtain a time queue of a set of suspected high-temperature regions. It should be understood that the Otsu threshold segmentation algorithm divides an image into foreground (target region) and background by automatically calculating the optimal segmentation threshold of the image gray histogram. In the technical solution of the present application, the temperature distribution in the thermal infrared image is usually represented by different gray values, and the Otsu threshold segmentation algorithm can automatically determine the segmentation threshold according to the image gray histogram to distinguish the high-temperature region from the low-temperature region, solving the failure problem of the traditional fixed threshold method under the day-night temperature difference and seasonal temperature fluctuation (such as false alarm caused by high temperature in summer). Specifically, for each image in the time queue of the thermal infrared tracking key frames, the Otsu algorithm needs to be applied separately for segmentation processing. This frame-by-frame processing method can ensure that each image can find the optimal segmentation threshold according to its own gray distribution characteristics, thereby adapting to temperature changes in different scenarios. The core of the Otsu algorithm is to calculate the maximum inter-class variance of the image gray histogram. By statistically analyzing the gray values of all pixels in the image, a histogram describing the pixel distribution can be obtained. Then, the algorithm tries to divide the pixels into two categories: one is the background pixel below a certain threshold, and the other is the target pixel above the threshold. By continuously adjusting the threshold and calculating the variance between the two categories of pixels, the threshold that maximizes the inter-class variance is finally found as the best segmentation point. It is worth noting that in the thermal infrared image, since the high-temperature region corresponds to a higher gray value, the target region after segmentation represents the high-temperature part of the image. Then, after completing the image segmentation, the segmentation result is converted into a set of suspected high-temperature regions, and arranged in time sequence to form a time queue of the set of suspected high-temperature regions.

[0032] Specifically, in the step S123, the time queue of the set of suspected high-temperature regions is data classified based on the position of the suspected high-temperature region in the thermal infrared tracking key frame to obtain the set of time queues of the suspected high-temperature regions. It should be understood that the position information generally includes the center coordinates, boundary range, and spatial relationship with other regions, etc. of the region. By classifying the suspected high-temperature regions at different times into a time sequence through the position information, the temperature evolution pattern of different region positions over time (such as the temperature of the fire source rising over time and the periodic fluctuation of the high-temperature region of the machine) can be analyzed, so as to exclude temporary high-temperature interference (such as welding sparks), and only keep the regions that exist continuously or spread as candidate fire sources, to provide time sequence data support for subsequent fire source region identification and fire source movement path prediction (such as the spread of the flame along the ventilation duct) and tracking. Specifically, in the technical solution of the present application, first, the suspected high-temperature regions in each key frame are preliminarily marked and numbered. Since the high-temperature regions in each key frame can exist independently or can be associated with the high-temperature regions in other frames, a unique identifier needs to be assigned to record the basic attributes (such as area, temperature range, etc.) of the region and its position information in the current frame. Next, the marked high-temperature regions are matched across frames. In order to accurately match the same high-temperature source in different frames, the geometric distance between the high-temperature regions in adjacent frames can be calculated to determine whether they belong to the same source; or the shape similarity analysis method can be used to compare the contour features of the high-temperature regions in different frames to determine their consistency. Finally, the regions belonging to the same high-temperature source are grouped into a group, and the set of time queues of suspected high-temperature regions is formed.

[0033] In the intelligent dynamic tracking method based on the infrared thermal imaging identification technology described above, the step S130 uses a deep learning model to classify each time queue of suspected high-temperature regions in the set of time queues of suspected high-temperature regions to determine whether each suspected high-temperature region is a fire source region or a non-fire source region. It should be understood that in order to analyze the temperature distribution space-time features of each suspected high-temperature region to determine whether each suspected high-temperature region is a fire source region, a deep learning model is used in the technical solution of the present application to classify each time queue of suspected high-temperature regions in the set of time queues of suspected high-temperature regions to determine whether each suspected high-temperature region is a fire source region or a non-fire source region. Specifically, the unique spatial features of the flame (such as edge fractal dimension, temperature gradient directionality) are extracted through a convolutional neural network model to distinguish the fire source from the high-temperature equipment (such as the high-temperature region of the boiler surface). In addition, the RNN model is used to model the temperature fluctuation characteristics to identify the diffusion mode of the fire source (such as exponential temperature rise) and the stable / periodic temperature change of the non-fire source (such as the start and stop of the motor). In this way, the fire source region or the non-fire source region can be effectively distinguished to provide a basis for subsequent intelligent planning of the movement path of the infrared thermal imaging camera and tracking of the ignition point.

[0034] Figure 3 A flowchart of a process of using a deep learning model to classify each time queue of suspected high-temperature regions in the set of time queues of suspected high-temperature regions to determine whether each suspected high-temperature region is a fire source region or a non-fire source region in the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application. As shown in the figure, in the embodiment of the present application, the step S130 includes: S131, performing temperature distribution space feature extraction and key clue feature filtering on the set of time queues of suspected high-temperature regions to obtain a time queue of suspected high-temperature region temperature distribution features; and S132, performing high-temperature region classification based on temperature time series fluctuation feature analysis on the time queue of suspected high-temperature region temperature distribution features to determine whether a suspected high-temperature region is a fire source region or a non-fire source region. Figure 3

[0035] A flowchart of a process of performing temperature distribution space feature extraction and key clue feature filtering on the set of time queues of suspected high-temperature regions to obtain a time queue of suspected high-temperature region temperature distribution features in the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to an embodiment of the present application. As shown in the figure, in the embodiment of the present application, the step S131 of performing temperature distribution space feature extraction and key clue feature filtering on the set of time queues of suspected high-temperature regions to obtain a time queue of suspected high-temperature region temperature distribution features includes: S1311, inputting each suspected high-temperature region in the time queue of suspected high-temperature regions into a temperature distribution space feature extractor based on a convolutional neural network model to obtain a time queue of suspected high-temperature region temperature distribution feature encoding vectors; and S1312, performing hot anomaly area temperature distribution key feature filtering on the time queue of suspected high-temperature region temperature distribution feature encoding vectors to obtain a time queue of filtered suspected high-temperature region temperature distribution feature encoding vectors as the time queue of suspected high-temperature region temperature distribution features. Figure 4 Figure 4

[0036] ​​Specifically, in the step S1311, each suspected high-temperature region in the time queue of suspected high-temperature regions is input into a temperature distribution spatial feature extractor based on a convolutional neural network model to obtain a time queue of suspected high-temperature region temperature distribution feature encoding vectors. It should be understood that, considering that the temperature distribution of a high-temperature region often has a complex spatial pattern, for example, the temperature gradient of the edge of a flame is large, and the temperature distribution of a high-temperature device is generally uniform, these spatial features cannot be represented by simple temperature values or region areas. In addition, a fire source (such as a flame) usually has unique spatial features, such as irregular edges, temperature gradient directionality (outward diffusion), and local high-temperature points. The temperature distribution of a non-fire source (such as a high-temperature device) usually presents a regular shape (such as a circle or a rectangle) and a uniform gradient, and effectively extracting these spatial features helps to distinguish fire sources from non-fire sources. Based on this, in the technical solution of the present application, each suspected high-temperature region in the time queue of suspected high-temperature regions is input into a temperature distribution spatial feature extractor based on a convolutional neural network model to obtain a time queue of suspected high-temperature region temperature distribution feature encoding vectors. It is worth mentioning that by using the temperature distribution spatial feature extractor based on the convolutional neural network model, detailed feature information in the temperature distribution of the suspected high-temperature region can be captured, such as edges, textures, and gradient directions, which are crucial for distinguishing fire sources from non-fire sources. Furthermore, by using the temperature distribution spatial feature extractor based on the convolutional neural network model, multi-layer convolution and pooling operations can be used to learn from low-level features (such as temperature gradients) to high-level features (such as flame shapes) step by step, thereby comprehensively representing the temperature distribution pattern of the high-temperature region.

[0037] Specifically, in the step S1312, the time queue of the suspected high-temperature area temperature distribution feature coding vectors is filtered by a thermal anomaly area temperature distribution key feature to obtain a filtered time queue of suspected high-temperature area temperature distribution feature coding vectors as the time queue of the suspected high-temperature area temperature distribution features. It should be understood that the time queue of suspected high-temperature area temperature distribution feature coding vectors obtained by feature extraction of each suspected high-temperature area by the temperature distribution space feature extractor based on the convolutional neural network model is usually high-dimensional, which may contain a large amount of information irrelevant to fire source identification (such as background temperature, equipment thermal radiation, etc.). These redundant features will increase the computational complexity and may interfere with the judgment of the classifier. In addition, there are various thermal noise sources (such as sunlight reflection, periodic thermal fluctuations of equipment) in the industrial environment, which may be similar to the features of the fire source in the feature space, and the influence of which needs to be effectively suppressed by feature filtering. Therefore, in the technical solution of the present application, the time queue of the suspected high-temperature area temperature distribution feature coding vectors is further filtered by a thermal anomaly area temperature distribution key feature to obtain a filtered time queue of suspected high-temperature area temperature distribution feature coding vectors as the time queue of the suspected high-temperature area temperature distribution features. Feature filtering can screen out features highly related to fire source identification (such as temperature gradient of flame edge, dynamic change pattern of temperature distribution), thereby improving the accuracy and robustness of the classifier in identifying the fire source of the suspected high-temperature area. For example, in an industrial scene, the surface temperature of a high-temperature reaction kettle may be similar to that of an initial fire source, but its temperature distribution features (such as uniform gradient) are significantly different from those of a fire source (such as irregular edge, outwardly spreading temperature gradient). Through feature filtering, the unique features of the fire source can be retained while the interference features of the reaction kettle are removed. In addition, the periodic thermal fluctuations of a high-temperature furnace may cause dramatic changes in the temperature distribution features, which are easily misjudged as a fire source by traditional methods. Through feature filtering, the importance evaluation of the features can be flexibly adjusted by a dynamic gating mechanism and context coordination relationship, and the influence of these periodic noises can be suppressed, thereby adapting to the dynamic changes in complex industrial environments.

[0038] Figure 5 For the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to the embodiments of the present application, the time queue of the suspected high-temperature area temperature distribution feature coding vectors is filtered by a thermal anomaly area temperature distribution key feature to obtain a filtered time queue of suspected high-temperature area temperature distribution feature coding vectors as the time queue of the suspected high-temperature area temperature distribution features. As shown in FIG. 13B, the feature filtering process includes the following steps: Figure 5In the embodiments of the present application, as shown in the step S1312, the time queue of the suspected high-temperature area temperature distribution feature coding vectors is filtered by a thermal anomaly area temperature distribution key feature to obtain a filtered time queue of suspected high-temperature area temperature distribution feature coding vectors as the time queue of the suspected high-temperature area temperature distribution features, which includes: S1312-1, sequence reconstruction is performed on the time queue of the suspected high-temperature area temperature distribution feature coding vectors to obtain a sequence of suspected high-temperature area temperature distribution feature reconstruction feature vectors; S1312-2, an i-th suspected high-temperature area temperature distribution feature reconstruction feature vector is extracted from the sequence of suspected high-temperature area temperature distribution feature reconstruction feature vectors as a suspected high-temperature area temperature distribution feature to-be-analyzed feature vector; S1312-3, the first i-1 suspected high-temperature area temperature distribution feature reconstruction feature vectors in the sequence of suspected high-temperature area temperature distribution feature reconstruction feature vectors are input into a time sequence feature extraction network to obtain a suspected high-temperature area temperature distribution feature historical feature guide vector; S1312-4, the suspected high-temperature area temperature distribution feature historical feature guide vector and the suspected high-temperature area temperature distribution feature to-be-analyzed feature vector are input into a feature correlation gate unit to obtain a suspected high-temperature area temperature distribution feature correlation strength factor; S1312-5, based on a comparison between the suspected high-temperature area temperature distribution feature correlation strength factor and a preset threshold, it is determined whether to filter the suspected high-temperature area temperature distribution feature to-be-analyzed feature vector to obtain the filtered time queue of suspected high-temperature area temperature distribution feature coding vectors.

[0039] Specifically, in the step S1312, the thermal anomaly area temperature distribution key feature filtering is a modular network architecture for feature screening through a multi-stage processing mechanism, which enhances the context adaptability and task relevance of feature selection by constructing a closed-loop screening mechanism for fusion of saliency rough description and key clues, while effectively balancing the elimination of feature redundancy and the retention of useful information.

[0040] In the embodiments of the present application, the step S1312-1, the sequence reconstruction is performed on the time queue of the suspected high-temperature area temperature distribution feature coding vectors to obtain the sequence of suspected high-temperature area temperature distribution feature reconstruction feature vectors, which includes: S1312-11, the maximum value of each suspected high-temperature area temperature distribution feature coding vector in the time queue of the suspected high-temperature area temperature distribution feature coding vectors is extracted as a saliency rough description factor to obtain a sequence of suspected high-temperature area temperature distribution saliency feature description factors; S1312-12, based on the ascending arrangement of the sequence of suspected high-temperature area temperature distribution saliency feature description factors, the sequence reconstruction is performed on the time queue of the suspected high-temperature area temperature distribution feature coding vectors to obtain the sequence of suspected high-temperature area temperature distribution feature reconstruction feature vectors.

[0041] Specifically, in step S1312-11, the maximum value of each temperature distribution feature encoding vector of the suspected high-temperature region in the time queue is extracted as a significant coarse descriptor to obtain a sequence of significant feature descriptors of the temperature distribution of the suspected high-temperature region, which is expressed by the significant descriptor formula as follows:

[0042] ;

[0043] ;

[0044] ;

[0045] in, This is a time queue of the temperature distribution feature encoding vector of the suspected high-temperature area. , , and These are the 1st, 2nd, and 3rd times in the time queue of the temperature distribution feature encoding vector of the suspected high-temperature area, respectively. The and the first A temperature distribution feature encoding vector for a suspected high-temperature area. To perform the maximum value operation, It is a sequence of descriptive factors for significant characteristics of temperature distribution in suspected high-temperature regions. , , and These are the first, second, and third descriptive factors in the sequence of significant characteristics of temperature distribution in the suspected high-temperature region, respectively. The and the first A significant descriptive factor for the temperature distribution of suspected high-temperature areas. It should be understood that in the initial stage of data processing for filtering key features of temperature distribution in the thermal anomaly area, the maximum value of the temperature distribution feature encoding vector of each suspected high-temperature area is extracted as a coarse descriptive factor, thus performing a low-dimensional feature expression compression. This process theoretically relies on the representational role of the polarity distribution in the temperature distribution feature encoding vector of suspected high-temperature areas; capturing the maximum value directly provides a rough but discriminative feature ranking reference.

[0046] Specifically, in step S1312-12, based on the ascending order of the sequence of descriptive factors of the significant features of the temperature distribution in the suspected high-temperature region, the time queue of the temperature distribution feature encoding vector of the suspected high-temperature region is reconstructed to obtain the sequence of the reconstructed representation vector of the temperature distribution features in the suspected high-temperature region, expressed by the sequence reconstruction formula as follows:

[0047] ;

[0048] in, Based on ascending order pairs Perform sequence reconstruction. It is a sequence of characterization vectors reconstructed from the temperature distribution features of suspected high-temperature regions. , , and The first, second, and third vectors in the sequence of reconstructed representation vectors of temperature distribution characteristics in the suspected high-temperature region are respectively the first, second, and third vectors. The and the first The representation vector of temperature distribution features in suspected high-temperature areas is reconstructed. It should be understood that sequence reconstruction is not only a data reconstruction but also an injection of "distribution order semantics" into the features. By rearranging the time queue of the temperature distribution feature encoding vectors of suspected high-temperature areas according to their salience level, a good information flow structure can be naturally introduced. Features with low salience are placed first, reducing unnecessary high-weight allocation, while highly salience features are pushed to the later part of the sequence, thus having a more significant impact on subsequent screening mechanisms. It is worth mentioning that this sorting also enhances the contextual consistency of features, allowing them to better cooperate with sequence modeling in subsequent capture modules. Specifically, in fire source identification scenarios, flame areas typically exhibit high temperature peaks, and therefore their corresponding salience feature descriptor values ​​are also high. By arranging in ascending order, the relevant features of flame areas are automatically pushed to the later part of the sequence, thus gaining higher attention in subsequent analysis.

[0049] Specifically, in step S1312-2, the i-th reconstructed representation vector of the temperature distribution features of the suspected high-temperature region is extracted from the sequence of reconstructed representation vectors of the temperature distribution features of the suspected high-temperature region as the representation vector to be analyzed. It should be understood that after the sequence reconstruction is completed, each reconstructed representation vector of the temperature distribution features of the suspected high-temperature region has been assigned a specific sequential number. These numbers reflect the relative importance of the feature vectors at the saliency level. Therefore, by extracting the i-th reconstructed representation vector of the temperature distribution features of the suspected high-temperature region, the system can focus on the target feature that most needs to be analyzed, thereby improving computational efficiency and reducing interference from redundant information. Wherein, the i-th... The reconstructed representation vector of temperature distribution characteristics in a suspected high-temperature region is extracted as the representation vector to be analyzed for the temperature distribution characteristics of the suspected high-temperature region. Here, the first... The reconstructed representation vector of temperature distribution characteristics in suspected high-temperature areas is merely an illustrative method and does not refer to a specific feature vector. Rather, it is an index value dynamically determined based on actual task requirements.

[0050] In the embodiments of this application, step S1312-3, which involves inputting the first i-1 suspected high-temperature region temperature distribution feature reconstruction representation vectors from the sequence of suspected high-temperature region temperature distribution feature reconstruction representation vectors into a temporal feature extraction network to obtain a historical feature guidance vector for suspected high-temperature region temperature distribution features, includes: calculating the significance distribution representation value of each suspected high-temperature region temperature distribution feature reconstruction representation vector in the first i-1 suspected high-temperature region temperature distribution feature reconstruction representation vectors to obtain the significance distribution representation value of the first i-1 suspected high-temperature region temperature distribution; performing weighted standardization processing on each suspected high-temperature region temperature distribution significance distribution representation value in the first i-1 suspected high-temperature region temperature distribution significance distribution representation values ​​to obtain the first i-1 standardized significance distribution representation values ​​of the suspected high-temperature region temperature distribution; and using the first i-1 standardized significance distribution representation values ​​of the suspected high-temperature region temperature distribution as a sequence of weight values, calculating the weighted sum of the first i-1 suspected high-temperature region temperature distribution feature reconstruction representation vectors to obtain the historical feature guidance vector for suspected high-temperature region temperature distribution features.

[0051] Specifically, step S1312-3 is expressed as a feature filtering formula as follows:

[0052] ;

[0053] in, To capture key clues for reconstructing the representation vector of temperature distribution characteristics in the first i-1 suspected high-temperature regions, The first i-1 suspected high-temperature regions are used to reconstruct the representation vector of temperature distribution characteristics. Reconstructing the representation vector of temperature distribution characteristics in suspected high-temperature areas. and They are respectively The mean and variance, It is an edge value. It is the th value among the significance distribution representation values ​​of the temperature distribution of the first i-1 suspected high-temperature areas. The significance of temperature distribution in suspected high-temperature areas is represented by the following values. It is the i-1th value among the standardized values ​​of the temperature distribution significance of suspected high-temperature regions. The significance distribution of temperature distribution in a standardized suspected high-temperature region is represented by the following values. This is a historical feature guide vector indicating the temperature distribution characteristics of suspected high-temperature areas. It can be understood that, to enrich the contextual information of the features to be analyzed, the network captures the first few elements of the sequence of reconstructed representation vectors of temperature distribution characteristics in suspected high-temperature areas. - 1 distribution clue of the feature reconstruction vector of the temperature distribution characteristics of the suspected high temperature area. This operation is realized by the time sequence feature extraction network, and the main task is to infer and extract the hidden context mode in the previous sequence characteristics. The capture network is essentially a global modeling module, which can extract potential semantics from the significance distribution mode of the input sequence. By capturing these clues, the network can more comprehensively understand the context environment in which the current feature to be analyzed is located, thereby providing more rich background support. In technical implementation, the time sequence feature extraction network calculates the significance distribution representation value of the feature reconstruction vector of the temperature distribution characteristics of the suspected high temperature area in the previous i−1 suspected high temperature area, to quantify the importance and contribution of each feature vector. Then, by weight standardization processing to these significance distribution representation values, the attention to core features is further enhanced, while the influence of noise or redundant information is weakened. Finally, based on the weighted sum operation of the standardized weight value sequence, a highly abstracted suspected high temperature area temperature distribution feature history feature guide vector is generated, which condenses the important key information in the context environment and provides an important reference for subsequent feature fusion.

[0054] It is worth mentioning that in the embodiments of the present application, the saliency distribution representation value of the suspected high-temperature region temperature distribution feature reconstruction feature vector is related to the maximum value, the mean value, the variance and the edge value of the suspected high-temperature region temperature distribution feature reconstruction feature vector, wherein the edge value is used to measure the edge consistency of the suspected high-temperature region temperature distribution and improve the evaluation of the overall self-consistency contribution. The role of the edge value is to capture the extreme characteristics in the data distribution, which may reflect the key information of the high-temperature region, such as the rapid spread of the flame or the abnormal fluctuation of the local high-temperature point. By introducing the edge value, the system can pay more attention to these extreme cases in the analysis process, thereby improving the sensitivity and accuracy of the fire source identification. Specifically, the time sequence feature extraction network uses the edge saliency factor of the extreme value of the feature relative to the overall self-consistent trend, the edge-global consistency factor of the volatility of the feature and the overall self-consistent level, and the edge correlation factor of the fluctuation range of the feature based on the overall self-consistent relationship to measure the global saliency self-consistent edge synergy, that is, to perform edge saliency-based synergy of the global self-consistent relationship, so as to model the consistency and synergy within the global range based on the edge distribution in the saliency distribution pattern of the input model. By combining local edge characteristics and global distribution characteristics, more comprehensive data understanding can be achieved. The edge saliency factor is used to evaluate the degree of deviation of the extreme value from the overall trend, the edge-global consistency factor of the volatility and the overall self-consistent level reflects the coordination degree of local fluctuation and global stability, and the edge correlation factor of the fluctuation range based on the overall self-consistent relationship further quantifies the boundary behavior of the data distribution. Through the comprehensive analysis of these indicators, the system can more accurately capture the dynamic change law of the high-temperature region. In this way, while ensuring that the extreme edge data distribution is consistent with the overall global self-consistent trend, it also ensures that the data fluctuation based on the edge distribution also has self-consistency with the global whole. Therefore, the distribution key clue vector can essentially be a feature with edge fluctuation characteristics in the context saliency, to improve the evaluation of its overall self-consistency contribution through edge-global self-consistent re-inference, thereby providing a reference index for global self-consistent fusion based on edge saliency synergy for potential pattern of sequence distribution.

[0055] Specifically, in the step S1312-4, the suspected high-temperature region temperature distribution feature historical feature clue vector and the suspected high-temperature region temperature distribution feature to be analyzed feature vector are input into the feature association gate unit to obtain the suspected high-temperature region temperature distribution feature association strength factor, which is represented by a clue weaving gate formula as follows:

[0056] ;

[0057] wherein, is a transpose operation, is and inverse of the covariance matrix of the, is an absolute value operation, is corresponding suspicious high-temperature region temperature distribution feature correlation strength factor. It should be understood that after the key clue extraction is completed, the suspicious high-temperature region temperature distribution feature historical feature guide vector and the suspicious high-temperature region temperature distribution feature to-be-analyzed feature vector are sent into the feature correlation gating unit together. The role of this gating network is to fuse the two input information streams and dynamically evaluate their importance. This fusion process is not a simple superposition or splicing, but a deep information interaction through a complex computer mechanism. For example, the network will adjust the weight distribution of the suspicious high-temperature region temperature distribution feature to-be-analyzed feature vector according to the global context information in the suspicious high-temperature region temperature distribution feature historical feature guide vector, so that those features that are crucial to the task can get more attention. In actual operation, these control gates will dynamically adjust the degree of opening or closing according to the characteristics of the input data, so as to achieve fine modeling of the feature context relationship. For example, when the feature is highly related to the context information, the corresponding control gate will be completely opened, allowing the feature to fully participate in subsequent calculations; when the feature is considered to be noise or redundant information, the corresponding control gate will be partially or completely closed to reduce its influence on the overall result. In this way, the gating mechanism forms a deep information fusion between feature contexts through a design similar to a gated recurrent unit (GRU), while suppressing the interference of noise features. The output of the gating network is the suspicious high-temperature region temperature distribution feature correlation strength factor, which, as a comprehensive index, quantifies the consistency, saliency and task relevance between the to-be-analyzed feature and its context. The generation of the suspicious high-temperature region temperature distribution feature correlation strength factor not only depends on the direct evaluation of saliency, but also incorporates the distribution synergy relationship between features, making this index more global.

[0058] Specifically, in the step S1312-5, based on a comparison between the suspicious high-temperature region temperature distribution feature correlation strength factor and a preset threshold, it is determined whether to filter the suspicious high-temperature region temperature distribution feature to-be-analyzed feature vector to obtain the filtered time queue of the suspicious high-temperature region temperature distribution feature encoding vector, which is expressed by a filtering formula as follows:

[0059] ;

[0060] wherein, is a feature filtering operation on ​is a preset threshold. It should be understood that the suspected high-temperature area temperature distribution feature correlation strength factor is compared with a preset threshold to achieve the final feature screening. The principle behind the filtering process is that the network can dynamically adjust the criteria for judging the importance of features through contextual support and local saliency analysis. By setting a threshold, the system can flexibly reduce features according to actual task requirements (for example, focusing on precision or complexity). For example, in the case of focusing on precision, the threshold can be appropriately increased to retain more high-quality features; while in the case of focusing on complexity optimization, the threshold can be reduced to reduce the computational burden. This flexibility enables the system to adapt to the needs of different application scenarios and find the best balance between performance and resource consumption. This screening result will directly determine the quality and availability of the final retained feature set.

[0061] It is worth mentioning that the heat anomaly area temperature distribution key feature filtering has the following technical advantages:

[0062] First, there are often a large number of redundant features in high-dimensional data, and these unhelpful or even interfering features can cause the computational cost to increase dramatically during model training, and the implicit correlation may affect the generalization ability of the model. Traditional methods rely on manual selection of features or some coarse-grained dimensionality reduction methods, which are difficult to carefully explore the relationship between redundant features and core features. The heat anomaly area temperature distribution key feature filtering introduces saliency description, sequence reconstruction and dynamic gating mechanism, which progressively filters features from two levels of saliency level and contextual synergy relationship, thereby eliminating unnecessary redundant features while avoiding rough deletion of useful features, optimizing the precision and efficiency of feature selection.

[0063] Second, feature selection methods usually only consider the saliency of isolated features, ignoring the hidden distribution correlation between features and their context features. This fragmented processing approach can lead to important non-salient features being mistakenly discarded or joint features that implicitly contain important information not being correctly captured. By introducing the key clue capturing module, the network not only analyzes the saliency of the current feature to be analyzed, but also uses the suspected high-temperature area temperature distribution feature history feature guide vector generated by the context distribution as a global constraint to establish a dynamic association between the feature to be analyzed and its context, thereby making up for the lack of context in local selection.

[0064] Moreover, for dynamic distribution features (e.g., time series data, etc.), the traditional method using static screening rules often cannot adjust the distribution changes and task dependence of the features in real time, resulting in unstable screening effect. The closed-loop framework of distribution key clue weaving flexibly adjusts the importance evaluation of the current features through the correlation strength factor of the temperature distribution features of the suspected high-temperature area, thereby realizing high adaptability screening for dynamic distribution feature environment. In addition, the dynamic comparison mechanism of the correlation strength factor of the temperature distribution features of the suspected high-temperature area and the threshold further reflects the balance ability of the network between reducing noise and retaining task key information.

[0065] In the embodiment of the present application, the step S132, high-temperature area classification based on temperature time sequence fluctuation feature analysis is performed on the time queue of the suspected high-temperature area temperature distribution features to determine whether the suspected high-temperature area is a fire source area or a non-fire source area, comprising: S1321, inputting the time queue of the filtered suspected high-temperature area temperature distribution feature encoding vector into an RNN model-based suspected high-temperature area temperature time sequence fluctuation feature extractor to obtain a suspected high-temperature area temperature time sequence fluctuation feature encoding vector; S1322, inputting the suspected high-temperature area temperature time sequence fluctuation feature encoding vector into a high-temperature area recognition module based on a classifier to obtain a classification result, wherein the classification result is whether the suspected high-temperature area is a fire source area or a non-fire source area.

[0066] Specifically, in the step S1321, the time queue of the filtered suspected high-temperature region temperature distribution feature encoding vector is input into the suspected high-temperature region temperature time sequence fluctuation feature extractor based on the RNN model to obtain a suspected high-temperature region temperature time sequence fluctuation feature encoding vector. It can be understood that, considering that the temperature distribution of a fire source (such as a flame) usually has significant time sequence dynamic change features, for example, rapid temperature rise over time, flame edge outward diffusion, etc. These dynamic features are key to distinguishing fire sources from static high-temperature regions (such as high-temperature equipment). The temperature distribution of a non-fire source region (such as a high-temperature machine or sunlight reflection) usually presents periodicity or stability, and the temperature fluctuation mode thereof is significantly different from that of a fire source. However, a single frame of image or static feature cannot reflect the dynamic process of temperature change, and therefore, time sequence modeling needs to be performed on the suspected high-temperature region temperature distribution features to capture the time sequence evolution law of the temperature distribution, so as to more accurately identify a fire source. Based on this, in the technical solution of the present application, the time queue of the filtered suspected high-temperature region temperature distribution feature encoding vector is further input into the suspected high-temperature region temperature time sequence fluctuation feature extractor based on the RNN model to obtain a suspected high-temperature region temperature time sequence fluctuation feature encoding vector. Through the suspected high-temperature region temperature time sequence fluctuation feature extractor based on the RNN model, the dependence relationship and dynamic change information of the suspected high-temperature region temperature distribution features in the time dimension can be captured. It is worth mentioning that the RNN can adapt to temperature fluctuations of different time scales, for example, rapid flame diffusion (on a second level) and slow heat fluctuation of high-temperature equipment (on a minute level), and the obtained suspected high-temperature region temperature time sequence fluctuation feature encoding vector is an abstract representation of the high-temperature region temperature change mode, which can reflect the dynamic characteristics (such as rise rate, fluctuation amplitude, diffusion direction, etc.) of the temperature distribution, and can be used to predict the diffusion track of a fire source, thereby providing decision support for subsequent camera tracking and fire extinguishing systems.

[0067] Specifically, in the step S1322, the suspected high-temperature region temperature time sequence fluctuation feature coding vector is input into the high-temperature region recognition module based on the classifier to obtain a classification result, which is whether the suspected high-temperature region is a fire source region or a non-fire source region. That is, the temperature time sequence fluctuation feature of the suspected high-temperature region is used for classification processing of the high-temperature region, so as to determine whether the suspected high-temperature region is a fire source region or a non-fire source region. In this way, it is helpful to calculate the optimal moving path of the infrared thermal imaging camera and control the infrared thermal imaging camera to continuously track the fire source. Specifically, by inputting the suspected high-temperature region temperature time sequence fluctuation feature coding vector into the high-temperature region recognition module based on the classifier, the system can accurately distinguish different types of high-temperature regions with the help of machine learning algorithms. Common classifiers include support vector machine (SVM), random forest (Random Forest) and deep neural network (DNN), which can automatically learn and extract key features according to known labels in training data, so as to realize efficient classification task. If a region is determined to be a fire source, it needs to be ensured that it is always located at the center of the camera field of view; if it is a non-fire source region, the attention to it can be appropriately reduced, thereby saving computing resources and improving overall efficiency.

[0068] In the technical solution of the present application, each suspected high-temperature region temperature distribution feature coding vector in the time queue of the filtered suspected high-temperature region temperature distribution feature coding vector represents the temperature distribution space feature of each suspected high-temperature region. Since the time queue of the filtered suspected high-temperature region temperature distribution feature coding vector is obtained by feature screening on the time queue of the suspected high-temperature region temperature distribution feature coding vector, compared with the time queue of the suspected high-temperature region temperature distribution feature coding vector, the time queue of the filtered suspected high-temperature region temperature distribution feature coding vector has a feature sparsity characteristic distribution. Therefore, when the suspected high-temperature region temperature time sequence fluctuation feature is extracted based on the RNN model, the sparsity of the feature in the sample dimension will lead to insufficient sequence coding, so that the suspected high-temperature region temperature time sequence fluctuation feature coding vector obtained may be unbalanced in the correlation coupling degree between local sequences at the feature grain level, affecting the mapping accuracy when mapping to the target classification domain.

[0069] Based on this, in an optimized embodiment of the present application, in the process of inputting the suspected high-temperature region temperature time sequence fluctuation feature coding vector into the high-temperature region recognition module based on the classifier to obtain a classification result, the suspected high-temperature region temperature time sequence fluctuation feature coding vector is enhanced in feature grain level correlation, and the process includes:

[0070] Calculate the first-order difference feature between any two positions of the feature value in the temperature time-series fluctuation feature encoding vector of the suspected high-temperature region to obtain a fine-grained first-order difference distribution map of the temperature time-series fluctuation feature of the suspected high-temperature region, as shown below:

[0071] ;

[0072] ;

[0073] in, This represents the encoding vector of the temporal temperature fluctuation characteristics of the suspected high-temperature region. and These respectively represent the first element in the temperature time-series fluctuation feature encoding vector of the suspected high-temperature region. The eigenvalue of the and the eigenvalue of the . Feature values ​​at each position, The fine-grained first-order difference distribution map represents the temporal fluctuation characteristics of temperature in suspected high-temperature regions. The value of the position.

[0074] Calculate the second-order difference feature between any two positions of the feature values ​​in the temperature time-series fluctuation feature encoding vector of the suspected high-temperature region to obtain a fine-grained second-order difference distribution map of the temperature time-series fluctuation feature of the suspected high-temperature region, as shown below:

[0075] ;

[0076] ;

[0077] in, The fine-grained second-order difference distribution map represents the temporal fluctuation characteristics of temperature in suspected high-temperature regions. The value of the position.

[0078] Based on the spatial structure summary information of the fine-grained first-order difference distribution map and the fine-grained second-order difference distribution map of the temperature time-series fluctuation characteristics of the suspected high-temperature region, a cross-scale difference distribution topology map of the temperature time-series fluctuation characteristics of the suspected high-temperature region is constructed between the two maps, as shown below:

[0079] ;

[0080] in, This is a fine-grained first-order difference distribution map showing the temporal fluctuation characteristics of temperature in suspected high-temperature regions. This is a fine-grained second-order difference distribution map showing the temporal fluctuation characteristics of temperature in suspected high-temperature regions. Denotes the F-norm, A cross-scale difference distribution topological atlas of the suspected high-temperature region temperature time sequence fluctuation characteristics is represented.

[0081] A fine-grained self-association expression of the suspected high-temperature region temperature time sequence fluctuation characteristic encoding vector is constructed to obtain a suspected high-temperature region temperature time sequence fluctuation characteristic particle-level self-correlation representation matrix, represented as:

[0082] ;

[0083] Wherein, represents matrix multiplication, represents the transpose of a vector, represents a suspected high-temperature region temperature time sequence fluctuation characteristic particle-level self-correlation representation matrix.

[0084] Based on the cross-scale difference distribution topological atlas of the suspected high-temperature region temperature time sequence fluctuation characteristics, a cross-scale mapping model is constructed for the suspected high-temperature region temperature time sequence fluctuation characteristic particle-level self-correlation representation matrix and the suspected high-temperature region temperature time sequence fluctuation characteristic encoding vector to obtain a suspected high-temperature region temperature time sequence fluctuation characteristic cross-scale excitation response encoding vector and a suspected high-temperature region temperature time sequence fluctuation characteristic self-correlation clustering response excitation encoding matrix, represented as:

[0085] ;

[0086] ;

[0087] Wherein, represents a suspected high-temperature region temperature time sequence fluctuation characteristic self-correlation clustering response excitation encoding matrix, represents a suspected high-temperature region temperature time sequence fluctuation characteristic cross-scale excitation response encoding vector.

[0088] The suspected high-temperature region temperature time sequence fluctuation fine-grained coupling constraint vector is extracted from the suspected high-temperature region temperature time sequence fluctuation characteristic cross-scale excitation response encoding vector, taking the suspected high-temperature region temperature time sequence fluctuation characteristic self-correlation clustering response excitation encoding matrix as a dynamic feedback tuning parameter, represented as:

[0089] ;

[0090] Wherein, represents a standardization exponential function, represents a suspected high-temperature region temperature time sequence fluctuation fine-grained coupling constraint vector.

[0091] The position-wise dot product between the suspected high-temperature area temperature time sequence fluctuation fine-grained coupling constraint vector and the suspected high-temperature area temperature time sequence fluctuation feature coding vector is calculated to obtain an optimized suspected high-temperature area temperature time sequence fluctuation feature coding vector, denoted as:

[0092] ;

[0093] wherein, denotes the position-wise dot product, denotes the optimized suspected high-temperature area temperature time sequence fluctuation feature coding vector.

[0094] In this way, for the suspected high-temperature area temperature time sequence fluctuation feature coding vector, a cross-scale difference distribution topology map is generated by different order difference information metrics, and then the suspected high-temperature area temperature time sequence fluctuation feature coding vector and its autocorrelation representation are respectively subjected to mapping modeling of cross-modal inter-node interaction, so as to simulate the component activation function of the correlation system based on sparse difference response. The discrete distribution of feature clustering is modulated by a dynamic feedback tuning parameter, and through the cross-modal mapping feature expression of the cross-scale difference distribution topology map, the constraint mechanism of coupling discreteness in each dimension is realized, so as to avoid the negative impact of correlation vulnerability caused by discrete distribution on the classification reasoning efficiency.

[0095] In the above intelligent dynamic tracking method based on infrared thermal imaging recognition technology, the step S140 calculates the optimal moving path of the infrared thermal imaging camera based on the time queue of the fire source area to track the ignition point through the infrared thermal imaging camera. It should be understood that the time queue of the fire source area records the position and shape information of the fire source at different time points. By analyzing the fire source area in the time queue, the trend of its change over time can be extracted, such as the speed, direction, and shape evolution of flame spread. These features not only reflect the dynamic behavior of the fire source itself, but also provide important clues for predicting its future possible position. Then, according to the physical characteristics and working environment of the infrared thermal imaging camera, its moving path is planned, which needs to consider many factors such as the field of view range of the camera, the focal length adjustment ability, the speed of the pan-tilt rotation, and the complexity of the target area. For example, if the fire source is located at the edge of the camera's field of view, the angle or position of the camera needs to be adjusted to re-align the center of the fire source.

[0096] Specifically, in one specific example of the present application, the Kalman filter or particle swarm algorithm is used to predict the fire source spread direction, and the gimbal is controlled to adjust the viewing angle of the infrared thermal imaging camera to ensure that the fire source is always located in the center of the screen. Kalman filter is a recursive optimization algorithm that combines prior knowledge and measurement data through prediction and update steps to effectively estimate the state value of the system and reduce uncertainty. Particle swarm algorithm is an optimization method based on swarm intelligence, which iteratively updates the position and speed of individual particles to search for the optimal solution in the solution space.

[0097] In summary, the intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to the embodiments of the present application is illustrated, which collects thermal infrared video stream of the target area, applies image processing algorithm to segment the image, extracts high temperature area, and sets dynamic temperature threshold to preliminarily screen out possible high temperature target area. Then, an artificial intelligence and deep learning model is used to classify the suspected high temperature area to distinguish between fire source and non-fire source area. Finally, based on the real-time position and motion trajectory of the fire source, the optimal moving path of the infrared thermal imaging camera is calculated to control the infrared thermal imaging camera to continuously track the fire source. In this way, the time correlation of thermal image video stream can be utilized to adapt to the overlapping challenge of high temperature equipment and initial fire in industrial scenarios, and the accuracy and efficiency of fire source identification and positioning can be improved.

[0098] Figure 6 A block diagram of the intelligent dynamic tracking device based on infrared thermal imaging recognition technology according to the embodiments of the present application is shown. As shown in Figure 6 The intelligent dynamic tracking device based on infrared thermal imaging recognition technology according to the embodiments of the present application 100 includes: a thermal infrared tracking video stream acquisition module 110 for acquiring thermal infrared tracking video stream of a target area collected by an infrared thermal imaging camera; a thermal infrared tracking video stream preprocessing module 120 for performing key frame discrete sampling on the thermal infrared tracking video stream, and then performing preprocessing on the time queue of the obtained thermal infrared tracking key frame based on image segmentation and data classification to obtain a set of time queues of suspected high temperature areas; a fire source area determination module 130 for using a deep learning model to classify each time queue of suspected high temperature area in the set of time queues of suspected high temperature area to determine whether each suspected high temperature area is a fire source area or a non-fire source area; and a fire point tracking module 140 for calculating the optimal moving path of the infrared thermal imaging camera based on the time queue of the fire source area to track the fire point through the infrared thermal imaging camera.

[0099] Here, those skilled in the art can understand that the specific operations of each step in the above intelligent dynamic tracking device based on infrared thermal imaging recognition technology have been described above with reference to Figures 1 to 5The description of the intelligent dynamic tracking method based on infrared thermal imaging recognition technology is described in detail in the foregoing description, and thus repeated description will be omitted.

[0100] As described above, the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology according to the embodiments of the present application can be implemented in various terminal devices. In one example, the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology can be integrated into a terminal device as a software module and / or a hardware module. For example, the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology can be a software module in the operating system of the terminal device, or can be an application program developed for the terminal device. Of course, the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology can also be one of the many hardware modules of the terminal device.

[0101] Alternatively, in another example, the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology and the terminal device can also be separate devices, and the intelligent dynamic tracking device 100 based on infrared thermal imaging recognition technology can be connected to the terminal device through a wired and / or wireless network, and transmit interactive information in a conventional data format.

[0102] In summary, the intelligent dynamic tracking device based on infrared thermal imaging recognition technology according to the embodiments of the present application is illustrated, which acquires thermal infrared video stream of the target area, applies image processing algorithm to segment the image, extracts high temperature area, and sets dynamic temperature threshold to preliminarily screen out possible high temperature target area. Then, the artificial intelligence and deep learning model is used to classify the suspected high temperature area to distinguish the fire source and non-fire source area. Finally, based on the real-time position and motion trajectory of the fire source, the best moving path of the infrared thermal imaging camera is calculated to control the infrared thermal imaging camera to continuously track the fire source. In this way, the time sequence correlation of thermal image video stream can be utilized to adapt to the overlapping challenge of high temperature equipment and initial fire in industrial scene, and the accuracy and efficiency of fire source recognition and positioning are improved.

Claims

1. An intelligent dynamic tracking method based on infrared thermal imaging recognition technology, characterized in that, include: Acquire thermal infrared tracking video streams of the target area captured by an infrared thermal imaging camera; After performing keyframe discrete sampling on the thermal infrared tracking video stream, the time queue of the obtained thermal infrared tracking keyframes is preprocessed based on image segmentation and data classification to obtain a set of time queues of suspected high-temperature areas. Using a deep learning model, the time queues of each suspected high-temperature region in the set of time queues are classified to determine whether each suspected high-temperature region is a fire source region or a non-fire source region. This includes: extracting spatial features of temperature distribution and filtering key clue features from the set of time queues of suspected high-temperature regions to obtain time queues of temperature distribution features of suspected high-temperature regions; classifying the time queues of temperature distribution features of suspected high-temperature regions based on temperature time-series fluctuation feature analysis to determine whether the suspected high-temperature regions are fire source regions or non-fire source regions. Based on the time queue of the fire source area, the optimal moving path of the infrared thermal imaging camera is calculated to track the fire point through the infrared thermal imaging camera. Specifically, the time queues of the suspected high-temperature areas are subjected to spatial feature extraction of temperature distribution and key clue feature filtering to obtain time queues of temperature distribution features of the suspected high-temperature areas, including: Each suspected high-temperature region in the time queue of the suspected high-temperature region is input into a temperature distribution spatial feature extractor based on a convolutional neural network model to obtain a time queue of temperature distribution feature encoding vectors for the suspected high-temperature regions. The process of filtering the time queue of temperature distribution feature encoding vectors for suspected high-temperature areas using key features of thermal anomaly temperature distribution to obtain a filtered time queue of temperature distribution feature encoding vectors for suspected high-temperature areas, and using this time queue as the time queue of temperature distribution features for suspected high-temperature areas, includes: reconstructing the time queue of temperature distribution feature encoding vectors for suspected high-temperature areas to obtain a sequence of reconstructed representation vectors for temperature distribution features of suspected high-temperature areas; extracting the i-th reconstructed representation vector of temperature distribution features of suspected high-temperature areas from the sequence of reconstructed representation vectors for temperature distribution features of suspected high-temperature areas as the representation vector to be analyzed for temperature distribution features of suspected high-temperature areas; and reconstructing the temperature distribution features of suspected high-temperature areas... The first i-1 suspected high-temperature region temperature distribution feature reconstruction vectors in the sequence of representation vectors are input into a temporal feature extraction network to obtain a historical feature guidance vector for the temperature distribution features of suspected high-temperature regions. The historical feature guidance vector and the suspected high-temperature region temperature distribution feature to be analyzed representation vectors are input into a feature association gating unit to obtain a correlation strength factor for the temperature distribution features of suspected high-temperature regions. Based on the comparison between the correlation strength factor for the temperature distribution features of suspected high-temperature regions and a preset threshold, it is determined whether to filter the suspected high-temperature region temperature distribution feature to be analyzed representation vectors to obtain a time queue of the filtered suspected high-temperature region temperature distribution feature encoding vectors.

2. The intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to claim 1, characterized in that, After performing keyframe discrete sampling on the thermal infrared tracking video stream, the resulting time queue of thermal infrared tracking keyframes is preprocessed based on image segmentation and data classification to obtain a set of time queues for suspected high-temperature areas, including: The thermal infrared tracking video stream is discretely sampled for keyframes to obtain a time queue of thermal infrared tracking keyframes. The Otsu threshold segmentation algorithm is used to segment each thermal infrared tracking key frame in the time queue of the thermal infrared tracking key frame to obtain a time queue of suspected high temperature regions. Based on the position of the suspected high-temperature region in the thermal infrared tracking keyframe, the time queues of the suspected high-temperature region set are classified to obtain the set of time queues of the suspected high-temperature region.

3. The intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to claim 2, characterized in that, The time sequence of the temperature distribution feature encoding vector of the suspected high-temperature region is reconstructed to obtain a sequence of reconstructed representation vectors of the temperature distribution features of the suspected high-temperature region, including: The maximum value of each temperature distribution feature encoding vector of the suspected high-temperature region in the time queue is extracted as a significant coarse descriptor to obtain a sequence of significant feature descriptors of the temperature distribution of the suspected high-temperature region. Based on the ascending order of the sequence of descriptive factors of the significant features of the temperature distribution in the suspected high-temperature region, the time sequence of the encoding vector of the temperature distribution features in the suspected high-temperature region is reconstructed to obtain the sequence of the reconstructed representation vector of the temperature distribution features in the suspected high-temperature region.

4. The intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to claim 3, characterized in that, The first i-1 reconstructed representation vectors of temperature distribution features of the suspected high-temperature areas are input into a temporal feature extraction network to obtain historical feature guidance vectors of temperature distribution features of the suspected high-temperature areas, including: Calculate the significance distribution representation value of each suspected high temperature region temperature distribution feature reconstruction representation vector in the first i-1 suspected high temperature region temperature distribution feature reconstruction representation vector to obtain the significance distribution representation value of the temperature distribution of the first i-1 suspected high temperature regions. The significance distribution values ​​of the temperature distribution of the first i-1 suspected high-temperature regions are weighted and standardized to obtain the significance distribution values ​​of the temperature distribution of the first i-1 suspected high-temperature regions. Using the saliency distribution representation values ​​of the temperature distribution of the first i-1 standardized suspected high-temperature areas as a sequence of weight values, the weighted sum of the reconstructed representation vectors of the temperature distribution features of the first i-1 suspected high-temperature areas is calculated to obtain the historical feature guidance vector of the temperature distribution features of the suspected high-temperature areas.

5. The intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to claim 4, characterized in that, The significance distribution representation value of the temperature distribution feature reconstruction representation vector of the suspected high temperature region is related to the maximum value, mean, variance and marginal value of the temperature distribution feature reconstruction representation vector of the suspected high temperature region. The marginal value is used to measure the edge coordination consistency of the temperature distribution of the suspected high temperature region and improve the evaluation of its contribution to the overall self-consistency.

6. The intelligent dynamic tracking method based on infrared thermal imaging recognition technology according to claim 5, characterized in that, The time-series data on the temperature distribution characteristics of the suspected high-temperature areas are used to classify the suspected high-temperature areas based on temperature time-series fluctuation characteristics to determine whether the suspected high-temperature areas are ignition source areas or non-ignition source areas, including: The time queue of the filtered suspected high-temperature area temperature distribution feature encoding vector is input into the suspected high-temperature area temperature time-series fluctuation feature extractor based on the RNN model to obtain the suspected high-temperature area temperature time-series fluctuation feature encoding vector. The temperature time-series fluctuation feature encoding vector of the suspected high-temperature area is input into the high-temperature area identification module based on the classifier to obtain the classification result, which is that the suspected high-temperature area is a fire source area or a non-fire source area.

7. An intelligent dynamic tracking device based on infrared thermal imaging recognition technology, used to execute the intelligent dynamic tracking method based on infrared thermal imaging recognition technology as described in claim 1, characterized in that, include: The thermal infrared tracking video stream acquisition module is used to acquire the thermal infrared tracking video stream of the target area captured by the infrared thermal imaging camera; The thermal infrared tracking video stream preprocessing module is used to perform key frame discrete sampling on the thermal infrared tracking video stream, and then perform preprocessing based on image segmentation and data classification on the time queue of the obtained thermal infrared tracking key frames to obtain a set of time queues of suspected high temperature areas. The fire source area determination module is used to classify the time queues of each suspected high temperature area in the set of time queues of the suspected high temperature area using a deep learning model to determine whether each suspected high temperature area is a fire source area or a non-fire source area. The ignition point tracking module is used to calculate the optimal moving path of the infrared thermal imaging camera based on the time queue of the fire source area in order to track the ignition point through the infrared thermal imaging camera.

Images