Hot target extraction method and system based on infrared image recognition

Abstract

The invention relates to the technical field of infrared image recognition, and discloses a hot target extraction method and system based on infrared image recognition, and the system comprises a multi-sensor collection module, a data dimension reduction module, a data verification module, a segmentation optimization module, and a dynamic feedback module. Multi-modal data dimension reduction processing is achieved by collecting infrared images, visible light images and environment dynamic parameters and combining Laplacian feature mapping, and a target area is extracted through geometric verification and topological analysis. An energy optimization model and a distribution alignment method are adopted to complete accurate segmentation of a target area, and meanwhile, segmentation model parameters are adjusted in real time based on environment dynamic parameters, so that the system stability is ensured. According to the method, high-precision and robust extraction of the thermal target is realized in a complex dynamic environment, the problems of fuzzy target boundary, noise interference and insufficient dynamic environment adaptability in the prior art are effectively solved, and the method is suitable for the fields of military reconnaissance, fireproof monitoring, electric power inspection and the like.

Description

Technical Field

[0001] The invention relates to the technical field of infrared image recognition, and in particular to a thermal target extraction method and an extraction system based on infrared image recognition. Background Art

[0002] Infrared thermal imaging technology is widely used in many fields, such as military reconnaissance, fire monitoring, power inspection, etc., due to its unique advantage of being able to operate in a dark environment. This technology captures the infrared radiation emitted by an object and converts it into a visible image, intuitively displays the temperature distribution, and realizes rapid perception and analysis of the target area in the environment. However, how to accurately and efficiently identify and extract thermal targets in a complex background is still a major challenge facing existing technologies.

[0003] At present, common infrared image processing methods usually rely on preset threshold segmentation or target recognition based on template matching. These methods are simple and intuitive to implement, but are easily interfered by background noise in practical applications, resulting in reduced target recognition accuracy and even misjudgment or missed detection.

[0004] In addition, since the above methods rely on static threshold settings and fixed template matching, their adaptability is poor. In a dynamically changing environment, especially when the background thermal radiation is close to the target thermal radiation, these methods cannot effectively distinguish between the target and the background, and are prone to false alarms, which significantly reduces the detection accuracy and operation efficiency. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a thermal target extraction method and extraction system based on infrared image recognition, which solves the problem that the method relies on static threshold setting and fixed template matching, and has poor adaptability. In a dynamically changing environment, especially when the background thermal radiation is similar to the target thermal radiation, these methods cannot effectively distinguish between the target and the background, and are prone to false alarms, which significantly reduces the detection accuracy and operating efficiency.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A thermal target extraction method based on infrared image recognition comprises the following steps: Infrared images, visible light images and environmental dynamic parameters are collected by multiple sensors, wherein the multiple sensors include: Infrared thermal imager, used to collect the thermal radiation characteristics of the target and generate infrared images reflecting the temperature distribution of the target; Visible light camera, used to collect the geometric shape and texture characteristics of the target, generate visible light images, and provide supplementary information for the boundary and texture of the target area; Environmental sensors are used to collect dynamic parameters of the target environment, including temperature, humidity, wind speed, light intensity, pressure, and gas concentration, and record dynamic changes in the form of time series to provide support for subsequent target extraction and parameter adjustment of the segmentation model; Perform dimensionality reduction on the collected multimodal data and map it into a low-dimensional feature space to preserve the geometric structure and semantic characteristics of the target; Perform geometric verification and topological analysis on the target area of ​​multimodal data, remove the noise area and extract the target area; Combining the energy optimization model and distribution alignment method to achieve the segmentation of the target area; Combined with a dynamic feedback mechanism, the parameters of the segmentation model are adjusted in real time according to changes in environmental parameters to ensure the accuracy of extraction and the stability of the system.

[0007] Preferably, the step of performing dimensionality reduction processing on the collected multimodal data comprises the following steps: constructing a distance weight matrix between multimodal data points , to describe the similarity between data points, where the element calculation formula of the weight matrix is in, and is the high-dimensional feature vector of multimodal data, is the Gaussian kernel parameter; Using the weight matrix Construct a diagonal matrix ,in For Node The calculation formula is and through Generate Laplacian matrix ; Solving the Eigenvalue Decomposition Problem , the constraints are in, is a low-dimensional embedding point, is the unit matrix, and the eigenvector of the Laplace matrix is ​​obtained by optimization; The optimized feature vector is used for low-dimensional embedding to generate the corresponding low-dimensional data representation in the manifold space for subsequent processing of the target area.

[0008] Preferably, the geometric verification uses a manifold metric based on Riemannian geometry to verify the target area of ​​the multimodal data, and verifies the geometric consistency of the target area through the inner product relationship between the feature mapping functions of the target area.

[0009] Preferably, the topological analysis uses a topological data analysis method to extract topological features of the target area, construct point cloud data of the target area, and calculate a persistent homologous topological barcode to verify the topological consistency of the multimodal data.

[0010] Preferably, the energy optimization model includes a smoothing term and a fitting term, and segmentation of the target area is achieved by minimizing the Euler-Lagrange equation of the energy optimization model.

[0011] Preferably, the distribution alignment is performed by matching the feature distributions of the infrared image and the visible light image in a consistent manner, and the alignment of the feature distributions is achieved by minimizing the transmission cost between the feature distributions.

[0012] Preferably, the dynamic feedback mechanism analyzes the stability of the segmentation model based on the Lyapunov function, and adjusts the parameters of the segmentation model by calculating the degree of deviation of the system state, including the regularization parameters and modal weight parameters in the energy optimization model.

[0013] Preferably, the dynamic feedback mechanism dynamically adjusts the parameters used for target area smoothing in the segmentation model according to the temperature, humidity and wind speed data collected by the environmental sensors.

[0014] Preferably, the dynamic parameters collected by the environmental sensor are used to optimize the regularization parameter weights in the segmentation model to improve the robustness of the target area segmentation in a dynamic environment, wherein the environmental sensor includes: Temperature sensor, used to detect ambient temperature in real time, monitor the dynamic changes of background thermal radiation, and optimize the ability to distinguish between target and background by adjusting the regularization parameters of the segmentation model; Humidity sensor, used to monitor changes in environmental humidity, especially under high humidity conditions, to adjust the segmentation model's response to thermal radiation characteristics to ensure that the model can adapt to different humidity environments; Wind speed sensor, used to detect changes in ambient wind speed. Dynamic changes in wind speed may cause changes in target thermal characteristics and background conditions. By adjusting the regularization weights, the model’s robustness to thermal characteristic fluctuations is improved. Light sensor, used to monitor changes in ambient light intensity, reduce the impact of light fluctuations on target boundary recognition in visible light mode, and optimize multi-modal feature fusion results; Pressure sensors are used to collect ambient pressure data and, combined with other dynamic parameters, provide supplementary context information for segmentation models in closed or highly dynamic environments; Gas sensors are used to detect changes in gas concentration in the target area, such as smoke in a fire scene, and assist the segmentation model in the dynamic modeling of background thermal radiation.

[0015] Preferably, a thermal target extraction system based on infrared image recognition comprises: Multi-sensor acquisition module, used to collect infrared images, visible light images and environmental dynamic parameters; Data dimension reduction module, used to reduce the dimension of multimodal data and map it into a low-dimensional feature space; Data verification module, used to remove noise areas and extract target areas through geometric verification and topological analysis; Segmentation optimization module, which is used to combine energy optimization model and distribution alignment method to achieve accurate segmentation of target area; The dynamic feedback module is used to adjust the parameters of the segmentation model in real time according to the dynamic parameters of the environment to ensure the stability of the system.

[0016] The present invention provides a thermal target extraction method and extraction system based on infrared image recognition, which has the following beneficial effects: 1. The present invention adopts a multi-sensor acquisition module, and through the technical solution of synchronously acquiring infrared images, visible light images and environmental dynamic parameters, it achieves the technical effect of comprehensively acquiring the target thermal radiation characteristics, geometric texture characteristics and environmental dynamic conditions. Compared with the technical solution in the prior art that only relies on a single sensor (such as an infrared thermal imager or a visible light camera), it solves the problem of target area feature loss and difficulty in eliminating background noise interference due to insufficient single-modal data.

[0017] 2. The present invention adopts a data dimensionality reduction module and a technical solution of performing nonlinear dimensionality reduction processing on multimodal data through Laplace eigenmapping, achieving the technical effect of effectively retaining the geometric characteristics and semantic information of multimodal data. Compared with the technical solution of directly performing data analysis and processing in high-dimensional space in the prior art, it solves the shortcomings of high computational complexity and difficulty in extracting core features of data.

[0018] 3. The present invention adopts a data verification module, and through geometric verification and topological analysis, removes the noise area and extracts the target area, achieving the technical effect of accurately screening the target area and eliminating noise interference. Compared with the technical solution of the prior art that only relies on fixed thresholds or simple template matching to extract the target area, it solves the problem of insufficient target area recognition accuracy caused by complex background and blurred target features.

[0019] 4. The present invention adopts a segmentation optimization module, constructs an energy optimization model and introduces a distribution alignment method to achieve a technical solution for accurate segmentation of the target area, achieving the technical effect of achieving accurate segmentation of the target boundary under a complex background. Compared with the technical solution of the prior art that relies solely on thermal thresholds or static segmentation models for target extraction, it solves the problems of blurred boundaries between the target and the background and unstable segmentation results.

[0020] 5. The present invention adopts a dynamic feedback module to adjust the segmentation model parameters in real time according to the dynamic parameters of the environment to ensure the stability of the system, achieving the technical effect of achieving robustness of target area extraction and stability of segmentation results in a dynamic environment. Compared with the technical solutions in the prior art that cannot dynamically adjust parameters according to environmental changes, it solves the problem that segmentation results are easily inaccurate in complex environments such as strong environmental noise interference and fast-changing lighting conditions.

[0021] 6. The present invention achieves the technical effect of enhancing the consistency of multimodal data and improving the accuracy of target area extraction by combining the technical solutions of multimodal data verification and distribution alignment model. Compared with the technical solutions in the prior art that do not fully utilize the complementary characteristics between modalities, the shortcomings of weak multimodal data fusion capability and excessive reliance on low signal-to-noise ratio modalities are solved.

[0022] 7. The present invention adopts the technical solution of Lyapunov stability theory to guide the dynamic feedback process, achieving the technical effect of optimizing the segmentation parameters in real time and ensuring the gradual stability of the system. Compared with the technical solution lacking dynamic stability analysis in the prior art, the problem that the dynamic parameter adjustment process easily leads to fluctuations in the segmentation results is solved.

[0023] 8. The present invention adopts modular design, including technical solutions of multi-sensor acquisition, data dimension reduction, verification segmentation and dynamic feedback modules, achieving the technical effect of clear system design, modular functions and easy expansion. Compared with the technical solutions in the prior art with high functional coupling and difficulty in expansion according to demand, the problem of insufficient system versatility and flexibility is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Please refer to the attached Figure 1 , an embodiment of the present invention provides a thermal target extraction method based on infrared image recognition, including the following contents: S1, collect infrared images, visible light images and environmental dynamic parameters through multiple sensors; In the present invention, multi-sensor data acquisition and multi-modal representation are used as the basic steps for thermal target extraction, mainly to obtain the thermal radiation characteristics, geometric texture characteristics and environmental dynamic parameters of the target. This step provides rich data support and multi-modal features for subsequent data dimensionality reduction processing, target area verification and segmentation optimization. In general, the synchronous acquisition of multi-sensor data can effectively improve the consistency and spatiotemporal resolution of the data and ensure the complementarity of different modal information.

[0027] Specifically, the multi-sensor of the present invention includes an infrared thermal imager, a visible light camera and an environmental sensor: Infrared thermal imager: used to collect the thermal radiation characteristics of the target and generate an infrared image reflecting the temperature distribution of the target. The image is represented by a two-dimensional grayscale matrix, and each pixel corresponds to the radiation intensity of the target area; Visible light camera: used to collect the geometric shape and texture characteristics of the target, generate visible light images, represented by a three-channel RGB image matrix, and provide additional information on the boundary and texture of the target area; Environmental sensors: used to collect dynamic parameters of the target environment, including the following specific types: Temperature sensor: used to monitor the ambient temperature in real time and provide support for modeling the dynamic changes of background thermal radiation; Humidity sensor: used to detect ambient humidity and support the adjustment of background modeling under complex humidity conditions; Wind speed sensor: used to monitor wind speed changes, affecting target thermal characteristics and heat dissipation modeling; Light sensor: used to monitor changes in light intensity and reduce target recognition errors caused by light changes; Pressure sensor: used to detect changes in ambient pressure and support background information supplementation in highly dynamic or closed environments; Gas sensor: used to detect changes in the concentration of special gases in the environment and assist in scene modeling (such as smoke detection in fire); Dynamic changes are recorded in the form of time series to provide an environmental basis for parameter adjustment of target extraction and segmentation models. All collected data are uniformly represented as a multimodal dataset, laying the foundation for subsequent processing.

[0028] Infrared thermal imagers are mainly used to obtain the thermal radiation distribution of the target area and form grayscale images by recording the radiation intensity within the infrared spectrum. In general, infrared image data can be represented as a two-dimensional matrix , where each pixel value corresponds to the radiation intensity of the target area, reflecting the temperature distribution characteristics of the target.

[0029] As an option, infrared thermal imagers can optimize the capture of thermal characteristics of target areas through infrared imaging modes of different wavelengths (such as medium-wave infrared or long-wave infrared) to adapt to a variety of complex environments. Specifically, the resolution of infrared images should be adjusted according to application requirements. For example, in fire monitoring scenarios, high-resolution infrared imaging equipment can be used to more accurately capture thermal anomaly areas.

[0030] The main function of a visible light camera is to obtain the geometric and texture information of a target, and to provide a rich description of the target area by supplementing the shape and texture characteristics that may be missing in infrared images. In general, visible light image data can be represented as a three-channel matrix , corresponding to the red, green and blue channel information of the image respectively.

[0031] Specifically, the visible light camera should ensure the field of view consistency with the infrared thermal imager to achieve spatial alignment of infrared and visible light modal data. In one possible implementation, the mechanical structure design of the multi-sensor installation can ensure that the optical axis of the camera and the infrared thermal imager are parallel, and the collected data can be geometrically corrected through calibration technology to eliminate parallax errors.

[0032] The role of environmental sensors is to record dynamic parameters related to the environmental status of the target area, such as ambient temperature, humidity, wind speed, etc. These dynamic parameters are used to model background conditions and assist in the dynamic adjustment of the segmentation model.

[0033] In some embodiments, the environmental sensor can transmit data in real time wirelessly, synchronously with the infrared and visible light data collection. Specifically, the environmental parameter data is represented as a time series , recording the temperature, humidity and wind speed information at each time point. As an option, the environmental data can also be bound to the spatial coordinates to analyze the local environmental change characteristics of the target area.

[0034] After completing the acquisition of infrared images, visible light images and environmental dynamic parameters, the three types of data are uniformly represented as a multimodal dataset. , which has the following form: in: represents the infrared image matrix, which contains the thermal radiation characteristics of the target area; Represents the visible light image matrix, which contains the geometric and texture characteristics of the target; Represents a time series of environmental parameters, containing the dynamic environmental conditions of the target area.

[0035] In one possible implementation, the three modal data are aligned through a time synchronization mechanism, such as setting a unified trigger signal to control the acquisition time points of multiple sensors to eliminate data inconsistency caused by time deviation.

[0036] For example, in a fire warning scenario, infrared thermal imagers are used to capture possible abnormal high temperature areas (such as fire sources or hot spots), visible light cameras provide visible light texture features of the fire source, and environmental sensors record the surrounding temperature and humidity to analyze the possible spread direction of the fire source; in power inspection scenarios, infrared thermal imagers are used to identify overheated areas of equipment, visible light cameras provide geometric contour information of the equipment, and environmental sensors record environmental conditions that may affect the heat dissipation performance of the equipment.

[0037] Through the above-mentioned acquisition method, the present invention forms a unified multimodal data set based on the collaboration of multi-sensor data, which provides rich input for subsequent data dimension reduction, target verification and segmentation optimization. In general, the complete data acquisition process ensures the temporal and spatial consistency and complementarity of modal data, and provides solid data support for hot target extraction in complex scenes.

[0038] S2, perform dimensionality reduction processing on the collected multimodal data and map it into a low-dimensional feature space to preserve the geometric structure and semantic characteristics of the target; In the present invention, after completing step S1 of multi-sensor acquisition and multi-modal data representation, the obtained multi-modal data Infrared thermal images included , visible light image And environmental parameters . These data are usually high-dimensional and have nonlinear characteristics. Directly processing them may lead to high computational complexity and fail to effectively extract the core features of the data. Therefore, the present invention proposes a dimensionality reduction processing method based on Laplace eigenmapping, which maps multimodal data from high-dimensional space to low-dimensional feature space to preserve the geometric structure and semantic characteristics of the data, and provide efficient data representation for subsequent target verification and segmentation.

[0039] Generally, Laplace eigenmap is a nonlinear dimensionality reduction method that reflects the similarity relationship between multimodal data points in a low-dimensional space by maintaining local geometric structures. As an option, the dimensionality reduction method of the present invention can adapt to the characteristics of multimodal data and show good performance in local consistency verification and global segmentation optimization of the target area.

[0040] The Laplacian eigenmap is constructed by constructing a weight matrix , Laplace matrix And eigenvalue decomposition, embed high-dimensional data into low-dimensional manifold space. The specific process is as follows: In general, we first calculate the distance weight matrix between multimodal data points , used to represent the similarity relationship between data points. Each element of the weight matrix Defined as: in: They are high-dimensional feature vectors of multimodal data; Represents the Euclidean distance between two points; is the Gaussian kernel parameter, which is used to adjust the relationship between distance and similarity.

[0041] As an option, to further enhance the local similarity weights between data points, the non-zero values ​​of the weight matrix can be restricted to exist only in -Between neighboring points, avoid the influence of long-distance data points on similarity calculation.

[0042] Diagonal Matrix Each diagonal element of is defined as the weight sum of the corresponding data point: Specifically, Representation Node The degree value of the data point The sum of similarities with its neighboring points. Then, the Laplacian matrix Defined as: in, is a symmetric positive definite matrix that describes the local geometric relationship of multimodal data. In general, the Laplacian matrix can reflect the topological structure of the data and is used to capture the intrinsic characteristics in the manifold space.

[0043] The low-dimensional representation of the data is completed by solving the following optimization problem: , the constraints are in: Representing multimodal data points Embedding coordinates in low-dimensional space; It is an orthogonality constraint to ensure that the coordinates of the low-dimensional embedding are non-redundant.

[0044] In one possible implementation, by applying the Laplace matrix Perform eigenvalue decomposition and take the smallest The eigenvectors corresponding to the non-zero eigenvalues ​​generate a low-dimensional embedding representation: in The dimension representing the low-dimensional space is usually chosen based on the dimensionality reduction goal.

[0045] Low-dimensional feature representation It not only preserves the geometric structure of the original high-dimensional data, but also significantly reduces the data complexity. In general, the low-dimensional representation can be directly used for consistency verification and segmentation optimization of the target area.

[0046] For example, in a dynamic environment, the local similarity of the target area can be analyzed through the low-dimensional embedding space of infrared thermal images and visible light images, and the global background modeling can be performed in combination with the environmental dynamic parameters, thus providing a more concise and accurate data basis for subsequent processing.

[0047] As an option, the low-dimensional features can be normalized after dimensionality reduction to ensure that the multimodal data have the same scale during object verification and segmentation.

[0048] For example, in a fire monitoring scenario, the high temperature area of ​​the infrared thermal image and the fire source boundary of the visible light image may have fuzzy overlap. Through the dimensionality reduction processing method of this embodiment, the core features of the target area and the background area can be clearly separated in the low-dimensional space. Similarly, in power inspections, the correspondence between the hot spots in the infrared thermal image and the equipment shapes in the visible light image can be more quickly identified through the low-dimensional representation after dimensionality reduction.

[0049] Through the above-mentioned dimensionality reduction processing, the present invention effectively reduces the complexity of multimodal data, and at the same time provides a unified low-dimensional representation for target verification and segmentation, laying a solid foundation for the subsequent hot target extraction step. In general, the dimensionality reduction method based on Laplace eigenmap can enhance the computational efficiency while maintaining the geometric characteristics of the data.

[0050] S3, perform geometric verification and topological analysis on the target area of ​​the multimodal data, remove the noise area and extract the target area; In the present invention, after the multimodal data is subjected to dimensionality reduction processing in step S2, a unified low-dimensional feature representation is obtained. . However, there may still be non-target areas introduced by noise, background interference or acquisition errors in multimodal data. In order to further ensure the accuracy of the target area, this step proposes a target area verification method based on geometric verification and topological analysis. In general, geometric verification is used to analyze the local consistency of the target area, while topological analysis eliminates noise areas from the perspective of the overall structure. The combination of the two can comprehensively improve the robustness and accuracy of target area verification.

[0051] Specifically, this step uses the Riemannian geometry method to verify geometric consistency, and combines topological data analysis to extract the topological features of the target area to verify the consistency of the multimodal data in the target area. As a possible implementation method, this step verifies the geometric relationship and topological structure of the target area in the low-dimensional manifold space, eliminates non-target areas, and ensures the accuracy of the final extracted area.

[0052] Generally, geometric verification determines the local geometric consistency of different modalities on the target area by calculating the similarity relationship between multi-modal target areas. This embodiment uses the metric relationship on the Riemann manifold to describe the geometric characteristics of the target area. The specific process is as follows: In the low-dimensional manifold space, the feature mapping function defining the multimodal target region is and ,in is the coordinate point in the manifold space. In general, the local geometric consistency of the target area can be calculated by the Riemann metric It means that it is defined as: in: Representing modality and A consistency measure on the target region; is the feature mapping function The gradient of , represents the local characteristics of the target area.

[0053] As an option, the global geometric consistency over the target region can be calculated by integration, as follows: in: is a low-dimensional popular space; is the geometric consistency indicator of the target area.

[0054] Specifically, when When it is less than the preset threshold, the mode is considered and otherwise, it is regarded as noise or background interference.

[0055] In a possible implementation, by calculating the geometric consistency of the infrared thermal image and the visible light image in the target area, inconsistent areas between the two, such as non-target objects with thermal characteristics in the background, are eliminated.

[0056] Topological verification analyzes the topological characteristics of the target area from the perspective of the overall structure, and is used to verify the global consistency of different modes in the target area. The specific process is as follows: In general, the spatial distribution of multimodal data on the target area is represented by constructing point cloud data P of the target area. The point cloud data can be extracted by a low-dimensional feature representation Y, where the coordinates of each point are represented by its low-dimensional embedding. To describe the topological structure of the target area, this embodiment uses a persistent homology method to calculate the topological barcode of the target area.

[0057] The calculation of persistent coherence consists of the following steps: Construct a simple complex from point cloud data P, such as k with dimension k - Simplex; Calculate k - Closed chain group and k-dimensional boundary chain groups ; Define k-dimensional homology group , whose order (Betti number) represents the topological characteristics of the target area: in: k represents the target area - Dimensional hole structure; Indicates k - Boundary structure of a 2-dimensional simplex.

[0058] Specifically, the topological barcodes of infrared thermal images and visible light images can be obtained by comparing the The topological consistency of the two modalities is verified by the algorithm. For example, when the topological features (such as connectivity or ring structure) of the two modalities are inconsistent, these regions are removed as non-target regions.

[0059] In a possible implementation, topology verification can be combined with geometric verification results to further improve the accuracy of target area verification. For example, in fire monitoring, topology verification can be used to remove high-temperature point clouds in non-fire source areas.

[0060] Generally, geometric verification and topological verification can be used in combination to comprehensively analyze the local and global characteristics of the target area. In this embodiment, the verification accuracy is improved by the following joint strategy: First, geometric verification is used to remove locally inconsistent noise points in the target area; Then, topology verification is used to analyze the global consistency of the remaining regions and remove the regions that do not conform to the target topological characteristics.

[0061] For example, in a power inspection scenario under a dynamic environment, geometric verification can eliminate high-temperature noise areas in the background, while topological verification can further ensure the overall structural consistency of the target area, thereby accurately identifying hot spots in the equipment.

[0062] Through the above-mentioned geometric verification and topological analysis, the present invention can effectively remove the noise area in the multimodal data and ensure the accuracy and consistency of the target area. In general, this step provides reliable input data for subsequent target segmentation optimization and significantly improves the robustness and accuracy of the system in complex scenarios.

[0063] S4, combining energy optimization model and distribution alignment method to achieve accurate segmentation of target area; In the present invention, after completing the target region verification in step S3, the target region in the multimodal data has been preliminarily screened through geometric and topological analysis. However, the boundary of the target region may still be ambiguous, especially in a complex background, the transition area between the target and the background may be difficult to separate accurately. To solve this problem, this step proposes a target segmentation method based on an energy optimization model, which achieves high-precision segmentation of the target region by constructing an energy function of the target region and combining distribution alignment technology.

[0064] In general, the energy optimization model segments the target and background regions by minimizing the energy function of the target region. The model contains smoothing terms and feature fitting terms, which are used to ensure the smoothness of the target boundary and the feature matching accuracy of the target region, respectively. As a possible implementation method, this step further introduces the optimal transmission theory to align the feature distributions of the infrared image and the visible light image, thereby improving the fusion effect of multimodal data and ensuring that the segmentation result of the target region is more accurate.

[0065] The segmentation of the target area is based on an energy optimization model, which constructs an energy function , to minimize the energy difference between the target area and the background area. Specifically, the energy function consists of the following two parts: in: u represents the target segmentation function, with a value range of [0,1], which is used to describe the probability that each pixel belongs to the target area; is a smoothing term, used to constrain the smoothness of the target boundary; is the regularization parameter, which is used to balance the weights of the smoothing term and the feature fitting term; f is the feature representation of the target area, which is determined by the multimodal data verification results; Represents the pixel space of an image.

[0066] Specifically, the smoothing term Usually in the quadratic form , used to suppress the noise in the boundary area and ensure the continuity of the segmentation boundary.

[0067] As an option, the feature fitting term It is used to measure the difference between the segmentation result and the target area characteristics, and ensure that the segmentation result is consistent with the verification result of the multimodal data by matching the low-dimensional feature f and the segmentation function u.

[0068] In general, the optimization of the energy function is solved by the Euler-Lagrange equation, which is in the following form: in: represents the divergence operation, which is used to measure the boundary changes of the target area; is the gradient of the smoothing term.

[0069] In a possible implementation, the minimum value of the energy function is gradually approached through an iterative optimization algorithm (such as a gradient descent method or a multi-scale optimization method) to obtain a segmentation result of the target area.

[0070] In order to further improve the multimodal consistency of the segmentation results, this embodiment combines the optimal transmission theory to align the feature distributions of the infrared image and the visible light image. Generally, optimal transmission is used to model the relationship between the feature distributions of the two modalities and achieves the alignment of feature distributions by minimizing the transmission cost.

[0071] The mathematical model of optimal transmission is defined as follows: in: x and y are the feature distribution spaces of infrared images and visible light images respectively; is the transmission cost function, usually defined as the Euclidean distance ; is the transmission mapping function, which represents the distribution mapping relationship from infrared mode to visible light mode.

[0072] Specifically, by solving the above optimization problem, the optimal transmission mapping can be obtained , which is used to align the target area distribution of infrared and visible light modalities, thereby improving the multimodal consistency of segmentation results.

[0073] As a possible implementation method, transmission mapping can be combined with the verification results of the target area, eliminating the background noise area and retaining only the target area where the two modalities are consistent.

[0074] In general, the energy optimization model and distribution alignment technology can be used in combination to further improve the accuracy of target region segmentation. In this embodiment, distribution alignment is used to optimize the feature fitting term in the energy function to ensure the multimodal consistency of the target region feature f. At the same time, the noise points in the background area are removed by distribution alignment to improve the convergence efficiency of the energy optimization model.

[0075] For example, in a fire monitoring scenario, the high-temperature area in the infrared image and the flame outline in the visible light image can be aligned through distribution alignment technology to accurately extract the boundary of the fire source. At the same time, the energy optimization model ensures the integrity and accuracy of the segmentation results by smoothing the target boundary.

[0076] Through the above-mentioned target region segmentation method based on energy optimization model and distribution alignment, the present invention realizes high-precision target region extraction of multimodal data. In general, this step provides high-quality segmentation results for subsequent dynamic feedback adjustment and system stability, and significantly improves the system's adaptability in complex scenarios.

[0077] S5. Combined with the dynamic feedback mechanism, the parameters of the segmentation model are adjusted in real time according to the changes in environmental parameters to ensure the accuracy of extraction and the stability of the system.

[0078] After completing the target area segmentation in step S4, the segmentation result of the present invention needs to be adjusted in real time according to the dynamic changes of the environment to ensure the accuracy of the segmentation and the stability of the system. In general, complex dynamic environments (such as temperature and humidity fluctuations, background light changes) may cause the parameters of the segmentation model to be incompatible, thereby affecting the segmentation effect. Therefore, the present invention designs a parameter optimization method based on a dynamic feedback mechanism, which dynamically adjusts the key parameters of the segmentation model by sensing environmental changes in real time, thereby adapting to environmental changes and maintaining system stability.

[0079] Specifically, the present invention combines Lyapunov stability theory to define the degree of deviation of the system state, and adjusts the regularization parameters and modal weight parameters of the segmentation model based on the theory to ensure that the segmentation performance of the system in a dynamic environment is not affected. As a possible implementation method, the environmental dynamic parameters are collected in real time by sensors and transmitted to the segmentation model through a feedback loop for optimization.

[0080] The dynamic feedback mechanism relies on data collected by environmental sensors, including dynamic parameters such as ambient temperature, humidity, wind speed, etc. Generally, these data are used to describe the background dynamic conditions of the target area, thus providing a basis for the optimization of model parameters.

[0081] Alternatively, environmental data are represented as time series ,in represents the environmental parameter vector at time t. Specifically, these parameters can be used to adjust the regularization weights in the energy optimization model and modal weight coefficients in the distribution alignment model.

[0082] For example, when the ambient temperature fluctuates greatly, by increasing the regularization weight The boundary continuity of the segmentation results can be improved; when the lighting conditions change greatly, the weight ratio of the infrared modality and the visible light modality can be dynamically adjusted to ensure the segmentation consistency of multimodal data.

[0083] In general, in order to evaluate the dynamic stability of the segmentation model, this embodiment measures the degree of deviation of the system state based on the Lyapunov function. The Lyapunov function is defined as: in: x represents the current state vector of the system, represents the target stable state vector of the system; Indicates the degree of deviation between the current state and the target state.

[0084] Specifically, the Lyapunov stability condition is: in is the time derivative of the Lyapunov function. When it is less than zero, the system is judged to be asymptotically stable.

[0085] In one possible implementation, the parameters of the segmentation model are adjusted by optimizing For example, when the system state deviates from the target state, by increasing the regularization weight Or the dynamic adjustment coefficient of the modal weight To suppress the degree of deviation.

[0086] Generally speaking, the dynamic feedback mechanism achieves parameter optimization through the following two parts: Regularization parameter Dynamic adjustment: When the environmental noise or background fluctuations are large, the regularization weight is appropriately increased , in order to enhance the constraint of the segmentation model on the boundary smoothness. The specific formula is: in: is the initial regularization parameter; k is the dynamic adjustment coefficient; is the deviation between the current environmental parameters and the reference environmental parameters.

[0087] Dynamic allocation of modal weights: When the signal-to-noise ratio of the infrared modality or visible light modality changes, the weights of the multimodal features are dynamically adjusted. The specific formula is: in: and are the weights of infrared and visible light modes respectively; and represent the signal-to-noise ratio of infrared and visible light modes, respectively.

[0088] In one possible implementation, when the noise of the infrared mode increases, the dynamic to reduce the influence of infrared modality on the segmentation results.

[0089] For example, in a fire monitoring scenario, changes in ambient temperature and humidity can significantly affect the thermal characteristics of infrared images. Through a dynamic feedback mechanism, the regularization parameters in the energy optimization model can be adjusted according to the environmental data. , thereby enhancing the robustness of the segmentation results.

[0090] Similarly, in the power inspection scenario, when the signal-to-noise ratio of the visible light mode is reduced due to strong light interference, the modal weight can be adjusted and The allocation ratio is adjusted to give priority to the use of infrared modality data to ensure the accuracy of the segmentation results.

[0091] Through the above dynamic feedback mechanism, the present invention can optimize the parameters of the segmentation model in real time under dynamic environment, ensuring the robustness of the target area extraction result and the stability of the system. In general, this mechanism not only adapts to complex dynamic environment, but also provides strong stability support for the long-term operation of the system.

[0092] Please see attached Figure 2 , a thermal target extraction system based on infrared image recognition, including Multi-sensor acquisition module, used to collect infrared images, visible light images and environmental dynamic parameters; The multi-sensor acquisition module is used to synchronously acquire infrared images, visible light images, and environmental dynamic parameters. The infrared thermal imager captures the thermal radiation characteristics of the target for temperature distribution detection; the visible light camera provides the geometric contour and texture information of the target; the environmental sensor records dynamic parameters (such as temperature, humidity, wind speed, etc.) to analyze the impact of the environment on the target characteristics. Through time synchronization and spatial correction, the multi-modal data is guaranteed to accurately cover the target area, providing accurate input for subsequent processing; Data dimension reduction module, used to reduce the dimension of multimodal data and map it into a low-dimensional feature space; The data dimension reduction module performs dimension reduction processing on multimodal data, maps high-dimensional data to low-dimensional feature space, retains core features and reduces computational complexity. The dimension reduction method based on manifold learning constructs data point similarity weights and generates a unified low-dimensional manifold representation, thereby retaining the geometric characteristics and semantic information of the target area, providing simplified and accurate data support for subsequent verification and segmentation; Data verification module, used to remove noise areas and extract target areas through geometric verification and topological analysis; The data verification module removes the noise area and extracts the target area through geometric verification and topological analysis. Geometric verification analyzes the local consistency of multimodal data and removes noise points that are inconsistent with the target features; topological analysis ensures regional consistency by extracting the global structural characteristics of the target area. Combining the complementary characteristics of infrared and visible light modalities, it effectively reduces the interference of complex background on the target.

[0093] Segmentation optimization module, which is used to combine energy optimization model and distribution alignment method to achieve accurate segmentation of target area; The segmentation optimization module combines the energy optimization model with the distribution alignment method to complete the accurate segmentation of the target area. Through the smoothing term and feature fitting term of the energy optimization model, the continuity of the segmentation boundary and the feature consistency of the segmentation result are ensured; through distribution alignment, the multimodal feature distribution is matched to enhance the distinction between the target boundary and the background, and a clear target area is output.

[0094] Dynamic feedback module, used to adjust the parameters of the segmentation model in real time according to the dynamic parameters of the environment to ensure the stability of the system; The dynamic feedback module adjusts the segmentation model parameters in real time according to the dynamic parameters of the environment to ensure system stability. The regularization weights and modal feature weights in the segmentation model are optimized through dynamic parameters such as temperature, humidity, and wind speed recorded by sensors to adapt to environmental changes. The real-time feedback mechanism effectively improves the robustness and accuracy of the segmentation results in dynamic backgrounds.

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the appended claims and their equivalents.

Claims

1. A thermal target extraction method based on infrared image recognition, characterized in that: The following steps are involved: Infrared images, visible light images and environmental dynamic parameters are collected by multiple sensors, wherein the multiple sensors include: Infrared thermal imager, used to collect the thermal radiation characteristics of the target and generate infrared images reflecting the temperature distribution of the target; Visible light camera, used to collect the geometric shape and texture characteristics of the target, generate visible light images, and provide supplementary information for the boundary and texture of the target area; Environmental sensors are used to collect dynamic parameters of the target environment, including temperature, humidity, wind speed, light intensity, pressure, and gas concentration, and record dynamic changes in the form of time series to provide support for subsequent target extraction and parameter adjustment of the segmentation model; Perform dimensionality reduction on the collected multimodal data and map it into a low-dimensional feature space to preserve the geometric structure and semantic characteristics of the target; Perform geometric verification and topological analysis on the target area of ​​multimodal data, remove the noise area and extract the target area; Combining the energy optimization model and distribution alignment method to achieve the segmentation of the target area; Combined with a dynamic feedback mechanism, the parameters of the segmentation model are adjusted in real time according to changes in environmental parameters to ensure the accuracy of extraction and the stability of the system.

2. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The step of performing dimensionality reduction processing on the collected multimodal data comprises the following steps: By constructing a distance weight matrix between multimodal data points , to describe the similarity between data points, where the element calculation formula of the weight matrix is in, and is the high-dimensional feature vector of multimodal data, is the Gaussian kernel parameter; Using the weight matrix Construct a diagonal matrix ,in For Node The calculation formula is and through Generate Laplacian matrix ; Solving the Eigenvalue Decomposition Problem , the constraints are in, is a low-dimensional embedding point, is the unit matrix, and the eigenvector of the Laplace matrix is ​​obtained by optimization; The optimized feature vector is used for low-dimensional embedding to generate the corresponding low-dimensional data representation in the manifold space for subsequent processing of the target area.

3. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The geometric verification adopts a manifold metric based on Riemannian geometry to verify the target area of ​​the multimodal data, and verifies the geometric consistency of the target area through the inner product relationship between the feature mapping functions of the target area.

4. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The topological analysis uses a topological data analysis method to extract topological features of a target area, construct point cloud data of the target area, and calculate a persistent homologous topological barcode to verify the topological consistency of multimodal data.

5. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The energy optimization model includes a smoothing term and a fitting term, and segmentation of the target area is achieved by minimizing the Euler-Lagrange equation of the energy optimization model.

6. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The distribution alignment achieves alignment of feature distributions by performing consistency matching between feature distributions of the infrared image and the visible light image, and by minimizing the transmission cost between feature distributions.

7. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The dynamic feedback mechanism analyzes the stability of the segmentation model based on the Lyapunov function, and adjusts the parameters of the segmentation model by calculating the degree of deviation of the system state, including the regularization parameters and the modal weight parameters in the energy optimization model.

8. The thermal target extraction method based on infrared image recognition according to claim 1 is characterized in that: The dynamic feedback mechanism dynamically adjusts the parameters used for target area smoothing in the segmentation model according to the temperature, humidity and wind speed data collected by the environmental sensors.

9. The thermal target extraction method based on infrared image recognition according to claim 8 is characterized in that: The dynamic parameters collected by the environmental sensor are used to optimize the regularization parameter weights in the segmentation model to improve the robustness of the target area segmentation in a dynamic environment, wherein the environmental sensor includes: Temperature sensor, used to detect ambient temperature in real time, monitor the dynamic changes of background thermal radiation, and optimize the ability to distinguish between target and background by adjusting the regularization parameters of the segmentation model; Humidity sensor, used to monitor changes in environmental humidity, especially under high humidity conditions, to adjust the segmentation model's response to thermal radiation characteristics to ensure that the model can adapt to different humidity environments; Wind speed sensor, used to detect changes in ambient wind speed. Dynamic changes in wind speed may cause changes in target thermal characteristics and background conditions. By adjusting the regularization weights, the model’s robustness to thermal characteristic fluctuations is improved. Light sensor, used to monitor changes in ambient light intensity, reduce the impact of light fluctuations on target boundary recognition in visible light mode, and optimize multi-modal feature fusion results; Pressure sensors are used to collect ambient pressure data and, combined with other dynamic parameters, provide supplementary context information for segmentation models in closed or highly dynamic environments; Gas sensors are used to detect changes in gas concentration in the target area, such as smoke in a fire scene, and assist the segmentation model in the dynamic modeling of background thermal radiation.

10. A thermal target extraction system based on infrared image recognition, applied to the thermal target extraction method based on infrared image recognition as claimed in any one of claims 1 to 9, characterized in that: include: Multi-sensor acquisition module, used to collect infrared images, visible light images and environmental dynamic parameters; Data dimension reduction module, used to reduce the dimension of multimodal data and map it into a low-dimensional feature space; Data verification module, used to remove noise areas and extract target areas through geometric verification and topological analysis; Segmentation optimization module, which is used to combine energy optimization model and distribution alignment method to achieve accurate segmentation of target area; The dynamic feedback module is used to adjust the parameters of the segmentation model in real time according to the dynamic parameters of the environment to ensure the stability of the system.

Images