Partition data processing method for infrared image in specific area

Abstract

The invention relates to the technical field of infrared image processing, and discloses a partition data processing method for an infrared image in a specific area, and the method comprises the following steps: obtaining the data of a multi-mode sensor; carrying out calibration and preprocessing on the obtained multi-modal data; constructing an objective function of partition processing, wherein the objective function is used for measuring the characteristics of the partition regions; performing optimization processing on the target function, and determining a partition boundary; and outputting a target region partition based on an optimization result, the multi-modal sensor comprising: an infrared image sensor configured to collect thermal radiation information; the visible light image sensor is used for collecting texture and color information; and the laser radar sensor is used for collecting depth and space information. By adopting the technical scheme of multi-modal data fusion and combining the infrared image, the visible light image and the laser radar data, the technical effect of remarkably improving the accuracy and robustness of the partitioning result is achieved.

Description

Technical Field

[0001] The invention relates to the technical field of infrared image processing, and in particular to a partition data processing method for infrared images of a specific area. Background Art

[0002] With the rapid development of information technology, infrared imaging technology is widely used in monitoring, security, military reconnaissance and other fields because it is not restricted by light conditions and has good recognition performance at night or in complex environments. However, traditional infrared image processing methods often lack targeted data processing strategies, especially when facing complex targets in a specific area, how to efficiently and accurately extract useful information has become a challenge.

[0003] At present, common infrared image processing technologies mainly include global threshold segmentation, edge detection technology, and target recognition methods based on deep learning. The global threshold segmentation method distinguishes the target from the background by setting a unified standard. It is easy to operate, but the accuracy is poor when the background is complex and changeable; the edge detection technology relies on the change of image gradient to locate the boundary of the object, and it is difficult to effectively capture the target in the low-contrast environment; although the method based on deep learning has significantly improved the target recognition accuracy, it requires a large number of labeled samples for training and consumes a lot of computing resources. These methods have their own advantages, but there are still obvious limitations in the fine data processing of specific areas. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a partition data processing method for infrared images of a specific area, which solves the problems of insufficient accuracy of target area partitioning in complex backgrounds, insufficient multimodal data fusion, and poor reliability of partition results and applicability to multiple scenarios in traditional infrared image processing.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A partition data processing method for infrared images of a specific area, comprising the following steps: Acquire data from multimodal sensors; Calibrate and preprocess the acquired multimodal data; constructing an objective function of the partitioning process, wherein the objective function is used to measure the characteristics of the partitioned area; Optimize the objective function and determine the partition boundaries; Output the target region partition based on the optimization results.

[0006] Preferably, the multimodal sensor comprises: Infrared image sensor, used to collect thermal radiation information; Visible light image sensor for collecting texture and color information; LiDAR sensor, used to collect depth and spatial information.

[0007] Preferably, the multimodal data calibration includes: Spatial registration and temporal synchronization of trimodal data: Spatial registration realizes the registration of infrared images and visible light images through feature matching algorithm: The LiDAR data is mapped to the image plane coordinate system through the point cloud projection matrix.

[0008] Preferably, the objective function includes a data consistency term, a boundary smoothing term and a modal interaction verification term, and its expression is: ; in, represents the total energy of the objective function; Represented as partitioned areas; Indicates the partition boundary; Represents the data consistency term, which is used to measure the characteristic consistency of different modal data in the target area; Represents the boundary smoothing term, which is used to constrain the smoothness of the partition boundary; Represents the modality interaction verification item, which is used to measure the data consistency between modalities; represents the weight of the smoothing term; Represents the weight of the modal interaction validation item.

[0009] Preferably, the optimization process of the objective function comprises the following steps: The objective function is modeled based on the Lagrange multiplier method, and the partition consistency constraint is introduced. The expression of the objective function is: ; in, Expressed as Lagrangian objective function; It is expressed as a consistency constraint, which is used to constrain the boundary consistency of data between modalities; The partition boundary is iteratively solved using the gradient descent method. The update expression of the partition boundary is: ; in, Indicates The partition boundaries at the iteration; represents the learning rate; represents the gradient of the objective function; Dynamically adjust the modal weight in the objective function. The modal weight is determined according to the noise level of the modal data. The calculation formula of the modal weight is: ; in, Representing modality The weight of Representing modality Noise level; Represents an index variable; Indicates The standard deviation of each mode; The objective function is continuously optimized until the energy converges to the optimal value and the final partition boundary is generated.

[0010] Preferably, the partitioning process comprises the following steps: Determine the partition boundary of the target area and the pixel set within the partition; Generate corresponding characteristic data based on the partition result of the target area; The characteristic data is analyzed and processed and then outputted, and the partition results are used for monitoring, security or fire detection of the target area.

[0011] Preferably, the data consistency term is calculated according to the dynamic modal weight, and the expression of the data consistency term is: ; in, Indicates data consistency item; Representing modality data; represents the estimated value of the characteristic of the target area within the partition; Representing modality The weight of .

[0012] Preferably, the modal interaction verification item verifies the consistency of the segmented region by analyzing the boundary probability distribution between the modalities, and evaluates the consistency of the modal boundaries based on a Bayesian inference model.

[0013] Preferably, the target area partitioning comprises the following steps: Construct initial partition boundaries of the target area; Based on the characteristics of multimodal data, data consistency analysis is performed on the initial partition boundaries; Use optimization algorithms to dynamically adjust partition boundaries so that they match the characteristic distribution of the target area; The partition boundaries of the target area are modified in combination with the modal interaction verification results to generate the final partition results.

[0014] The present invention also provides a partition data processing system for infrared images of a specific area, comprising: Multimodal sensor module for collecting infrared images, visible light images and lidar point cloud data; Data calibration module, used to perform spatial registration and temporal synchronization on the collected multimodal data: Partition modeling module, used to construct the partition processing objective function of the target area: The optimization solution module is used to model the objective function based on the Lagrange multiplier method, and iteratively optimize the partition boundaries through the gradient descent method, dynamically adjust the modal weights and optimize the target area partitions; Partition verification module, used to evaluate the reliability of partition results through modal interaction verification: The result output module is used to output the partition boundary of the target area and the pixel set within the partition. The partition result can be used for monitoring, security or fire monitoring scenarios of the target area.

[0015] The present invention provides a method for processing partition data of infrared images of a specific area, which has the following beneficial effects: 1. The present invention adopts a multimodal data fusion technology solution, combining infrared images, visible light images and lidar data, to achieve a technical effect of significantly improving the accuracy and robustness of the partitioning results. Compared with the problem of insufficient characteristic information of the target area caused by single-modal data processing in the prior art, the shortcomings of single data source and limited processing accuracy are solved.

[0016] 2. The present invention adopts the technical solution of objective function modeling, and quantitatively evaluates the partitioned area through data consistency terms, boundary smoothing terms and modal interaction verification terms, achieving the technical effect of accurate partitioning under complex backgrounds. Compared with the technical solutions of global threshold segmentation and edge detection in the prior art that have poor adaptability to low-contrast environments, the problem of insufficient partitioning accuracy under complex backgrounds is solved.

[0017] 3. The present invention adopts the technical solution of dynamic modal weight adjustment and optimization solution, and iteratively optimizes the partition boundary through Lagrangian modeling and gradient descent method, so as to achieve the technical effect of dynamically adjusting the partition result according to the quality of modal data. Compared with the technical solution in the prior art that fails to fully consider the noise level of multimodal data, it solves the problem of inaccurate optimization of partition boundaries.

[0018] 4. The present invention uses the technical solution of modal interaction verification and Bayesian inference model to evaluate the reliability and consistency of the partitioning results, achieving the technical effect of effectively excluding abnormal partitioning areas. Compared with the technical solution in the prior art where the reliability of the partitioning results is difficult to verify, the shortcomings of discontinuous partition boundaries and modal conflicts are solved.

[0019] 5. The present invention achieves the technical effect of improving the applicability and extensibility of partition results by adopting a technical solution of outputting partition results in multiple formats, generating partition boundary and regional characteristic data and supporting multiple application scenarios. Compared with the technical solution in the prior art where the partition results are limited to a single scenario application, the problem of insufficient applicability in multiple scenarios is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 The embodiment of the present invention provides a method for processing partition data of an infrared image of a specific area, comprising the following steps: S1, obtaining data from multimodal sensors; S2, calibrating and preprocessing the acquired multimodal data; S3, constructing an objective function for partition processing, wherein the objective function is used to measure characteristics of the partition area; S4, optimizing the objective function and determining the partition boundary; S5. Output the target area partition based on the optimization result.

[0023] For S1, the data of multimodal sensors is obtained. This step is the basis of the entire partition data processing process and directly affects the results of subsequent calibration, partition modeling and optimization processing. By rationally selecting and configuring multimodal sensors, sufficient raw data support can be provided for the precise partitioning of the target area, and the integrity of the spatial, temporal and characteristic information of the data can be ensured.

[0024] In general, data collection from multimodal sensors includes multiple data sources, and each modality has different characteristics and applicable scenarios. For example, infrared image data mainly reflects the thermal radiation characteristics of the target area, visible light image data provides texture and color information, and lidar data captures the depth and spatial geometric characteristics of the target area. The comprehensive acquisition and use of multimodal data can significantly improve the accuracy and robustness of partition processing.

[0025] Specifically, in this embodiment, step S1 includes the following contents: First, configure a multimodal sensor system. As an option, the following multimodal sensors are preferably used in the present invention: Infrared image sensor, used to capture the thermal radiation characteristics of the target area, whose output data contains the infrared intensity information of the target area, and is used to detect the thermal signal distribution of the target object; A visible light image sensor is used to capture the color and texture characteristics of the target area. Its output data contains the optical characteristics of the target area and is used to refine the outer boundary of the target object. The laser radar sensor is used to obtain the depth information of the target area. Its output data includes point cloud data, which can reflect the three-dimensional spatial geometric characteristics of the target area.

[0026] In one implementation, the multimodal sensor system achieves synchronous acquisition through a hardware trigger signal. Generally, the frame rate of the sensor synchronous acquisition should meet the requirements of the scene dynamics to ensure the data integrity and consistency of the target area during the sampling process.

[0027] In some embodiments, the acquisition range of the multimodal sensor should cover the entire area of ​​the target area. Specifically, the field of view of the infrared image sensor and the visible light image sensor needs to match to avoid the spatial misalignment problem between the different modal data; the detection range of the lidar needs to cover all significant features of the target area, including the boundaries and internal areas of the target object.

[0028] As an option, the multimodal data output by the sensor can be stored in a variety of formats. Infrared images and visible light images are usually stored in the form of a two-dimensional matrix, and the pixel values ​​in the matrix represent the thermal radiation intensity and optical characteristics of the target area under each mode; the data output by the lidar is usually stored in the form of a three-dimensional point cloud, and each point contains the three-dimensional spatial coordinates. and its reflection intensity value.

[0029] In one implementation, the acquisition of multimodal data can be performed through the following process: The thermal signal distribution of the target area is collected by an infrared image sensor and output in the form of a grayscale image; The optical characteristics of the target area are collected by a visible light image sensor and output in the form of a color image or a grayscale image; Scan the target area through the lidar sensor and output point cloud data, each point contains the three-dimensional space coordinates and its corresponding reflection intensity value.

[0030] In general, there may be data noise problems during the acquisition process. For example, infrared images may be affected by ambient temperature, visible light images may be affected by lighting conditions, and lidar data may have sparse point clouds or local missing points. Therefore, as an implementation method, a preprocessing mechanism can be introduced in the data acquisition stage, including noise filtering, dynamic range adjustment and other operations to improve the quality of multimodal data.

[0031] In this embodiment, the collected multimodal data can be expressed by the following expression: Infrared image data: ,in, Indicates the infrared image at pixel point The thermal radiation intensity at Visible light image data: ,in, Indicates the visible light image at pixel point Optical properties at LiDAR point cloud data: ,in, Represents the three-dimensional coordinates of the point cloud, Indicates the reflection intensity of the point cloud.

[0032] As an option, the acquisition parameters can be adjusted according to the needs of the application scenario during the sensor configuration process. For example, in night monitoring scenarios, the acquisition sensitivity of infrared image sensors can be improved first; in complex lighting environments, high dynamic range imaging technology can be introduced to enhance the detail capture of visible light images; in three-dimensional target segmentation scenarios, the point cloud density of the lidar can be adjusted to ensure accurate scanning of the target area.

[0033] The data collected by the above sensors will be directly used in the subsequent calibration and preprocessing steps. Through high-quality multimodal data input, reliable basic data support can be provided for the construction of the objective function and the partition optimization. Due to the complementary characteristics of multimodal data, infrared image data can provide the thermal characteristics of the target area, visible light image data can provide texture information, and lidar point cloud data can provide depth information. The combination of multimodal data will effectively solve the shortcomings of single-modal data processing and improve the accuracy and robustness of the target area partition results.

[0034] In summary, through the implementation of step S1, comprehensive acquisition of multimodal sensor data can be achieved, providing sufficient raw data support for accurate partitioning of the target area.

[0035] For S2, it involves calibration and preprocessing of the acquired multimodal data, which is an essential key link after data acquisition. Through the calibration and preprocessing steps, the differences in time, space and characteristics of multimodal data can be solved, laying a consistent foundation for the construction and optimization of subsequent partition objective functions. In general, multimodal data calibration includes spatial alignment and time synchronization, and preprocessing includes noise filtering, data format standardization, and dynamic range adjustment. Through this step, the fusion accuracy between multimodal data can be effectively improved, and the partitioning error caused by data differences can be reduced.

[0036] Specifically, in this embodiment, step S2 includes the following contents: In terms of spatial calibration, it is first necessary to achieve spatial alignment between infrared images, visible light images, and lidar point cloud data. In general, there may be differences in the field of view of infrared images and visible light images, and the spatial coordinate systems of lidar point cloud data and image data are inconsistent. As a possible implementation method, the calibration plate calibration method can be used to obtain the spatial transformation matrix between the infrared image and the visible light image, thereby completing the registration of the two. Specifically, the calibration plate calibration method detects feature points on the calibration plate and calculates the spatial transformation relationship between the feature points to achieve spatial alignment of the two modal data. In another implementation method, the ORB (Oriented FAST and RotatedBRIEF) feature point matching algorithm can be used to automatically detect corresponding feature points in the infrared image and the visible light image, and complete the registration of the two based on the homography matrix.

[0037] In the spatial mapping of point cloud data, the projection matrix of the laser radar can be used to map the three-dimensional point cloud data to the image plane coordinate system. In general, the projection matrix can be obtained through the geometric calibration of the laser radar and the image sensor, and its mapping relationship can be expressed as: ; in, Expressed as image plane coordinates, Represented as the three-dimensional coordinates of the point cloud, Represented as a projection matrix.

[0038] In general, in order to ensure the consistency of data, it is necessary to interpolate the registered infrared image, visible light image and point cloud data. For example, for sparse point cloud data, bilinear interpolation or nearest neighbor interpolation can be used to interpolate the point cloud data to the same resolution and grid structure as the image data for subsequent fusion processing.

[0039] In terms of time synchronization, in order to ensure the temporal consistency of multimodal data, data synchronization can be achieved through hardware trigger signals or timestamp alignment methods. As an option, the hardware trigger signal controls the sampling time of each sensor through a unified trigger device, thereby ensuring the temporal consistency of infrared images, visible light images, and lidar data. In another implementation, the timestamp-based alignment method achieves time synchronization by recording the acquisition timestamp of each modal data and interpolating or cropping the data according to the time difference.

[0040] In terms of preprocessing, the noise in the multimodal data needs to be filtered first. Infrared image data is generally affected by thermal noise, visible light images generally introduce random noise due to lighting conditions, and LiDAR point cloud data generally contains pseudo points or noise points. Specifically, infrared images can be smoothed using a Gaussian filter to reduce the impact of thermal noise; visible light images can be smoothed using a median filter to remove random noise; and LiDAR point cloud data can use statistical filtering methods or radius filtering methods to remove outliers and pseudo points.

[0041] As an option, the dynamic range of multimodal data can also be adjusted to enhance the contrast and feature saliency between different modal data. Dynamic range adjustment of infrared images can be achieved through histogram equalization. Specifically, the histogram equalization method extends the dynamic range of the image to the entire pixel value range by redistributing the probability distribution of pixel values, thereby enhancing the contrast of the image. Visible light images can use gamma correction to adjust the brightness distribution to adapt to the lighting conditions of the target area. The reflection intensity of lidar data can be standardized by linear normalization to enhance the expressiveness of point cloud features.

[0042] In terms of data format standardization, all multimodal data need to be converted into a unified format for subsequent processing. For example, infrared images and visible light images can be standardized into two-dimensional matrices with the same resolution and pixel depth; lidar data can be standardized into a three-dimensional point cloud structure with a fixed point cloud density.

[0043] In some embodiments, the effects of calibration and preprocessing can be further optimized by combining the characteristics of multimodal data. For example, for infrared images with lower resolution, the high-resolution characteristics of visible light images can be used for super-resolution reconstruction, thereby improving the resolution of infrared images. In another possible implementation, the three-dimensional geometric characteristics of lidar point cloud data can be used to correct the boundaries of infrared images and visible light images, thereby improving the accuracy of multimodal data fusion.

[0044] Through the calibration and preprocessing of this step, the acquired multimodal data can be converted into standardized data with good consistency, providing reliable data support for the construction of the objective function and partition optimization. At the same time, the noise filtering and dynamic range adjustment of multimodal data can significantly improve the data quality, thereby enhancing the accuracy and robustness of the target area partition. In summary, this step plays a key role in the entire partition data processing method and is the basis for ensuring the smooth implementation of subsequent steps.

[0045] For S3, it is the core link to realize partition data processing. Through the construction of the objective function, the optimization goal of partition processing can be clarified, and quantitative evaluation criteria can be provided to provide guidance for subsequent partition boundary optimization. In general, the design of the objective function needs to comprehensively consider the consistency of multimodal data characteristics, the smoothness of the boundary, and the interactive verification between modes, so as to ensure the accuracy and reliability of the target area partition results.

[0046] Specifically, in this embodiment, step S3 includes the following contents: The construction of the objective function includes three parts: data consistency term, boundary smoothing term and modal interaction verification term, which are described as follows: In general, the data consistency term is used to measure the consistency between the multimodal data in the partitioned area and the estimated value of the target area characteristics, so as to ensure that the partitioned area can accurately reflect the characteristics of the target area. The mathematical expression of the data consistency term is: ; in, It is expressed as a data consistency term; Represented as modal , which are infrared image data, visible light image data and lidar data respectively in the present invention; Represented as modal The weight of is dynamically determined by the noise level of the modal data and is calculated as: ; in, Representing modality Noise level; Representing modality The weight of Represents the index variable, which is used to traverse the data uncertainty of all modes; express The standard deviation of the mode (or data channel), which measures the degree of dispersion or uncertainty of the modal data.

[0047] Alternatively, the target region's characteristic estimate It can be calculated by the weighted average value in the partition area. The specific calculation formula is: ; The above calculation can effectively utilize the complementarity of multimodal data and reduce the interference of high noise modes on the partitioning results.

[0048] In the construction of boundary smoothing terms, the smoothness constraint of the partition boundary is mainly considered to avoid discontinuous or irregular boundary shapes in the partition results. The mathematical expression of the boundary smoothing term is: ; in, is represented as a boundary smoothing term, It is expressed as the gradient change of the partition boundary and is used to describe the smoothness of the partition boundary.

[0049] In one implementation, the weight parameter of the boundary smoothing term can be increased by To enhance the boundary smoothness of the partitioning result, so as to adapt to the target area partitioning requirements in complex scenes. The modal interaction verification item is used to measure the data consistency between the modalities and verify the reliability of the target area partitioning through comparative analysis of multimodal data. The mathematical expression of the modal interaction verification item is: ; in, Represented as a modal interaction verification item; Represented as modal and modal Observed value of Represented as modal and modal The interaction weight of .

[0050] As an option, interaction weights can be set based on the feature complementarity between the modalities , in order to improve the effect of modal interaction verification. For example, infrared image data and visible light image data have strong complementarity in thermal radiation and physical characteristics, so a higher value can be assigned to their interaction weights.

[0051] In this embodiment, the total energy expression of the objective function is: ; in, Expressed as the total energy of the objective function; Represented as partitioned areas; Represented as partition boundaries; Expressed as the weight of the boundary smoothing term; Expressed as the weight of the modal interaction validation term.

[0052] In general, the smaller the total energy value of the objective function, the more the partition area meets the characteristics of multimodal data, the smoother the partition boundary and the higher the consistency of modal interaction. and The value of controls the importance of boundary smoothness and modal interaction verification to meet the needs of different scenarios.

[0053] This step discusses the construction of the objective function and clarifies the evaluation criteria and optimization goals of the partition optimization. The construction of the objective function in the entire partition data processing process directly affects the accuracy and reliability of the partition results, and also provides a mathematical basis for subsequent optimization processing. By reasonably designing the data consistency term, boundary smoothing term, and modal interaction verification term in the objective function, the characteristics of multimodal data can be fully utilized to significantly improve the accuracy and robustness of the target area partition. In summary, this step is the key link in realizing the infrared image partition data processing method for a specific area.

[0054] For S4, the objective function is optimized to determine the optimal partition boundary of the target area. Optimization is the core step of partition data processing. Its main goal is to obtain the optimal partition result that meets the multimodal data characteristics and partition requirements by solving the energy minimization of the objective function. In general, optimization needs to comprehensively consider multiple constraints of the objective function, including data consistency, smoothness of partition boundaries, and modal interaction verification results, so as to ensure the accuracy and robustness of the partition.

[0055] Specifically, in this embodiment, step S4 includes the following contents: In general, the optimization of the objective function needs to strike a balance between data consistency, boundary smoothness, and modal interaction verification. In order to introduce partition consistency constraints, the objective function is modeled in Lagrangian in this embodiment. Specifically, the Lagrangian form of the objective function is expressed as: ; in, Expressed as Lagrangian objective function; It is expressed as the original objective function, including data consistency term, boundary smoothing term and modal interaction verification term; It is expressed as a partition consistency constraint to ensure consistency between multimodal data; It is expressed as a Lagrange multiplier and is used to adjust the trade-off between the objective function and the consistency constraint.

[0056] As an option, the partition consistency constraint It can be calculated based on the boundary consistency between modal data. The specific formula is: ; in, , , They are the observation values ​​of infrared image, visible light image and lidar respectively.

[0057] After the Lagrangian modeling is completed, the objective function can be iteratively optimized by the gradient descent method. Specifically, the update formula of the partition boundary is: ; in, Expressed as The partition boundaries at the iteration; Represented as the learning rate, used to control the step size of gradient descent; Expressed as the gradient of the Lagrangian objective function.

[0058] In general, the learning rate The choice of has an important influence on optimizing the convergence speed and stability. In one possible implementation, a method of dynamically adjusting the learning rate can be adopted, for example, by gradually reducing the learning rate to accelerate convergence.

[0059] During the optimization process, the modal weights in the objective function also need to be dynamically adjusted to improve the balance of the impact of different modal data on the partition results. The dynamic adjustment formula for modal weights is: ; in, Represented as modal The weight of Represented as modal Noise level; Represented as an index variable, it is used to traverse the data uncertainty of all modes; express The standard deviation of the mode (or data channel), which measures the degree of dispersion or uncertainty of the modal data.

[0060] As an option, the modal noise level It can be calculated through the standard deviation or variance estimate of multimodal data, thus reflecting the signal-to-noise ratio of different modal data.

[0061] In some embodiments, in order to improve the robustness of partition optimization, the partition boundary can be dynamically adjusted in combination with the boundary smoothing term. Specifically, the optimization goal of the boundary smoothing constraint is to minimize the gradient change of the partition boundary, and its optimization direction is: ; in, Expressed as Smooth boundaries at iterations; Denoted as the learning rate of the smoothing term; is represented as a boundary smoothing term.

[0062] In another implementation, the partitioning result can be corrected in combination with the modal interaction verification item. The goal of modal interaction verification is to ensure the consistency of different modal data on the partition boundary, and its correction method is based on the boundary probability distribution between modalities. The specific formula is: ; in, The boundary conditions are The posterior probability when ; Represented as modal data at the boundary Likelihood probability under conditions; Expressed as a priori probability of the boundary; Expressed as the joint probability of the modal data.

[0063] Through modal interaction verification, abnormal areas in the partitioning results can be identified and corrected.

[0064] The optimization iteration process usually ends when the following stopping conditions are met: The change in the partition boundary is less than the preset threshold; The energy value of the objective function converges to a local or global minimum; The number of iterations reaches the preset upper limit.

[0065] In summary, this step achieves the consistency fusion of multimodal data, the smoothness optimization of partition boundaries, and the reliability guarantee of modal interaction verification through the optimization of the objective function. Through reasonable optimization strategies and parameter settings, the accuracy and robustness of the partition results can be ensured, providing a reliable basis for the subsequent partition result output.

[0066] For S5, the target area partition based on the optimization result is output. The main task of this step is to organize the optimized partition results and output them in an appropriate form for subsequent application scenarios, such as target area monitoring, security and fire monitoring. In general, the output partition results include the partition boundary and the pixel set within the partition, aiming to provide a description of the spatial position, shape and target area characteristic data of the partition. This step needs to be closely connected with the aforementioned optimization processing step (S4) and further improve the availability of the partition results.

[0067] Specifically, in this embodiment, step S5 includes the following contents: In general, the optimized partition boundaries It is represented in the form of pixel-level boundary information. In order to facilitate subsequent application processing, the partition boundary can be further converted into a polygonal form or a point set form. In one implementation, the polygonal representation of the partition boundary can be achieved by a boundary tracing algorithm. For example, an algorithm based on Canny edge detection can extract a connected point set of the partition boundary and further generate a polygonal contour representation. Specifically, the partition boundary can be represented as: ; in, Represented as vertex coordinates on the partition boundary; The number of vertices that represent the partition boundary.

[0068] As an option, the partition boundaries can also be smoothed to further eliminate boundary noise or jaggedness that may occur during the optimization process. Generally, the smoothing process can be performed using Gaussian filtering or B Specifically, Gaussian filtering eliminates local irregular points by performing convolution operations on the coordinates of boundary points. B Spline fitting describes the partition boundaries by generating smooth curves.

[0069] In the generation of partitioned region characteristic data, the main task is to analyze and count the pixel sets inside the target area, so as to extract regional features and associate them with the partition boundaries. In general, the partitioned region characteristic data includes the average characteristic value, standard deviation and pixel distribution information in the region. For example, for infrared image data, the average thermal radiation intensity of the partitioned area can be calculated by the following formula: ; in, It is expressed as the average thermal radiation intensity of the target area; It is expressed as the number of pixels within the partition area; Represented as infrared image data at pixel points The observed value at .

[0070] As an option, the distribution characteristics of regional characteristics can be further extracted, such as calculating the variance or frequency distribution of pixel values ​​in the region. By generating these characteristic data, a more detailed description can be provided for the monitoring or analysis of the target area. In the result formatting output, the partitioning results need to be converted into an output format suitable for the specific application scenario. For example, in the target area monitoring, the partitioning results can be output in the form of image addition, that is, the partition boundary is superimposed with the original infrared image or visible light image to generate a partition annotation map. In one implementation, the partition annotation map can be implemented by the following formula: ; in, Represented as output image; Original image (infrared image or visible light image); Expressed as color values ​​or intensity increments used to label partition boundaries.

[0071] In another implementation, the partitioning result can be output in the form of vector data, such as GeoJSON format or SVG format, for spatial analysis or visualization. The advantage of this formatted output is that it can be easily integrated with a geographic information system or other spatial data analysis tools.

[0072] By representing the partition boundary, generating the regional characteristic data and formatting the output results, efficient output and flexible application of the partition results are finally achieved. In some embodiments, the output partition results can be directly used for classification, status monitoring or risk assessment of the target area. For example, in a fire monitoring application, the high temperature hot spot area can be quickly identified and a warning signal can be issued based on the distribution of thermal radiation intensity in the partition area.

[0073] In summary, this step not only enhances the availability of partition data by converting the partition optimization results into a partition result form that is easy to apply and understand, but also provides reliable technical support for the actual application of the target area. Through the accurate representation of partition boundaries and the generation of regional characteristic data, this step is of great significance in the entire partition data processing method and is a key link in achieving accurate analysis and processing of the target area.

[0074] See also Figure 2 The present invention also provides a partition data processing system for infrared images of a specific area, comprising: Multimodal sensor module for collecting infrared images, visible light images and lidar point cloud data; Data calibration module, used to perform spatial registration and temporal synchronization on the collected multimodal data: Partition modeling module, used to construct the partition processing objective function of the target area: The optimization solution module is used to model the objective function based on the Lagrange multiplier method, and iteratively optimize the partition boundaries through the gradient descent method, dynamically adjust the modal weights and optimize the target area partitions; Partition verification module, used to evaluate the reliability of partition results through modal interaction verification: The result output module is used to output the partition boundary of the target area and the pixel set within the partition. The partition result can be used for monitoring, security or fire monitoring scenarios of the target area.

[0075] Multimodal sensor module It is used to collect multimodal data of a specific area, including infrared images, visible light images and LiDAR point cloud data, and is the core module of data input in the system. The infrared image sensor collects thermal radiation information of the target area to monitor temperature distribution; the visible light image sensor obtains the color and texture characteristics of the area to identify surface details; the LiDAR sensor captures the three-dimensional depth information of the target area to build a spatial geometric model of the area. Through the collaborative work of these sensors, comprehensive and multi-dimensional raw data support can be provided for subsequent data calibration and partition processing.

[0076] The data calibration module is responsible for spatial registration and time synchronization of data collected by multi-modal sensors to ensure that data of different modalities can be processed and analyzed in the same coordinate system. The spatial transformation matrix between the infrared image and the visible light image is obtained through the calibration plate calibration method, and the lidar point cloud data is mapped to the image plane coordinate system through the projection matrix to achieve precise spatial alignment. At the same time, the time synchronization of the data is achieved through hardware trigger signals or timestamp alignment methods, and combined with noise filtering and interpolation processing, the consistency and quality of the data are improved to provide high-precision input data for subsequent partition modeling.

[0077] Partition Modeling Module The objective function used to construct the partition processing objective function of the target area is the mathematical basis for the system to achieve partition optimization. The objective function consists of three parts: data consistency term, boundary smoothing term and modal interaction verification term, which are used to measure the consistency of multimodal data in the partition area, constrain the smoothness of the partition boundary and evaluate the interaction consistency of data between modalities. By reasonably setting various weight parameters, the characteristic requirements of the partition area can be comprehensively reflected, providing clear evaluation criteria and optimization direction for subsequent optimization processing.

[0078] Optimization solution module Responsible for optimizing and solving the objective function to determine the optimal partition boundary of the target area. This module models the objective function through the Lagrange multiplier method, introduces partition consistency constraints, and uses the gradient descent method to iteratively optimize the partition boundary. At the same time, the module can dynamically adjust the modal weight according to the noise level of the modal data to optimize the impact of different modal data on the partition result. Through continuous iterative optimization, until the energy value of the objective function converges to the optimal, a high-precision partition boundary is generated, laying the foundation for partition verification and result output.

[0079] Partition verification module It is used to evaluate the reliability of the partition results and ensure that the final output partition boundaries can accurately reflect the consistency of multimodal data. This module analyzes the performance of partition boundaries in different modes through modal interaction verification, calculates the posterior probability distribution of partition results using the Bayesian inference model, identifies abnormal areas, and provides correction suggestions, thereby improving the robustness and credibility of the partition results.

[0080] Result output module It is used to output the final partition results in a form suitable for the application scenario, including the partition boundaries and the characteristic data within the partition area. The partition boundaries can be output in the form of polygons, point sets or vector data (such as GeoJSON or SVG format) for visualization and further analysis. The characteristic data of the partition area includes thermal radiation intensity, texture characteristics and depth information, etc., providing direct decision support for applications such as monitoring, security and fire monitoring. Through flexible output formats and content, this module effectively enhances the practicality and extensibility of the partition results.

[0081] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for processing partition data of infrared images of a specific area, characterized in that: The following steps are involved: Acquire data from multimodal sensors; Calibrate and preprocess the acquired multimodal data; constructing an objective function of the partitioning process, wherein the objective function is used to measure the characteristics of the partitioned area; Optimize the objective function and determine the partition boundaries; Output the target region partition based on the optimization results.

2. The method for processing regional data of infrared images of a specific area according to claim 1, characterized in that: The multimodal sensor comprises: Infrared image sensor, used to collect thermal radiation information; Visible light image sensor for collecting texture and color information; LiDAR sensor, used to collect depth and spatial information.

3. The method for processing regional data of infrared images of a specific area according to claim 1, characterized in that: The multimodal data calibration includes: Spatial registration and temporal synchronization of trimodal data: Spatial registration realizes the registration of infrared images and visible light images through feature matching algorithm: The LiDAR data is mapped to the image plane coordinate system through the point cloud projection matrix.

4. The method for processing regional data of infrared images of a specific area according to claim 1, characterized in that: The objective function includes a data consistency term, a boundary smoothing term and a modal interaction verification term, and its expression is: ； in, represents the total energy of the objective function; Represented as partitioned areas; Indicates the partition boundary; Represents the data consistency term, which is used to measure the characteristic consistency of different modal data in the target area; Represents the boundary smoothing term, which is used to constrain the smoothness of the partition boundary; Represents the modality interaction verification item, which is used to measure the data consistency between modalities; represents the weight of the smoothing term; Represents the weight of the modal interaction validation item.

5. The method for processing partition data of infrared images of a specific area according to claim 1, characterized in that: The optimization process of the objective function comprises the following steps: The objective function is modeled based on the Lagrange multiplier method, and the partition consistency constraint is introduced. The expression of the objective function is: ； in, Expressed as Lagrangian objective function; It is expressed as a consistency constraint, which is used to constrain the boundary consistency of data between modalities; The partition boundary is iteratively solved using the gradient descent method. The update expression of the partition boundary is: ； in, Indicates The partition boundaries at the iteration; represents the learning rate; represents the gradient of the objective function; Dynamically adjust the modal weight in the objective function. The modal weight is determined according to the noise level of the modal data. The calculation formula of the modal weight is: ； in, Representing modality The weight of Representing modality Noise level; Represents an index variable; Indicates The standard deviation of each mode; The objective function is continuously optimized until the energy converges to the optimal value and the final partition boundary is generated.

6. The method for processing regional data of infrared images of a specific area according to claim 1, characterized in that: The partitioning process comprises the following steps: Determine the partition boundary of the target area and the pixel set within the partition; Generate corresponding characteristic data based on the partition result of the target area; The characteristic data is analyzed and processed and then outputted, and the partition results are used for monitoring, security or fire detection of the target area.

7. The method for processing partition data of infrared images of a specific area according to claim 4, characterized in that: The data consistency term is calculated according to the dynamic modal weight, and the expression of the data consistency term is: ； in, Indicates data consistency item; Representing modality data; represents the estimated value of the characteristic of the target area within the partition; Representing modality The weight of .

8. The method for processing partition data of infrared images of a specific area according to claim 4, characterized in that: The modal interaction verification item verifies the consistency of the segmented region by analyzing the boundary probability distribution between the modalities, and evaluates the consistency of the modal boundaries based on the Bayesian inference model.

9. The method for processing partition data of infrared images of a specific area according to claim 1, characterized in that: The target area partitioning comprises the following steps: Construct initial partition boundaries of the target area; Based on the characteristics of multimodal data, data consistency analysis is performed on the initial partition boundaries; Use optimization algorithms to dynamically adjust partition boundaries so that they match the characteristic distribution of the target area; The partition boundaries of the target area are modified in combination with the modal interaction verification results to generate the final partition results.

10. A partition data processing system for infrared images of a specific area, applied to a partition data processing method for infrared images of a specific area as claimed in any one of claims 1 to 9, characterized in that: include: Multimodal sensor module for collecting infrared images, visible light images and lidar point cloud data; Data calibration module, used to perform spatial registration and temporal synchronization on the collected multimodal data: Partition modeling module, used to construct the partition processing objective function of the target area: The optimization solution module is used to model the objective function based on the Lagrange multiplier method, and iteratively optimize the partition boundaries through the gradient descent method, dynamically adjust the modal weights and optimize the target area partitions; Partition verification module, used to evaluate the reliability of partition results through modal interaction verification: The result output module is used to output the partition boundary of the target area and the pixel set within the partition. The partition result can be used for monitoring, security or fire monitoring scenarios of the target area.

Images