A method for digital image processing based on infrared polarized light imaging

Abstract

The invention belongs to the technical field of image processing, and specifically discloses an image digital processing method based on infrared polarized light imaging, comprising: setting a polarization angle in consideration of the category of a target scene, and acquiring an infrared image and a polarized image; setting a visible parameter for distinguishing a target object from surrounding objects, and extracting multidimensional polarization information of the infrared image according to the difference in the visible parameter; generating first weighted information and second weighted information through the multidimensional polarization information, so as to fuse the polarized image and the infrared image and obtain a multimodal image; identifying a target object in the multimodal image, generating digital information of the target object, and processing the multimodal image; and having the following advantages: through the acquisition of multi-angle polarized light and infrared images, visual parameter difference analysis and two fusion processes, effectively reducing noise and highlighting target features in a complex environment, achieving accurate identification, tracking and digital output of the target object, and providing efficient real-time support for emergency monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image digital processing method based on infrared polarized light imaging. Background Art

[0002] In various emergencies, it is necessary to monitor and locate target objects (such as high-speed moving objects, abnormally hot objects or other characteristic targets). Traditional monitoring methods mainly rely on satellite remote sensing, drone monitoring and ground sensors, but these methods have certain limitations in response speed, coverage and real-time performance. Although infrared imaging technology can effectively capture the position of target objects by detecting thermal radiation, it is easily interfered by background noise in complex environments, resulting in reduced accuracy in positioning and tracking of target objects.

[0003] In this case, how to monitor and locate the target object in a larger range and with higher time resolution has become a current technical difficulty. Especially when the environmental conditions are complex, or the target object is in unfavorable conditions such as insufficient light or obstacles, a single monitoring method is difficult to meet the requirements of high precision and real-time performance.

[0004] Therefore, an image digital processing method based on infrared polarized light imaging is proposed to solve the above-mentioned problems. Summary of the invention

[0005] The present invention aims to provide an image digital processing method based on infrared polarized light imaging to solve or improve the above-mentioned technical problem of poor positioning and tracking accuracy of target objects by traditional monitoring means in complex environments.

[0006] In view of this, a first aspect of the present invention is to provide an image digitization processing method based on infrared polarized light imaging.

[0007] A second aspect of the present invention is to provide a system.

[0008] The first aspect of the present invention provides an image digitization processing method based on infrared polarization imaging, comprising the following steps: setting at least one polarization angle considering the category to which the target scene belongs, and acquiring infrared images and polarization images of the target scene at all polarization angles in real time; setting visual parameters for distinguishing a target object in the target scene from surrounding objects in the target scene, and extracting multidimensional polarization information of the infrared image according to the difference in the visual parameters at each moment; generating first weighted information between the polarization images and second weighted information between the infrared image and the polarization image through the multidimensional polarization information; fusing all polarization images and infrared images according to the first weighted information and the second weighted information at each moment to obtain a multimodal image; identifying the target object in the multimodal image at continuous moments; generating digital information of the target object when the target object exists, and processing the multimodal image according to the digital information; packaging the digital information and the processed multimodal image and outputting them.

[0009] In any of the above technical solutions, the visible parameters include brightness values ​​of the infrared image at different polarization angles to characterize the surface characteristics, temperature characteristics, and observation states of light reflection and scattering behaviors of the target object on the image.

[0010] In any of the above technical solutions, the step of extracting the multidimensional polarization information of each infrared image at each moment based on the difference in the visible parameters includes: extracting the brightness value of the infrared image at all polarization angles at the same moment; calculating the polarization degree, polarization angle and polarization state matrix of the light wave in the infrared image through all the brightness values; and forming the multidimensional polarization information at the current moment with the polarization degree, polarization angle and polarization state matrix.

[0011] In any of the above technical solutions, the step of fusing the multimodal images includes: generating the first weighted information of each of the polarization images through the polarization degree, assigning weights to each of the polarization images through the first weighted information and synthesizing the fused image; generating the second weighted information through a matrix constructed by the brightness value, the polarization degree and the polarization angle, assigning weights to the fused image and the infrared image respectively through the second weighted information and synthesizing the multimodal image.

[0012] In any of the above technical solutions, before the fused image is merged with the infrared image, the fusing step of the fused image also includes: setting optimization parameters according to the observation state of the target object on the image, and using weighted least squares method to correct the contours of all objects in the fused image according to the optimization parameters.

[0013] In any of the above technical solutions, the observation state of the target object in the image is also used to divide and determine the category of the target scene.

[0014] In any of the above technical solutions, after the fused image is merged with the infrared image, the fusing step of the fused image also includes: constructing a Laplacian pyramid of the infrared image and the polarization image, and fusing and reconstructing the multimodal image at each resolution scale to correct the position and shape of all objects in the multimodal image.

[0015] In any of the above technical solutions, the step of generating the digital information of the target object includes: obtaining the position information of the identified target object at the current moment, estimating and generating the position estimation information at the previous moment, and correcting the position estimation information at the previous moment; using the position information at the current moment, the position estimation information at the previous moment and the corresponding contour information as the digital information of the target object.

[0016] In any of the above technical solutions, the step of processing the multimodal image according to the digitized information includes: generating contour information of the target object through the position information and the position estimation information, and marking all contour information on the multimodal image.

[0017] The second aspect of the present invention provides a system for executing the image digitization processing method in any of the above-mentioned technical solutions, comprising: a data transceiver module, used to receive the infrared image and the polarization image, and output the digitized information and the processed multimodal image; an image fusion module, used to merge the infrared image and the polarization image into the multimodal image; an image processing module, used to generate digitized information of the target object and process the multimodal image.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] Since one or more polarization angles are flexibly selected according to the target scene category, and the infrared image and polarization image are synchronously or quickly switched to be acquired at all selected polarization angles, the present invention can be compatible with more diverse and complex environmental requirements. The multi-angle acquisition mode has better adaptability to high-speed moving objects, complex material surfaces and low-light shielding environments, thereby reducing the risk of missed detection and false detection.

[0020] By setting visual parameters specifically used to distinguish the target object from surrounding objects, and extracting multi-dimensional polarization information of infrared images based on the differences in these parameters at each moment, the present invention can highlight the key features of the target object in scenes with high noise or severe interference. It effectively eliminates cluttered backgrounds or other irrelevant heat sources, significantly improves the visibility of the target object in the image, and reduces the amount of calculation and interference required for subsequent recognition or tracking algorithms.

[0021] In the fusion step, the present invention uses two types of weighted information to achieve two fusions: first, all polarization images are fused with the first weighted information to obtain a polarization fusion image; then the polarization fusion image is fused with the infrared image according to the second weighted information to finally generate a multimodal image. It can better retain the advantages of polarization images for surface reflection / scattering characteristics, and at the same time use the strong recognition of infrared images for temperature / heat distribution, so that the shape, material, temperature, movement and other information of the target object can be more completely and clearly presented in the final image.

[0022] Target object recognition is performed at continuous moments, and the detected target object information is digitized and superimposed on the image output, so that it can not only provide intuitive visualization, but also package the target location, contour, temperature and other attributes in the form of data to the back-end device or application. This approach not only supports continuous and real-time tracking of target objects, but also provides an efficient interface for subsequent archiving, playback, statistical analysis, automatic intervention, etc., meeting the timely response and data sharing requirements required for dealing with emergencies or complex application scenarios.

[0023] Based on the multi-channel fusion of infrared and polarization information, the present invention can automatically combine the characteristic advantages of different image sources to compensate or offset when encountering various interferences (such as environmental heat source clutter, strong reflection of complex surfaces, insufficient light, etc.), and obtain stable output that is difficult to achieve in a single mode. Multi-dimensional and multi-angle information reinforcement not only reduces the impact of noise and artifacts, but also provides technical guarantees for the system to maintain a high recognition accuracy in harsh environments.

[0024] Additional aspects and advantages of embodiments according to the present invention will become apparent in the following description or may be learned through practice of embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0026] Figure 1 is a flow chart of the method steps of the present invention;

[0027] Figure 2 It is a flowchart of a specific implementation case of the present invention;

[0028] Figure 3 It is a system logic block diagram of the present invention;

[0029] Figure 4 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0032] See also Figure 1-Figure 4 , the following describes an image digitization processing method based on infrared polarized light imaging according to some embodiments of the present invention.

[0033] The embodiment of the first aspect of the present invention provides an image digitization processing method based on infrared polarized light imaging. In some embodiments of the present invention, Figure 1 As shown, the method comprises the following steps:

[0034] S101, setting at least one polarization angle in consideration of the category to which the target scene belongs, and acquiring infrared images and polarization images of the target scene at all polarization angles in real time.

[0035] Here, the image digitization processing method based on infrared polarized light imaging needs to set at least one polarization angle according to the category of the target scene before acquiring the infrared image and polarization image of the target scene at all polarization angles in real time. The core goal of this step is to select a suitable polarization angle or multiple polarization angles for different types of target scenes to ensure more accurate target positioning and tracking effects in the subsequent fusion and processing stages. In this context, the target scenes may include the following example scenes:

[0036] High-speed moving object scenes (such as vehicles, drones, etc.): require higher temporal resolution and more sensitive angle settings.

[0037] Abnormal heating scene (such as overheated areas of industrial equipment, suspected fire points, etc.): Special attention should be paid to the saturation of infrared images in high-temperature areas, and specific polarization angles should be combined to highlight the target outline.

[0038] Scenes with special surface materials (such as metal reflections, smooth surfaces, and transparent materials): You may need to capture images at more polarization angles to better distinguish between environmental reflections and object surface features.

[0039] Low-light or obstructed scenes: In situations where there is insufficient light or many obstructions, infrared imaging can provide thermal radiation information, and the choice of polarization angle can further reduce interference from cluttered backgrounds.

[0040] According to the scene characteristics (such as the target object's movement speed, material reflectivity, ambient light, the presence of multiple obstacles, etc.), one or more polarization angles can be flexibly set. At this time, it is no longer limited to the conventional four angles of 0°, 45°, 90°, and 135°, but any angle combination that better fits the target scene can be selected. For example, it can be 10°, 52°, 73°, 129°, etc., or it can be a continuous angle range that is adaptively and dynamically adjusted to capture the strongest response of the object surface to a specific polarization state. For example: in an industrial workshop scene where liquid leakage may be monitored, due to the special reflectivity and refractive index of the liquid surface, the common 45° or 135° cannot fully capture the reflection difference. At this time, according to the experiment or sensor calibration, you can set angles such as 30°, 60°, 120°, 150°, etc. to observe, which may obtain more obvious liquid level differences. In a highway traffic scene, the vehicle's reflective points may be related to the curved surface structure of the vehicle body. If it is found through experiments that the two angles of 20° and 95° can maximize the difference between the vehicle's curved surface reflection and the background, then the polarization angle can be set to 20° and 95° in this step to capture better vehicle contour features later.

[0041] Once the polarization angle combination for the target scene is set, use the infrared polarization imaging device to synchronize or quickly switch these angles for shooting. For each polarization angle, two types of image information will be generated at the same time: infrared image (mainly used to capture the thermal radiation information of the target scene), which can intuitively display the heating area or temperature difference in low light or obstacle environment; polarization image (highlighting the polarization reflection and scattering characteristics of light on the surface of different objects), making certain surfaces, materials or motion trajectories easier to distinguish.

[0042] As can be seen from the above, the polarization angle is determined, and infrared and polarization images are obtained at each selected angle, so as to prepare the input for subsequent extraction of polarization degree, polarization angle, polarization state matrix (Stokes parameters) and multi-angle fusion operations. If there is only a single angle, the later algorithm will be stretched in suppressing noise and highlighting the target outline; and multiple angles can significantly improve the accuracy and robustness of target positioning by differential or fusion means in subsequent steps. It is no longer limited to a few common fixed angles and can be closely integrated with the needs of complex scenes. For example, if obstacles or optical interference in the scene mainly occur at a specific angle, then the polarization information at that angle will be more valuable; or for high-speed objects, only two key angles need to be collected to reduce redundant data and increase the frame rate. In this way, real-time and scene adaptability are guaranteed. When infrared images are affected by environmental heat sources, reflectors, etc. at certain angles, they will lose clarity at the target boundary; but the sensitivity of polarization images to these thermal noises may be completely different from infrared imaging. Through multi-angle acquisition, those angle data that are significant for the target object can be selected in the subsequent contrast enhancement or weighted fusion strategy to enhance the target features, suppress background noise, and make target positioning and tracking more accurate. Because the position change or state of the target object (such as possible trajectory, temperature development trend, etc.) needs to be quickly evaluated in emergencies. By optimizing at the hardware level (such as high-speed electric filter rotation or multi-filter array) and software level (such as multi-threaded parallel processing), multi-angle data capture can be completed in a short time, so that the overall system can be maintained in a high frame rate and high-resolution monitoring state. In this way, it can capture high-speed moving targets and meet the needs of detail extraction in complex environments.

[0043] Furthermore, the specific steps of S101 include image acquisition, denoising, contrast enhancement and geometric correction.

[0044] Image acquisition: An infrared camera equipped with an adjustable polarization filter is used to synchronously acquire infrared images at different polarization angles (represented as 1, 2, 3, and 4) in the target scene, which are respectively .

[0045] Denoising: Using wavelet transform denoising algorithm, the image is decomposed into sub-bands of different frequencies through wavelet decomposition:

[0046]

[0047] in, The value contained is the brightness value of the original image at position (x, y); is the denoised image; is the scale and direction number of wavelet decomposition; Wavelet coefficients represent the detailed information of the image at level i and direction j; It is a wavelet basis function, which is used to represent the different frequency and direction components of an image.

[0048] Contrast enhancement: Adaptive Histogram Equalization (AHE) is applied, the formula is as follows:

[0049]

[0050] in, This is the contrast-enhanced image: It is an adaptive histogram equalization operation that adjusts the contrast of each pixel according to the local image block.

[0051] Geometric correction: using camera calibration matrix Perform geometric correction using projection transformation:

[0052]

[0053] in, is the coordinate of the corrected image; is the camera internal parameter matrix, including parameters such as focal length, principal point coordinates and distortion coefficient; is the three-dimensional coordinate in the actual scene.

[0054] S102, setting a visible parameter for distinguishing a target object in a target scene from surrounding objects in the target scene, and extracting multi-dimensional polarization information of the infrared image according to the difference in the visible parameter at each moment.

[0055] Here, after setting at least one polarization angle according to the target scene in step S101 and acquiring multi-angle infrared images and polarization images in real time, this step (S102) further proposes a key concept - visual parameter. Visual parameter can be understood as a quantitative index used to distinguish the difference between the target object and the surrounding objects or background. Since the present invention is aimed at the detection and positioning of target objects in complex scenes and emergencies, it is necessary to find key information that helps to distinguish the difference between the target object and the surrounding objects in the multidimensional data of the infrared polarization image. Therefore, in this step, the system will first set the visual parameters, and then monitor or calculate these parameters at each moment, and use the difference in parameters to extract the multidimensional polarization information of the infrared image. Through this step, the target object can be accurately separated from the surrounding environment, providing a clearer and more operational judgment basis for the next step of image fusion, enhancement and positioning algorithm. This step helps the method to greatly reduce the impact of background noise on target detection in extremely complex and changeable scenes, and highlight the significant features of the target object in the infrared and polarization dimensions.

[0056] By analyzing the properties of the target scene and surrounding objects, environmental interference, and imaging requirements, a set of indicators or numerical features that help distinguish the target object from the background environment are defined or selected. These indicators are not limited to a single brightness or temperature, but may also be a composite quantity of multiple dimensions. For example, degree of polarization (DoP), angle of polarization (PoA), infrared brightness contrast, temperature gradient, reflectivity difference, etc. can all be regarded as visible parameters in the present invention. For example: in an abnormal heating body monitoring scene, the visible parameter may be the difference between thermal radiation intensity (infrared brightness value) and degree of polarization: when the target object is a high-temperature device and has certain polarization reflection characteristics, it is different from the surrounding background in both aspects. In high-speed moving target monitoring, visible parameters may include motion vector information (such as optical flow) based on infrared images and reflection polarization differences in polarization images; in this way, high-speed motion and high polarization difference points can be found simultaneously in the image sequence.

[0057] When the system is performing real-time or quasi-real-time monitoring, how to judge the difference between the target object and the surrounding objects in the infrared polarization image based on the previously set visual parameters. In particular, when the target object is significantly different from the environment in some polarization dimensions or infrared brightness distribution, this difference is used as multi-dimensional polarization information extracted and retained by the system. Multi-dimensional polarization information does not refer to the difference between two frames of images, but may include a variety of imaging-derived information such as degree of polarization (DoP), polarization angle (PoA), polarization state matrix (Stokes parameters), and combined with dimensions such as thermal distribution (infrared brightness / temperature) for multi-modal or multi-parameter cross-comparison. For example: assuming that a target object has special polarization reflection characteristics (such as metal reflection or smooth surface), the polarization degree will increase significantly at a specific polarization angle, while there will be no obvious reflection at other angles; the system confirms that this area is the location of the target object by observing the visual parameters (polarization degree difference or polarization angle significant jump), and extracts relevant polarization information, so that the subsequent algorithm can focus on this area for further identification or positioning. If the target object is in a night search and rescue mission and has high thermal radiation, the temperature distribution of the object in the infrared image will be different from the background, and it may also show a unique scattering pattern in the polarization image; the system can combine infrared brightness and polarization degree as visual parameters, extract these difference information in each time frame, and map it into multi-dimensional polarization information for subsequent fusion or analysis.

[0058] From the above, we can see that in a complex environment, there may be multiple heat sources or messy light reflection sources in the background. If the system does not have a clear definition of visual parameters, it is easy to be disturbed by background noise. By setting the visual parameters first and then detecting or calculating these visual parameters at each moment, the area or pixel that best matches the characteristics of the target object can be screened out, reducing the amount of invalid data processing. Furthermore, multi-dimensional polarization information extraction can be regarded as a simplification process of the original infrared image and the polarization image, retaining only the difference data that plays a key role in distinguishing the target object, thereby improving the efficiency and accuracy of subsequent algorithm operation.

[0059] Fusion processing (such as image weighted fusion, Laplace pyramid fusion, multi-scale polarization fusion, etc.) is often required in subsequent steps. However, if there is no quantitative indicator of the difference between the target object and the background, it will be difficult for the fusion algorithm to focus on strengthening the target features and suppressing irrelevant areas. By confirming the visual parameters and extracting multi-dimensional polarization information in this step, the subsequent fusion or enhancement algorithm can adjust the weight or brightness more intelligently based on this information, so that the fusion result achieves the best contrast or clarity in the target object area.

[0060] When the system needs to track the target object in continuous frames or locate it in real time, relying solely on infrared brightness information may not be enough, especially when there are multiple heat sources or optical reflection points in the environment, the false detection rate may be high. Multi-dimensional polarization information helps to distinguish objects with similar temperatures but different materials or surface characteristics, significantly improving the tracking stability and confidence of the system. Differences in visual parameters can also be used to determine changes in the target object's motion trajectory or deformation in a time series. For example, on a high-speed moving feature target, its polarization reflection changes with direction, and the visual parameters will reflect these changes in real time, allowing the system to determine whether the object is rotating, deformed, or obscured.

[0061] For various emergencies, such as equipment with abnormally high temperature, suspicious moving objects, etc., different scenes may correspond to different definitions and emphases of visual parameters; by selecting appropriate visual parameters in this step and monitoring them in real time in the time series, the target object can be quickly discovered and captured, thereby expanding the application of the present invention to a wider range of actual situations.

[0062] In any of the above embodiments, the visible parameters include brightness values ​​of the infrared image at different polarization angles to characterize the surface characteristics, temperature characteristics, and observation states of light reflection and scattering behaviors of the target object on the image.

[0063] In this embodiment, the visual parameters act as a bridge and link, converting the infrared brightness values ​​at different polarization angles into an observational description of the various physical properties of the target object. Specifically, when the infrared brightness difference of the target object at different polarization angles corresponds to its surface characteristics (such as smooth, rough, metal or medium surface), temperature characteristics (thermal radiation strength) and optical reflection / scattering characteristics (including polarization degree, polarization angle, etc.), the system can define the brightness values ​​at these polarization angles as visual parameters and use them in the subsequent processing stage.

[0064] The visible parameters specifically include the brightness information of each pixel in the multi-frame or multi-channel images. The brightness value of the infrared image at different polarization angles means that the system captures the infrared light under a pre-set set of polarization angles (which may be a flexible combination of 10°, 52°, 90°, 133°, etc.) and records the brightness of the light at each pixel. These brightness values ​​essentially contain the reflection, refraction or scattering effects of the target object at different angles, and also have the temperature distribution information unique to infrared imaging. In this way, if the brightness value increases or decreases abnormally at certain angles, it may correspond to a change in a physical property of the target object (such as high temperature areas, strong reflections from smooth surfaces, etc.).

[0065] The purpose of collecting and analyzing the brightness values ​​of infrared images at different polarization angles is to infer or characterize the deeper physical characteristics of the target object from the appearance of the image. In other words, the visual parameters are not only a bunch of brightness numbers, but also reveal specific physical information such as surface characteristics (smooth / rough / material type), temperature characteristics (heat distribution / abnormal high temperature / local overheating), light reflection and scattering behavior (high reflection / low reflection / polarization angle sensitivity). For the system, these physical characteristics are the key basis for judging the difference between the target object and the surrounding environment: high temperature means obvious infrared characteristics, strong polarized reflection means that it may be a metal or mirror surface; no significant changes at multiple angles may indicate that the target object has uniform material or uniform temperature.

[0066] Specifically, the step of extracting the multi-dimensional polarization information of each infrared image according to the difference in the visible parameters at each moment includes:

[0067] Extract the brightness values ​​of infrared images at all polarization angles at the same time.

[0068] The polarization degree, polarization angle and polarization state matrix of light waves in infrared images are calculated through all brightness values.

[0069] The polarization degree, polarization angle and polarization state matrix constitute the multi-dimensional polarization information at the current moment.

[0070] In view of the above specific description, ensure that the system collects the brightness value of the infrared image at each predetermined polarization angle as the basic data for extracting polarization characteristics. At different polarization angles, the brightness value of the same target object may be different due to the different reflection and absorption characteristics of materials, structures and light sources. By mathematically processing the brightness values ​​collected at different polarization angles, the degree of polarization and the angle of polarization, as well as the polarization state matrix that integrates these data, are calculated. The degree of polarization and the angle of polarization provide a quantitative description of the target object's response to the polarization of the light wave, while the polarization state matrix provides a comprehensive description, including polarization state and light wave phase information. The calculated degree of polarization, angle of polarization and polarization state matrix are combined into a set of comprehensive multi-dimensional polarization information. This not only provides a comprehensive view of the optical and thermal properties of the object at each observation moment, but also enables this information to be used for more complex analysis and decision support in subsequent steps.

[0071] From the above, we can see that at each moment, the steps of extracting multi-dimensional polarization information through the difference in visual parameters are to use the brightness value of the infrared image at different polarization angles to calculate the degree of polarization, polarization angle and polarization state matrix, and combine this information to form a multi-dimensional data set. This process provides real-time and detailed physical property analysis for infrared polarization imaging technology, and provides strong data support for image processing and target monitoring in practical applications.

[0072] Furthermore, obtaining the degree of polarization, the polarization angle and the polarization state matrix includes: calculating the degree of polarization (DoP), calculating the polarization angle (PoA) and constructing the polarization state matrix (Stokes Parameters).

[0073] Calculate the degree of polarization (DoP), which indicates the degree of polarization state of the light wave. The calculation formula is:

[0074]

[0075] in, The values ​​included are the brightness values ​​of infrared images under different labels; The degree of polarization indicates the degree of polarization of light. The range is [0,1]. The larger the value, the stronger the polarization.

[0076] Calculate the polarization angle (PoA), which indicates the polarization direction of the light wave. The calculation formula is:

[0077]

[0078] in, is the polarization angle, which indicates the polarization direction of the light wave and ranges from [0°, 180°).

[0079] The polarization state matrix is ​​constructed, and the Stokes parameters are used to describe the polarization state:

[0080]

[0081] in, is the total light intensity; is the intensity difference of linearly polarized light; is the intensity difference of linear polarized light in the other direction; is the circular polarized light intensity, which is assumed to be 0 here. The polarization state matrix is ​​used for subsequent image fusion and polarization characteristic analysis. By integrating data under multiple labels, it provides comprehensive polarization information and supports high-precision detection and positioning of target objects. S103, generates first weighted information between polarization images and second weighted information between infrared images and polarization images through multi-dimensional polarization information; at each moment, all polarization images and infrared images are fused according to the first weighted information and the second weighted information to obtain a multimodal image.

[0082] Here, when the system has acquired infrared images and polarization images at multiple angles in the previous steps (such as S101 and S102), and extracted multi-dimensional polarization information using visual parameters, it is necessary to fuse these multi-source images to further reduce noise and highlight the characteristics of the target object. In order to achieve high-precision multimodal fusion, this step (S103) relies on the first weighted information and the second weighted information to fuse the information between the polarization images and between the polarization image and the infrared image. This step is also performed in real time or quasi-real time in the time dimension, that is, the weighted fusion operation is performed at each moment, ensuring the instant capture of the target object or scene changes. Through this step, the system finally obtains one or more multimodal images, which have both the thermal radiation advantages of infrared imaging and the ability of polarization imaging to enhance the optical characteristics of objects, providing a high-quality visual foundation for subsequent image enhancement, target positioning and tracking processes.

[0083] When fusing all polarization images (from different polarization angles), it is necessary to first use multi-dimensional polarization information (such as polarization degree, polarization angle, polarization state matrix parameters, and visual parameter differences extracted in the previous step) to calculate a set of first weighted information. This set of first weighted information essentially reflects which image has more weight among images with different polarization angles under the characteristics of the current environment and the current target. For example, if the polarization degree at a certain angle shows the target object most clearly, while the polarization signal at other angles is weaker, then the image at that angle can be given a larger weight when fused. Example: Metal surface scene: Assume that at polarization angle 1 (such as 30°), the polarization degree at the metal reflection is the strongest, which can best highlight the outline of the target; while at another polarization angle 2 (such as 80°), the target outline is relatively unclear due to the mismatch of the incident / reflection angle. At this time, the system can calculate the first weighted information, give the image at angle 1 a higher weight, and amplify it in the fusion process, so that the final fused image can better show the metal reflection characteristics of the target. Fluid surface scene: If the fluctuations and reflections of the fluid surface are strongly captured at polarization angle 1 (such as 45°), but the influence of water disturbances is weak at polarization angle 2 (such as 120°), the system uses multi-dimensional polarization information (such as DoP difference value) to evaluate which angle captures the most recognizable fluctuations and increases the fusion coefficient of the image at that angle in the first weighted information.

[0084] After all polarization images are processed by the first weighted information to obtain a polarization fusion image or a set of polarization fusion layers, they are fused with infrared images for the second time. Because infrared images and polarization images are significantly different in physical sense: infrared images focus on thermal radiation characteristics, while polarization images mainly highlight optical reflection / scattering properties. In order to achieve the complementarity of the two, a set of second weighted information is required to determine the relative importance of infrared channel information and polarization channel information in the final fusion. If the target object is very obvious in infrared (large temperature difference) and slightly weak in polarization signal, the weight of infrared image can be increased in the second weighted information; vice versa. Example: Fire monitoring: In a certain high temperature abnormal area, such as a heating body or burning point, the infrared brightness (temperature) is significantly higher than the surrounding environment; but if the surface material is complex, some details are easily confused with background thermal interference in infrared, while these details may be clearer in polarization images. By calculating the second weighted information, the system can first evaluate the distinction between the current target in infrared and polarization. For example: if the infrared brightness difference is large and the polarization degree is relatively obvious, both are set to higher weights and the contributions of both sides are balanced in fusion. Night search and rescue: If the object of observation is a pedestrian wearing thin clothes, the pedestrian may have a higher brightness in the infrared image, but if the clothes do not have strong polarization characteristics, the polarization image cannot provide more details. At this time, the system may rely mainly on the infrared image through the second weighted information; conversely, if the pedestrian carries some highly reflective backpacks or metal utensils, the polarization image may also show important information, so the second weighted information increases the proportion of the polarization image.

[0085] Through the first two sets of weighted information (the first weighted information is used for the fusion between polarization images, and the second weighted information is used for the fusion between polarization fusion images and infrared images), a unified fusion operation is performed on different source images at each acquisition moment (frame). The resulting multimodal image inherits the strengths of the polarization channel in surface feature and reflection / scattering feature recognition, as well as the advantages of the infrared channel in thermal radiation and temperature difference comparison. This multimodal image often has better distinguishability, clarity and contrast for the target object, providing a solid foundation for subsequent target positioning and tracking, image enhancement, scene analysis and other links.

[0086] As can be seen from the above, with the support of the first weighted information, the system can effectively merge images originally collected at multiple polarization angles, avoiding the loss of some information and suppressing the redundancy or mutual interference between polarization channels, thereby obtaining one or a group of polarization fusion results. In complex scenes, the reflection / scattering characteristics required for the target object can be best preserved in this fusion.

[0087] Infrared imaging is very advantageous for detecting thermal anomalies and body contours in low-light environments, but pure thermal imaging may not be adequate for different materials, surface light reflections, and even certain camouflage forms. Polarized imaging can enhance surface features in some scenarios or help remove certain heat reflection illusions. By effectively allocating the second weighted information during the fusion process, the advantages of both can be presented in a multimodal image, making the target object more prominent in the image and the background noise weakened.

[0088] The steps emphasize that all polarization images and infrared images must be fused according to weighted information at each moment, and the fusion operation must be performed at a higher frame rate and shorter delay. For emergencies or rapidly changing scenes (such as high-speed moving objects, instantaneous heat outbreaks, etc.), multi-modal images that can be updated at all times can provide accurate input for subsequent positioning and tracking.

[0089] Once the infrared features of the target object are weakened due to temperature changes or partial occlusion in some frames, the polarization channel information can be relied on to maintain its significance; if the polarization features are not obvious for a while, the infrared thermal radiation channel can also make up for it in time. By updating the first weighted information and the second weighted information in a timely manner, this fusion can maintain a good expression of the target object's contour and details for a long time, making the tracking algorithm more robust in subsequent steps.

[0090] Specifically, the fusion steps of multimodal images include:

[0091] First weighted information of each polarization image is generated according to the polarization degree, and a weight is assigned to each polarization image according to the first weighted information to synthesize a fused image.

[0092] The second weighted information is generated by a matrix constructed by brightness value, polarization degree and polarization angle, and weights are respectively assigned to the fused image and the infrared image through the second weighted information to synthesize a multimodal image.

[0093] In view of the above specific description, the degree of polarization is used to generate weighted information of the polarization image. The degree of polarization, as a parameter that characterizes the polarization state of light waves, can effectively reflect the material characteristics and light propagation conditions. Each polarization image is assigned different weights based on its degree of polarization. Images with high weights have greater influence in the fusion process, which helps to highlight images with obvious specific polarization characteristics, thereby enhancing the relevant features in the fused image. The second weighted fusion generates weighted information by comprehensively considering the composite matrix of brightness value, degree of polarization and polarization angle. This method takes into account more optical and thermal parameters, so that the final multimodal image can integrate the optical information of the polarization image while maintaining the thermal characteristics of the original infrared image, thereby achieving more comprehensive data fusion.

[0094] From the above, we can see that in the fusion step of multimodal images, through the two-stage fine weighting and intelligent fusion, not only the quality and expressiveness of the image data are improved, but also the adaptability and accuracy in processing complex scenes are enhanced.

[0095] Furthermore, the synthesis step of the fused image includes weight allocation and image fusion.

[0096] Weight allocation, assign weights to images under each label according to the degree of polarization, weight Proportional to DoP:

[0097]

[0098] in, For the i The weight of the labeled image; For the i The degree of polarization of the label; i The value is [0,k].

[0099] Image fusion, fused polarization image The calculation is as follows:

[0100]

[0101] in, is the fused polarization image; For the i Preprocessed images under labels.

[0102] Furthermore, the step of synthesizing a multimodal image by fusing the image and the infrared image includes image registration and multimodal fusion.

[0103] Image registration: The infrared image and the fused polarization image are registered, and feature point matching algorithms (such as SIFT or ORB) are used for geometric alignment to ensure the pixel correspondence between the two images.

[0104] Multimodal fusion is performed by combining weighted average method and principal component analysis (PCA):

[0105]

[0106] in, is the image after multimodal fusion; is an infrared image; is the fused polarization image; is the fusion weight coefficient, which is determined by PCA or other algorithms.

[0107] Specifically, before the fused image is combined with the infrared image, the fusion step of the fused image also includes:

[0108] The optimization parameters are set by the observation state of the target object in the image, and the weighted least squares method is used to correct the contours of all objects in the fused image according to the optimization parameters.

[0109] In view of the above specific description, the optimization parameters are set by analyzing the observation status of the target object in the image (such as brightness, contrast, boundary clarity, etc.). These parameters provide specific adjustment basis for contour correction, such as edge sharpening intensity, contour smoothness, etc., aiming to enhance the visual performance and feature distinction of the target object.

[0110] From the above, we can see that by optimizing the parameter settings and applying the weighted least squares method, the accuracy and clarity of the target object contour can be greatly improved. For multimodal image fusion, clear contours not only help improve the visual effect, but also enhance the analytical value of the image.

[0111] Furthermore, the optimization step of the fused image includes an edge-preserving fusion algorithm.

[0112] Edge-preserving fusion algorithm uses weighted least squares (WLS) to preserve image edge information:

[0113]

[0114] in, To optimize the parameters, used to control the degree of smoothness; is the image gradient, which is used to preserve edge information.

[0115] In any of the above embodiments, the observation state of the target object in the image is also used to classify and determine the category of the target scene.

[0116] In this embodiment, the categories of target scenes include high-speed motion scenes, abnormal temperature scenes, multi-material reflection scenes, insufficient light scenes, environmental shielding interference scenes, and relatively uniform or non-significant feature scenes. Among them, abnormal temperature scenes are scenes where there may be interference from multiple heat sources. When the target temperature is too small or too large compared to the surrounding temperature, it is easy to misjudge, and the abnormal position needs to be determined. For example: Overheating of factory equipment: Combine the temperature distribution of infrared images and the optical characteristics of polarized images to determine the fault point - Forest fire: Monitor the high temperature area of ​​the fire source and distinguish the thermal noise of the surrounding environment.

[0117] Specifically, after the fused image is combined with the infrared image, the fusion step of the fused image further includes:

[0118] A Laplacian pyramid of infrared images and polarization images is constructed, and multimodal images are reconstructed by fusion at various resolution scales to correct the position and shape of all objects in the multimodal images.

[0119] In view of the above specific description, in the processing stage after the infrared image and the polarization image are merged. By constructing the Laplacian pyramid of the infrared image and the polarization image, each image is decomposed into different resolution levels. The image is allowed to retain detailed feature information at different scales. Each layer of the image only contains specific details at that scale, from the finest high-frequency information to the coarsest low-frequency information, and each level is processed separately. Subsequently, in the fusion process, the thermal characteristics of the infrared image are combined with the optical characteristics of the polarization image using a layer-by-layer weighted method, thereby achieving effective integration of information at each resolution level. This hierarchical fusion strategy makes the final reconstructed multimodal image not only more visually coherent, but also more accurately corrects and optimizes the position and morphology of the target object in the image.

[0120] As can be seen from the above, the fused image can more clearly show the detailed outline and structure of the target object, while strengthening the important features in the image, such as boundary clarity and thermal contrast of the target object, greatly improving the image's usability and analysis accuracy. It is suitable for occasions that require high-precision target recognition and fine visual performance, such as security monitoring, disaster assessment, and advanced driver assistance systems.

[0121] Furthermore, the step of fusing and reconstructing the multimodal images includes Laplacian pyramid fusion.

[0122] Laplacian pyramid fusion, constructing the Laplacian pyramid of infrared images and polarization images, and fusing them at various scales, and finally reconstructing the multimodal image:

[0123]

[0124] The specific correction steps are: constructing the Laplacian pyramid of infrared image and polarization image respectively, performing weighted fusion at each level, and reconstructing the fused multimodal image. Laplacian pyramid fusion further optimizes the details and contrast of the multimodal image, making the features of the target object more prominent, and providing high-quality input images for subsequent positioning and tracking.

[0125] S104, identifying a target object in the multimodal image at continuous moments; when the target object exists, generating digital information of the target object, and processing the multimodal image according to the digital information; packaging the digital information and the processed multimodal image and outputting them.

[0126] Here, the previous steps (such as S101, S102, S103) have successively completed the acquisition of multi-angle infrared and polarization images, the extraction of visual parameters, and two weighted fusions to form a multimodal image. At this point, it is necessary to further identify the target object in real time or quasi-real time in the multimodal image of the time series (continuous moments). If the system determines that the target object does exist, it is also necessary to generate digital information and post-process the image to package and output these data for subsequent monitoring, recording or decision-making. This step (S104) is centered around target object identification, digital information generation, multimodal image processing and output, aiming to maximize the value of the multimodal image obtained earlier - not only intuitively displaying the location and characteristics of the target object, but also encapsulating the analysis results within the system into a digital form that can be read by other modules or users. In this way, the system can identify and record the target object in an emergency or complex environment, and quickly output the processing results to support further operations.

[0127] For the multimodal image sequence obtained from the previous step, recognition is performed in the time dimension. In other words, it is not just for the detection of a single frame image, but also for the comparison or tracking of multimodal images at multiple times, so as to ensure the continuous monitoring of the position and state of the target object. The system can use various recognition algorithms, such as target detection based on deep learning (YOLO, Faster R-CNN, etc.), detection based on threshold or visual parameter segmentation, or combined with motion analysis (such as optical flow or state estimation-based methods) to lock the position and contour of suspected target objects from multimodal images. For example: High-speed vehicle monitoring: The system detects the thermal characteristics (infrared) and the metal reflection (polarization) characteristics of the vehicle in each frame in the 30 frames / second multimodal image stream. When the detection algorithm determines that a high-speed moving target object appears at certain times, it can be identified as a vehicle. Abnormal heating body detection: For a heat exchange pipe in a factory, if a high temperature and abnormal reflection characteristics appear in the multimodal image, it is determined that there is a suspicious target object (such as a broken pipe or hot spot). Through time series monitoring, it is possible to track whether the hot spot persists, spreads or transfers.

[0128] Digital information usually refers to the detected target object position, outline, size, temperature distribution, polarization characteristics or other related information packaged into a computer-readable data structure, such as JSON, XML, or a custom binary format. Its core purpose is that in addition to displaying the image, the system also needs to accurately mark the target object in the form of data so that the back-end or external application can perform higher-level processing. Processing multimodal images based on digital information can use this digital information to further emphasize the visibility of the target object area in the image, such as drawing a bounding box for the target object, marking the temperature value, highlighting the outline; or performing noise reduction, masking or blurring on the background that is not related to the target object, etc. For example: Human body search and rescue: When the human body outline is detected in the polarization-infrared fusion image at night, the system generates digital information of the target object, such as "target ID = 001, center coordinates (450, 320), outline polygon, average temperature = 37.2°C, speed = 1.5m / s", etc. The system then uses this information to draw a striking target outline on the multimodal image and mark the temperature value, so that search and rescue personnel can confirm the target location and dynamic information at a glance on the monitoring screen. Equipment safety monitoring: When it is determined that a temperature abnormality or polarization reflection abnormality occurs in a local area of ​​a certain equipment, the system generates data such as "abnormal area number = 01, location = upper pipeline, temperature peak = 135°C, polarization degree = 0.65", and highlights or enhances the contrast of the local area, allowing operators to quickly view the problem point.

[0129] Not only does it generate target object information internally, but it also outputs (possibly through the network, storage device or real-time display) the processed image and its corresponding target data. "Packaging" refers to integrating digital information (target location, attributes, recognition confidence, etc.) and processed images (such as multimodal images with labels) into a unified data structure or communication protocol so that the receiver (monitoring center, cloud, other subsystems) can make response decisions in the future, such as alarm, intervention, and continued tracking. For example: Urban security system: The system can transmit detailed information of detected suspicious targets (such as drones appearing in restricted areas at night) together with multimodal images marked with the location and path of the drone to the command center server. The command center then automatically or manually analyzes these images and data and performs subsequent intervention operations. Remote industrial monitoring: In an industrial environment, whenever the system detects an abnormal heating area, it sends its digital information (hot zone size, temperature upper limit, time of occurrence) and images to the remote monitoring platform through a secure channel. The platform also archives and visualizes the information for subsequent analysis or automatic alarm.

[0130] As can be seen from the above, when the target object is identified and digitized, the system can make real-time alarms or decisions. For example, in a very short time, it can tell the operator or related subsystems that the target object is a high-speed moving object that is suspected of violating traffic safety regulations or that a certain industrial equipment is overheating and the temperature is close to the safety threshold, so that the back-end control system can quickly take corresponding measures (such as speed limit warnings, alarms or temperature reduction treatment).

[0131] By packaging and outputting multimodal images and digital information, the system can not only visualize the current moment, but also review historical moments. If it is necessary to track the movement trajectory and state changes of the target object during safety inspections or post-investigations, it can be easily retrieved and analyzed from the output image sequences and digital data. Based on this, the system can play a pivotal role in accident playback, cause analysis, and responsibility determination.

[0132] In complex scenes, without the support of multi-dimensional information, recognition often relies on a single visible light or infrared image, which is easily affected by background changes or noise. Through the aforementioned multimodal processing and the target recognition inspection mechanism of this step, the system can better determine the existence and characteristics of the target object. This dynamic recognition performed at continuous moments can significantly reduce the false detection or missed detection rate, ensuring that the system remains highly robust and accurate during long-term operation and in a changing environment.

[0133] Furthermore, the target object recognition step includes image enhancement and fire source detection.

[0134] Image enhancement, for Perform bilateral filtering and Laplace sharpening to obtain an enhanced image :

[0135]

[0136] in, is a factor that controls the strength of sharpening; is the Laplace operator.

[0137] Fire source detection, using the pre-trained YOLOv5 model Perform fire source detection and identify the bounding box and confidence level of the fire source:

[0138]

[0139] Specifically, the steps for generating digital information of the target object include:

[0140] The position information of the identified target object at the current moment is obtained, the position estimation information at the previous moment is estimated and generated, and the position estimation information at the previous moment is corrected.

[0141] The position information at the current moment, the position estimation information at the previous moment, and the corresponding contour information are used as the digitized information of the target object.

[0142] In the above specific description, the location information is usually provided by an image recognition algorithm, based on image processing techniques such as machine learning or pattern recognition, to ensure the accuracy and reliability of the location data. The dynamic location estimation process then involves using the currently acquired location information to predict the possible location of the target object at the next moment. This prediction is usually based on the object's motion pattern, speed and other environmental factors, using various prediction algorithms, such as Kalman filtering or particle filtering, to generate the next moment's location estimate information.

[0143] From the above, it can be seen that the position information at the current moment is integrated with the position estimation information and contour information of the next moment to form detailed digital information of the target object, which is used for more complex data analysis, such as behavior analysis, trend prediction or security response.

[0144] Furthermore, Kalman filtering is used to track the detected target object in consecutive frames to predict the position change of the target object:

[0145]

[0146] in, is the predicted state vector; is the state estimate at the previous moment, which means the estimated value of the state variable (such as the position and speed of the fire source) based on the measured value at the previous moment; is the updated state vector; is the state transfer matrix; is the control input matrix; is the updated state vector; is the state transfer matrix; is the control input matrix; is the control input; is the prediction error covariance matrix; is the estimation error covariance matrix of the previous moment, which indicates the credibility of the estimation of the system state at the previous moment; is the estimated error covariance matrix at the current moment, which represents the new error covariance matrix obtained after the Kalman filter fuses and updates the observation data at the current moment; is the Kalman gain; is the observation vector; is the observation matrix; is the process noise covariance matrix; is the observation noise covariance matrix, is the identity matrix, that is, a matrix whose diagonal elements are all 1 and other elements are 0. In Kalman filtering, the identity matrix is ​​often used to represent matrix operations during state transfer or covariance update.

[0147] Specifically, the steps of processing the multimodal image according to the digital information include:

[0148] The contour information of the target object is generated through the position information and the position estimation information, and all the contour information is marked on the multimodal image.

[0149] In view of the above specific description, multimodal image processing involves generating contour information of the target object through accurate position information and predicted position estimation information, and further marking it on the multimodal image. First, the position information obtained from various sensors or image sources is combined with the dynamic position estimation data of the previous or current moment to accurately define and update the position of the target object. These position data reflect the specific coordinates of the target object at different time points, providing a basis for subsequent image processing. Based on these dynamically updated position information, the system will calculate the contour information of the target object. This step usually involves edge detection, morphological operations or deep learning models to extract clear contour lines from the image. The contour information not only includes the shape of the target object.

[0150] As can be seen above, contour information is generated for marking on multimodal images, and the contour of each target object is drawn on the final fused image. This not only improves the visual effect, allowing observers to more intuitively identify and analyze target objects, but also provides auxiliary information for automatic image processing algorithms, such as target tracking, behavior analysis, etc. In actual operation, these contour marks are usually highlighted with high-contrast lines or colors to ensure that they are clearly visible in multimodal images.

[0151] Furthermore, the processing steps of the multimodal images include artifact removal, color mapping, contour information annotation, and image compression and transmission.

[0152] Artifact removal, enhanced image Perform median filtering to obtain the image after artifact removal :

[0153]

[0154] in, It is a median filter operation, which replaces the current pixel value with the median value of the neighboring pixels.

[0155] Color mapping, Mapping to pseudo-color image , and superimpose the bounding box and tracking trajectory of the target object on the image to generate a visual image :

[0156]

[0157] in, Histogram matching operation, mapping grayscale values ​​to a predefined color mapping table; A predefined color map used to convert grayscale values ​​to colors.

[0158] Contour information annotation: superimpose the location information and tracking path of the target object on the image, and use graphic annotations (such as bounding boxes and trajectory lines) to visualize the information:

[0159]

[0160] in, For the overlay operation, the bounding box and trajectory are superimposed on the image; BoundingBoxes are bounding boxes; Trajectories are trajectories, which usually refer to the movement path of an object or target in the image. Through continuous image frames or position data, the trajectory of the object can be drawn to reflect its movement process.

[0161] Image compression and transmission: Use efficient image compression algorithms, such as JPEG2000 or HEVC, to compress visualization images for real-time transmission to the monitoring center:

[0162]

[0163] in, It is compressed image data. It maintains high-quality information after multimodal fusion and polarization enhancement, and is transmitted to the monitoring center in real time through a wireless network or wired transmission module to achieve remote monitoring and command decision-making.

[0164] The present invention provides an image digitization processing method based on infrared polarized light imaging, which achieves high adaptability to various complex environments by flexibly selecting polarization angles and synchronously collecting infrared and polarized images, and effectively reduces the risk of missed detection and false detection in high-speed moving objects, complex material surfaces and low-light environments. By setting special visual parameters to extract key multi-dimensional polarization information, the characteristics of the target object are highlighted, its visibility in the image is significantly improved, and interference elements are eliminated, reducing the computational burden of subsequent processing. In the fusion step, dual weighted information is used to preferentially fuse the polarized image to retain the advantages of its surface reflection and scattering characteristics, and then fuse it with the infrared image to enhance the expression of temperature and heat distribution information, generate a multimodal image, and thus provide a more complete and clear visual presentation of the target object. In addition, the system can also digitize and output the target object information in real time, support continuous tracking and analysis, and enhance the response capability to emergencies and complex scenes. Through multi-channel information fusion, it shows excellent stability and accuracy in the face of various environmental interferences, significantly improving the operating efficiency and recognition accuracy of the system in various harsh environments.

[0165] Another embodiment of the first aspect of the present invention proposes a forest fire monitoring and fire source location method based on an image digital processing method, and the device configuration is: an infrared polarization imaging system, including a high-resolution infrared camera and an adjustable polarization filter; a computer processing platform, equipped with a high-performance GPU for deep learning algorithms and real-time image processing; a wireless transmission module for transmitting the processed image data to a monitoring center in real time. In some embodiments of the present invention, such as Figure 3 As shown, the forest fire monitoring and fire source location method includes the following steps:

[0166] Step 1, image acquisition: When a forest fire occurs, the infrared polarization imaging system simultaneously collects infrared images at polarization angles of 0°, 45°, 90°, and 135° to obtain four infrared polarization images.

[0167] Step 2: Preprocessing: Perform wavelet denoising, AHE contrast enhancement and geometric correction on the four collected images to obtain the preprocessed images.

[0168] Step 3: Extract polarization information: Calculate the degree of polarization DoP and angle of polarization PoA of each preprocessed image; construct the polarization state matrix.

[0169] Step 4: Polarization image fusion: The four polarization images are fused according to the DoP weights, and the fused polarization image is obtained using the weighted least squares method.

[0170] Step 5: Multimodal fusion: The fused polarization image is registered with the infrared thermal image and fused using the Laplacian pyramid to obtain a multimodal image.

[0171] Step 6: Fire source location and tracking: bilateral filtering and Laplace sharpening are performed to obtain an enhanced image; the pre-trained YOLOv5 model is used to detect the fire source and identify the bounding box and confidence of the fire source; the Kalman filter is used to track the detected fire source in real time, predict the position change of the fire source, and generate the movement trajectory of the fire source.

[0172] Step 7, image post-processing: perform median filtering on the enhanced image to obtain an image with artifacts removed; map it to a pseudo-color image, and overlay the bounding box and tracking trajectory of the fire source on the image to generate a visualized image; use an efficient image compression algorithm, such as JPEG2000 or HEVC, to compress the visualized image for real-time transmission to the monitoring center.

[0173] The present invention provides a method for forest fire monitoring and fire source location. Through the above processing, the monitoring system can capture the thermal radiation information of the fire source in real time when a forest fire occurs, and enhance the significance of the fire source in combination with polarization information. The accuracy of fire source location and tracking is significantly improved. Even in complex forest environments and severe weather conditions, the system can effectively identify and track the diffusion path of the fire source. The compressed image data can be transmitted to the monitoring center in real time, assisting the commander to make quick response decisions, significantly improving the efficiency and effectiveness of fire emergency response.

[0174] The second aspect of the present invention provides a system for implementing the image digitization processing method in any of the above embodiments. Figure 4 As shown, the system includes:

[0175] The data transceiver module 201 is used to receive the infrared image and the polarization image, and output the digitized information and the processed multimodal image.

[0176] The image fusion module 202 is used to merge the infrared image and the polarization image into the multimodal image.

[0177] The image processing module 203 is used to generate digital information of the target object and process the multimodal image.

[0178] The present invention provides a system in which a data transceiver module effectively receives infrared images and polarization images, and can output accurate digital information and optimized multimodal images, thereby ensuring the efficiency and accuracy of data transmission. The image fusion module generates a multimodal image with enhanced visual information by merging infrared and polarization images, so that the features of the target object are more clearly displayed in the image, thereby improving the accuracy and reliability of image analysis. The image processing module not only generates detailed digital information of the target object, but also optimizes the processing of multimodal images, thereby enhancing the practicality and application scope of the image. Overall, the monitoring and analysis capabilities of target objects in complex environments are improved, operational efficiency is improved, errors are reduced, and the processing quality and response speed of image data are enhanced. It is an ideal solution for dynamic and changing environments.

[0179] Embodiments of the third aspect of the present invention provide electronic devices. In some embodiments of the present invention, such as Figure 4 As shown, an electronic device is provided, which includes: electronic devices such as desktop computers, notebooks, handheld computers, and cloud servers. The electronic device 3 may include but is not limited to a processor 301 and a memory 302. Those skilled in the art will understand that Figure 4 The electronic device 3 is merely an example and does not limit the electronic device 3 , and may include more or less components than those shown in the figure, or different components.

[0180] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0181] The memory 302 may be an internal storage unit of the electronic device 3, for example, a hard disk or memory of the electronic device 3. The memory 302 may also be an external storage device of the electronic device 3, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 3. The memory 302 may also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0182] The embodiment of the fourth aspect of the present invention proposes a computer-readable storage medium. In some embodiments of the present invention, a computer-readable storage medium is provided, and when the computer-readable storage medium is executed by the processor 301, the steps of the above method are implemented. Therefore, the computer-readable storage medium provided by the fourth aspect of the present invention has all the technical effects of the above steps, which will not be repeated here.

[0183] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0184] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0185] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0186] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0187] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. An image digital processing method based on infrared polarized light imaging, characterized in that: The steps include: At least one polarization angle is set considering the category to which the target scene belongs, and infrared images and polarization images of the target scene at all polarization angles are acquired in real time; Setting a visual parameter for distinguishing a target object in the target scene from surrounding objects in the target scene, and extracting multi-dimensional polarization information of the infrared image according to the difference in the visual parameter at each moment; Generate first weighted information between the polarization images and second weighted information between the infrared image and the polarization image through the multi-dimensional polarization information; fuse all polarization images and infrared images according to the first weighted information and the second weighted information at each moment to obtain a multimodal image; Identify the target object in the multimodal image at consecutive moments; when the target object exists, generate digital information of the target object, and process the multimodal image according to the digital information; package the digital information and the processed multimodal image and output them; The visual parameters include brightness values ​​of the infrared image at different polarization angles to characterize the surface characteristics, temperature characteristics, and observation states of light reflection and scattering behaviors of the target object on the image; The step of extracting the multi-dimensional polarization information of each infrared image according to the difference in the visible parameters at each moment comprises: Extract the brightness value of infrared images at all polarization angles at the same time; Calculate the polarization degree, polarization angle and polarization state matrix of the light wave in the infrared image through all the brightness values; The polarization degree, the polarization angle and the polarization state matrix constitute the multi-dimensional polarization information at the current moment.

2. The image digital processing method according to claim 1, characterized in that: The multimodal image fusion step comprises: generating the first weighted information of each of the polarization images according to the polarization degree, assigning a weight to each of the polarization images according to the first weighted information, and synthesizing a fused image; The second weighted information is generated by a matrix constructed by the brightness value, the polarization degree and the polarization angle, and weights are respectively assigned to the fused image and the infrared image through the second weighted information to synthesize the multimodal image.

3. The image digital processing method according to claim 2, characterized in that: Before the fused image is combined with the infrared image, the fusion step of the fused image further includes: The optimization parameters are set according to the observed state of the target object in the image, and the contours of all objects in the fused image are corrected according to the optimization parameters using the weighted least square method.

4. The image digital processing method according to claim 3, characterized in that: The observed state of the target object in the image is also used to classify and determine the category of the target scene.

5. The image digital processing method according to claim 2, characterized in that: After the fused image is combined with the infrared image, the fusion step of the fused image further includes: A Laplacian pyramid of the infrared image and the polarization image is constructed, and the multimodal image is reconstructed by fusing them at various resolution scales to correct the positions and shapes of all objects in the multimodal image.

6. The image digital processing method according to claim 1, characterized in that: The step of generating the digital information of the target object comprises: Obtain the position information of the identified target object at the current moment, estimate and generate the position estimation information at the next moment, and correct the position estimation information at the previous moment; The position information at the current moment, the position estimation information at the next moment, and the corresponding contour information are used as the digitized information of the target object.

7. The image digital processing method according to claim 6, characterized in that: The step of processing the multimodal image according to the digitized information comprises: The contour information of the target object is generated by using the position information and the position estimation information, and all the contour information is marked on the multimodal image.

8. A system for executing the image digital processing method according to any one of claims 1 to 7, characterized in that: include: A data transceiver module, used for receiving the infrared image and the polarization image, and outputting the digitized information and the processed multimodal image; An image fusion module, used for merging the infrared image and the polarization image into the multimodal image; The image processing module is used to generate digital information of the target object and process the multimodal image.

Images