An aircraft cargo hold monitoring method, monitoring device, medium and program product based on infrared fusion technology

Abstract

An aircraft cargo hold monitoring method and device based on infrared fusion technology, a medium and a program product, relate to the field of aviation safety monitoring. Based on the association of the three-dimensional point cloud of the near-infrared depth camera with the two-dimensional radiation temperature map of the long-wave thermal infrared camera to construct a four-dimensional data field containing three-dimensional space components and temperature components, the target real temperature eliminating the influence of dynamic changes of the flight environment is separated from the radiation temperature value, and then through spatial clustering of the three-dimensional space coordinates of the abnormal temperature pixels and matching with the preset loading unit model, the temperature anomaly can be accurately positioned to the specific cargo loading unit in the cargo hold from the abstract hot spot on the two-dimensional image plane, so that the crew can still obtain the alarm information containing the loading unit identification and three-dimensional space coordinates under the condition that they cannot enter the C-class cargo hold.

Description

Technical Field

[0001] This application relates to the field of aviation safety monitoring, and in particular to an aircraft cargo hold monitoring method, monitoring equipment, media, and program products based on infrared fusion technology. Background Technology

[0002] In modern aviation safety systems, monitoring fires and abnormal temperatures in aircraft cargo holds is a critical aspect of ensuring flight safety.

[0003] In related technologies, infrared thermal imaging monitoring systems are typically introduced into aircraft cargo holds. These systems utilize long-wave infrared cameras deployed within the cargo hold to capture real-time images of the thermal radiation from the cargo and its surroundings. When processing these images, based on standard atmospheric pressure and preset fixed ambient temperature parameters, the radiation grayscale values ​​received by the infrared detectors are converted into a two-dimensional temperature distribution map. A fixed high-temperature alarm threshold is then set for the entire cargo hold. When the calculated temperature of a certain area in the image exceeds this fixed threshold, a temperature anomaly is detected, triggering an alarm. Simultaneously, the two-dimensional thermal map is transmitted to the cockpit for the crew to view the thermal status of the cargo hold.

[0004] However, during actual flight operations, the cargo hold environment changes dynamically with each flight phase, especially in complex cargo stacking environments. This makes it difficult for related technologies to isolate environmental disturbances and displacement changes during flight when hot spots are detected, based on the distribution of heat radiation. Furthermore, Class C cargo holds do not allow crew members to enter directly during flight, making it difficult to determine the specific cargo loading unit to which the abnormal heat source belongs. Consequently, it becomes difficult for the crew to accurately assess the location of the hazard and take targeted isolation or firefighting measures. Summary of the Invention

[0005] This application provides an aircraft cargo hold monitoring method, monitoring equipment, media, and program product based on infrared fusion technology, which can improve the accuracy of abnormal temperature monitoring and spatial positioning precision in aircraft cargo holds.

[0006] Firstly, this application provides an aircraft cargo hold monitoring method based on infrared fusion technology, applied to monitoring equipment. The method includes: transforming a three-dimensional point cloud inside the cargo hold acquired by a near-infrared depth camera from the coordinate system of the near-infrared depth camera to the coordinate system of a long-wave thermal infrared camera based on a pre-calibrated extrinsic transformation matrix, to obtain an aligned three-dimensional point cloud; projecting each three-dimensional point in the aligned three-dimensional point cloud onto the pixel plane corresponding to a two-dimensional radiation temperature map synchronously acquired by the long-wave thermal infrared camera using the intrinsic parameter matrix of the long-wave thermal infrared camera, constructing a four-dimensional data field containing three-dimensional spatial components and temperature components; determining the current atmospheric transmittance based on the cargo hold's internal temperature, relative humidity, and atmospheric pressure within the current sampling period, and then, using the current atmospheric transmittance and the fundamental equation for infrared radiation temperature measurement, extracting the radiation temperature value from each pixel in the four-dimensional data field. The corresponding target true temperature is separated; after updating the four-dimensional data field with the target true temperature, based on the updated four-dimensional data field of each sampling period within a preset time window, the mean temperature and standard deviation of temperature for each pixel within the time window are calculated, and pixels whose target true temperature deviates from the mean temperature by more than a preset multiple of the standard deviation temperature in the current sampling period, or pixels whose rate of change of target true temperature between adjacent sampling periods exceeds a preset rate threshold, are marked as abnormal temperature pixels; after obtaining the clustering results by performing spatial clustering based on the three-dimensional spatial coordinates of the abnormal temperature pixels in the updated four-dimensional data field, the clustering results are matched with the preset loading unit model to determine the target cargo loading unit to which the abnormal temperature pixel belongs, and a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixel is generated.

[0007] By adopting the above technical solution, the three-dimensional point cloud of the near-infrared depth camera is transformed into the coordinate system of the long-wave thermal infrared camera based on the extrinsic parameter transformation matrix to obtain the aligned three-dimensional point cloud. Then, the aligned three-dimensional point cloud is projected onto the pixel plane of the two-dimensional radiation temperature map through the intrinsic parameter matrix, so that each pixel is simultaneously associated with three-dimensional spatial coordinates and temperature information, thereby constructing a four-dimensional data field. On this basis, the target true temperature that has eliminated the influence of dynamic changes in the flight environment is separated from the radiation temperature value. Furthermore, by performing spatial clustering on the three-dimensional spatial coordinates of abnormal temperature pixels and matching them with the preset loading unit model, the temperature anomaly can be accurately located from the abstract hot spot on the two-dimensional image plane to the specific cargo loading unit in the cargo hold. This allows the crew to obtain alarm information containing loading unit identification and three-dimensional spatial coordinates even when they cannot enter the Class C cargo hold.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes: when the target aircraft corresponding to the cargo hold is grounded, acquiring, based on a multi-mode calibration plate located at a preset calibration position within the cargo hold, a thermal infrared image collected by the long-wave thermal infrared camera, and a near-infrared depth image and an infrared intensity image collected by the near-infrared depth camera, wherein the bottom layer of the multi-mode calibration plate is a metal substrate with a surface long-wave infrared emissivity lower than a first emissivity threshold, the middle layer is a heat insulation pad with a thermal conductivity lower than a preset thermal conductivity threshold and hollowed out according to a preset checkerboard pattern, and the top layer is a filling covering the hollowed-out area. A coating with a mouth-side and surface long-wave infrared emissivity higher than a second emissivity threshold and absorption characteristics for near-infrared light, wherein the first emissivity threshold is lower than the second emissivity threshold; a first corner point coordinate set is determined based on the infrared radiation grayscale jump between the top and bottom layers in the thermal infrared image; a second corner point coordinate set corresponding to the checkerboard pattern is jointly determined based on the reflection brightness difference in the infrared intensity image and the depth jump between the hollowed-out and unhollowed-out areas in the near-infrared depth image; a registration constraint is constructed using the corresponding corner points of the first and second corner point coordinate sets, and the extrinsic parameter transformation matrix is ​​obtained by solving.

[0009] By adopting the above technical solution, the traditional power-consuming heat source design is replaced by utilizing the properties of passive materials. Due to the extremely low emissivity of the bottom polished aluminum and other metals, and the fact that the middle layer's thermal insulation is unaffected by cross-temperature variations, the top high-emissivity coating spontaneously forms a strong grayscale checkerboard pattern with the base plate in the long-wave thermal infrared domain under passive, room-temperature conditions. Simultaneously, the absorption characteristics of this top coating for near-infrared light, combined with the realistic physical hollowed-out recesses in the middle layer, cut out extremely sharp depth transition edges in the near-infrared ranging and reflection signals from the depth camera. This three-layer composite structure allows for the extraction of corresponding correlated corner points for the two distinctly different spectral modes of long-wave and near-infrared light on the same physical reference object. Highly robust parameter determination can be achieved in harsh cabin environments without external power supply.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, after updating the four-dimensional data field with the target true temperature, the method further includes: using the four-dimensional data field of the previous sampling period as a reference data field, the reference data field including the reference three-dimensional spatial coordinates and reference true temperature of each pixel; registering the three-dimensional spatial coordinates of each pixel in the four-dimensional data field of the current sampling period with the reference three-dimensional spatial coordinates, and solving for the spatial rigid body displacement matrix from the reference data field to the current sampling period; using the point set formed by transforming the reference three-dimensional spatial coordinates through the spatial rigid body displacement matrix as a transformation reference point set; and determining that in the four-dimensional data field of the current sampling period, the proportion of pixels whose three-dimensional spatial coordinates are outside the preset neighborhood range of the transformation reference point set exceeds a preset discrete proportion threshold, and the difference between the mean of the target true temperature of all pixels in the four-dimensional data field and the mean of the reference true temperature is less than a negative preset temperature drop threshold, determining that a smoke scattering and masking event has occurred in the cargo hold, and triggering a correction operation for the current atmospheric transmittance.

[0011] By employing the above technical solution, the spatial rigid body displacement matrix is ​​calculated through 3D point cloud registration of adjacent historical sampling periods, effectively filtering out interference from spatial absolute coordinate variations caused by normal flight turbulence or minor natural displacement of cargo. Detecting a large number of out-of-position pixels deviating from expected displacements under this benchmark reveals that the near-infrared signal has been subject to false distortion due to significant reflection and scattering by high-concentration aerosols. If this is accompanied by a sudden drop in the average global calculated temperature, it further confirms that the long-wave infrared radiation propagation optical path is suffering severe attenuation due to particle blockage. This greatly avoids false alarms caused by single sensor failure or accidental local blockage, ensuring the monitoring system's robust self-feedback capability against abnormal situations when the physical optical path is contaminated and deteriorated.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the step of triggering the correction operation on the current atmospheric transmittance specifically includes: extracting all displaced pixels whose three-dimensional spatial coordinates are outside a preset neighborhood range of the transformation reference point set in the four-dimensional data field of the current sampling period to form a smoke spatial sampling point set; constructing a three-dimensional smoke concentration distribution field of the cargo hold by spatial interpolation based on the local point density of each sampling point in the three-dimensional space of the smoke spatial sampling point set, using the local point density of each sampling point as the node value; for each pixel in the four-dimensional data field, performing path integration on the three-dimensional smoke concentration distribution field along the line of sight from the optical center of the long-wave thermal infrared camera to the three-dimensional spatial coordinates corresponding to the pixel to obtain the smoke optical thickness corresponding to the pixel; and correcting the current atmospheric transmittance based on an empirical correction formula to obtain the corrected atmospheric transmittance, wherein the empirical correction formula is: in, This is the corrected atmospheric transmittance. This is the current atmospheric transmittance. The cabin atmospheric pressure during the current sampling period. Standard atmospheric pressure The optical thickness of the smoke. This is a power-law scaling exponent of optical depth with respect to air pressure ratio. Under normal atmospheric conditions without smoke, a preset first calibration value is taken. After determining that a smoke scattering and masking event has occurred in the cargo hold, the preset second calibration value is switched. The second calibration value is obtained through experimental calibration of the cargo hold in a smoke-containing aerosol environment. The corrected atmospheric transmittance is substituted into the basic equation of infrared radiation thermometry, and the step of separating the corresponding target true temperature from the radiation temperature value of each pixel in the four-dimensional data field by using the current atmospheric transmittance and the basic equation of infrared radiation thermometry is re-executed.

[0013] By employing the aforementioned technical solution, a three-dimensional smoke concentration field with microscopic entity distribution characteristics is directly inverted and spatially interpolated based on the three-dimensional local density of these variable pixels. Furthermore, by performing a deep interval integration on this concentration field along the ray trajectory from the camera lens to each point on the cargo container surface, the unique smoke optical attenuation thickness experienced by each corresponding pixel is precisely quantified. Subsequently, combined with a pressure power calibration index specifically tailored for smoke-laden aerosol environments, a deep nonlinear compensation correction is performed on the basic atmospheric transmittance equation. This ensures that high-fidelity, accurate fire source surface temperature is still output to the generator unit even during severe fire hazard evolution.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of generating a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixel, the method further includes: taking the current sampling period as a reference, extracting all pixels within a preset spatial neighborhood range of the three-dimensional spatial coordinates of the abnormal temperature pixel from the updated four-dimensional data field of each sampling period within the preset time window; arranging the target real temperatures of all pixels in each sampling period in chronological order to form a temperature time series sequence of the abnormal temperature pixel; performing linear regression on the temperature time series sequence to obtain the temperature time series slope of the abnormal temperature pixel within the preset time window; and, if it is determined that the temperature time series slope exceeds a preset slope threshold, based on the target cargo loading unit... The three-dimensional spatial coordinates of all pixels in the unit constitute the heat source loading unit point set; based on the three-dimensional spatial coordinates of adjacent pixels within a preset contact distance of the heat source loading unit point set, a heat diffusion candidate point set is formed. The pixels in the heat diffusion candidate point set belong to other cargo loading units different from the target cargo loading unit to which the abnormal temperature pixel belongs; based on the temperature time-series slope of each pixel in the heat diffusion candidate point set within the preset time window, and the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set, the predicted remaining time for the container wall of the heat source loading unit to reach the preset thermal failure temperature is calculated. The preset thermal failure temperature is the pre-set structural failure critical temperature of the cargo loading unit container wall; the predicted remaining time is added to the temperature abnormality alarm.

[0015] By employing the aforementioned technical solution, when the temperature evolution of the abnormal pixel region exhibits a continuously steeply increasing linear slope, the overall boundary of the loading unit to which the heat source belongs can be precisely cut out in three dimensions, and adjacent wrapped cargo units with extremely close contact distances can be identified. By quantifying the actual three-dimensional contact exposure area between these two independent loading containers and observing the slope characterization of the unwarranted passive heating of adjacent innocent units at this stage, the time limit for the container wall at the ignition point to reach the failure threshold due to heat absorption can be deduced through spatial proximity attenuation characteristics. This provides the crew with a highly quantifiable time-series estimate of the safety boundary.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the step of calculating the predicted remaining time for the container wall of the heat source loading unit to reach the preset thermal failure temperature specifically includes: determining the heat absorption power of the loading unit to which the heat diffusion candidate point set belongs within the preset time window based on the temperature time-series slope of each pixel in the heat diffusion candidate point set within the preset time window and the heat capacity parameters of adjacent cargo loading units; dividing the heat absorption power by the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set to obtain the interface heat flux density; determining the heat release power inside the heat source loading unit through a heat balance equation based on the interface heat flux density and the temperature time-series slope of the heat source loading unit itself; and calculating the predicted remaining time for the container wall of the heat source loading unit to reach the thermal failure temperature based on the heat release power, the difference between the current target true temperature of the heat source loading unit and the preset thermal failure temperature of the cargo loading unit container wall, and the preset heat capacity parameters of the cargo loading unit container wall.

[0017] By employing the aforementioned technical solution, the temperature rise caused by the passive influence on the surface is transformed into the actual heat absorption power passively received by adjacent passive loading containers. Combined with the effective physical contact area between the two containers dynamically captured through 3D scanning, the precise interfacial heat flow overflow density is calculated. Subsequently, this enormous amount of heat that has already penetrated, conducted, and escaped is inversely superimposed and balanced with the heat dissipated by the internal heating of the ignition source's outer shell, thus deducing the hidden fire core inside the metal container, which is difficult to observe. This improves the scientific controllability of extreme threshold prediction and early warning.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of adding the predicted remaining time to the temperature anomaly alarm, the method further includes: when it is determined that there are multiple abnormal temperature pixels belonging to different cargo loading units in the four-dimensional data field of the current sampling period, the ratio of the number of observable pixels of the three-dimensional spatial contact surface in the four-dimensional data field to the theoretical total number of pixels of the three-dimensional spatial contact surface is determined as the observable integrity coefficient of the contact interface of the corresponding cargo loading unit, the theoretical total number of pixels being calculated based on the intersection area of ​​the three-dimensional spatial envelope of the heat source loading unit point set and the three-dimensional spatial envelope of the heat diffusion candidate point set and a preset pixel spatial resolution; the predicted remaining time corresponding to each cargo loading unit is divided by the observable integrity coefficient of the contact interface to obtain the correction priority score of the cargo loading unit; and the temperature anomaly alarms corresponding to each cargo loading unit are sorted in ascending order of the correction priority score.

[0019] By employing the aforementioned technical solution, when multiple thermal anomalies occur concurrently in a narrow, complexly stacked cargo hold, the dangerous contact surfaces of some containers will inevitably fall into the system's visual blind spot due to obstruction. The theoretical global intersection surface area derived from the boundary envelope is extracted and compared with the voxel pixels that can be realistically sampled at the front line. This feature further compresses the execution time by dividing the predicted remaining safety time by this smaller integrity coefficient. An intervention sequence for finding the optimal solution under concurrent anomaly conditions is established.

[0020] In a second aspect, this application provides a monitoring device comprising: one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the monitoring device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer program product containing instructions that, when run on a monitoring device, cause the monitoring device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a monitoring device, cause the monitoring device to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By associating the 3D point cloud of a near-infrared depth camera with the 2D radiation temperature map of a long-wave thermal infrared camera to construct a four-dimensional data field containing 3D spatial components and temperature components, the true temperature of the target is separated from the radiation temperature value. Then, a dynamic baseline is constructed using the temperature mean and temperature standard deviation within a preset time window to mark abnormal temperature pixels. Finally, spatial clustering of the 3D spatial coordinates of abnormal temperature pixels is performed and matched with a preset loading unit model to determine the target cargo loading unit. Therefore, under a unified four-dimensional data framework, both 3D spatial information and temperature information after adaptive compensation for the flight environment can be used for anomaly detection and spatial positioning. This improves the accuracy of temperature measurement and the precision of 3D spatial positioning in aircraft cargo hold temperature anomaly monitoring.

[0025] 2. By employing a technique of solving the spatial rigid body displacement matrix through 3D point cloud registration of adjacent sampling periods to construct a transformation reference point set, and jointly determining smoke scattering masking events when the proportion of out-of-position pixels exceeds the discrete proportion threshold and the global average temperature drops significantly, the interference of normal physical vibration displacement of the aircraft can be effectively filtered out. Through multi-dimensional verification of the severe depth false distortion characteristics caused by near-infrared signals being scattered by aerosols and the sudden cooling characteristics caused by the attenuation of long-wave infrared radiation being blocked, false alarms caused by single sensor failure or accidental masking are greatly avoided. This achieves high-confidence independent detection of early hidden smoke hazards in the cargo hold and improves the self-sensing ability against complex environmental interference.

[0026] 3. By employing the technical means of extracting abnormal heat sources and their adjacent unit contact candidate point sets, combining the three-dimensional physical contact area with the temperature time-series slope of the passive heating of adjacent innocent units, and jointly calculating the predicted remaining time for the container wall of the fire heat source loading unit to reach the thermal failure temperature and generating alarm additional parameters, the real physical boundary of heat deterioration and diffusion across the container can be accurately cut out at the three-dimensional level. This transcends the simple static over-limit temperature measurement thinking to the dynamic quantitative deduction of the fire's obstacle-breaking and spread trend, thereby realizing the scientific prediction of the safe threshold time of melt-through failure of aviation cargo hold containers and the improvement of the forward guidance capability for high-risk emergency rescue decisions in the cockpit (such as flight diversion). Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an aircraft cargo hold monitoring method based on infrared fusion technology in an embodiment of this application.

[0028] Figure 2 This is another flowchart illustrating an aircraft cargo hold monitoring method based on infrared fusion technology in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of an exemplary hardware structure of the monitoring device in this application embodiment. Detailed Implementation

[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0032] Please see Figure 1 This is a flowchart illustrating an aircraft cargo hold monitoring method based on infrared fusion technology in an embodiment of this application.

[0033] S101. Based on the pre-calibrated extrinsic transformation matrix, the three-dimensional point cloud inside the cargo hold acquired by the near-infrared depth camera is transformed from the coordinate system of the near-infrared depth camera to the coordinate system of the long-wave thermal infrared camera to obtain the aligned three-dimensional point cloud.

[0034] The extrinsic transformation matrix refers to the set of rigid body transformation parameters describing the spatial geometric relationship between the near-infrared depth camera and the long-wave thermal infrared camera. This matrix includes rotation and translation components, used to represent the spatial transformation relationship required to map three-dimensional coordinates in one camera coordinate system to the corresponding three-dimensional coordinates in another camera coordinate system. A near-infrared depth camera is an active depth sensing device operating in the near-infrared spectral range (typically around 940nm or 850nm). This device emits modulated near-infrared light signals into the cargo hold and receives the returned signals. Based on time-of-flight or structured light principles, it calculates the three-dimensional distance information of each surface point within the cargo hold relative to the camera's optical center, thereby generating a three-dimensional point cloud within the cargo hold composed of a large number of discrete three-dimensional coordinate points. A long-wave thermal infrared camera is a passive thermal radiation detection device operating in the long-wave infrared spectral range (8 to 14 micrometers). This device acquires a two-dimensional radiation temperature map by sensing the long-wave infrared electromagnetic wave energy spontaneously emitted from the surfaces of objects within the cargo hold. Its imaging principle differs fundamentally from that of a near-infrared depth camera; the former relies on the thermal radiation of the objects themselves, while the latter relies on the active illumination and reflection of near-infrared light. A three-dimensional point cloud within the cargo hold refers to the set of discrete three-dimensional spatial points obtained by a near-infrared depth camera after performing a depth scan of the cargo hold interior scene within a single sampling period, with the camera's own coordinate system as the reference. Each point contains three spatial coordinate components within the camera's coordinate system. The coordinate system of a near-infrared depth camera is a three-dimensional rectangular coordinate system established with the optical center of the depth camera as the origin and the normal direction of its imaging plane as the depth axis. The coordinate system of a long-wave thermal infrared camera is a three-dimensional rectangular coordinate system established with the optical center of the thermal infrared camera as the origin and the normal direction of its imaging plane as the depth axis. Aligning a 3D point cloud refers to mapping a set of 3D spatial points from the near-infrared depth camera coordinate system to the long-wave thermal infrared camera coordinate system after coordinate system transformation using an extrinsic parameter transformation matrix. The 3D coordinates of each point in this set are based on the coordinate system of the long-wave thermal infrared camera as a unified reference, thereby ensuring that the 3D spatial information is consistent with the 2D radiation temperature map subsequently acquired by the long-wave thermal infrared camera on the spatial reference reference.

[0035] Specifically, in each sampling cycle of the aircraft cargo hold monitoring system, the near-infrared depth camera and the long-wave thermal infrared camera work in parallel with synchronous triggering, respectively acquiring three-dimensional depth data and two-dimensional thermal radiation data of the cargo hold interior scene. Because the two cameras are installed in different positions and orientations within the cargo hold, and because the two sensors have different field of view and resolution characteristics, there are spatial rotation and translation differences between their respective coordinate systems. Therefore, the coordinate values ​​of each point in the three-dimensional point cloud output by the near-infrared depth camera are based on the depth camera itself and cannot be directly correlated point-by-point in space with the two-dimensional radiation temperature map acquired by the long-wave thermal infrared camera. To eliminate this coordinate system difference, the monitoring equipment reads the extrinsic parameter transformation matrix from pre-stored calibration parameters. This matrix has been obtained and stored through a dedicated calibration process during the deployment phase of the monitoring system. The algorithm iterates through each 3D point in the cargo hold 3D point cloud output by the near-infrared depth camera during the current sampling period. For each point's 3D coordinate vector, a rotation and translation transformation defined by the extrinsic transformation matrix is ​​applied. This involves first left-multiplying the point's coordinate vector by the rotation matrix component to align its spatial attitude, and then superimposing the translation vector component to align its spatial position. This transforms the point's coordinate representation from the near-infrared depth camera coordinate system to the long-wave thermal infrared camera coordinate system. After all points in the 3D point cloud have undergone the above coordinate transformation, all transformed 3D points are converged to form an aligned 3D point cloud. The 3D coordinates of each point in this aligned 3D point cloud use the long-wave thermal infrared camera coordinate system as a unified reference. This means that the spatial information in the aligned 3D point cloud and the 2D radiation temperature map acquired by the long-wave thermal infrared camera are within the same spatial reference frame, providing a fundamental guarantee of coordinate consistency for subsequently projecting the 3D points onto the 2D pixel plane and establishing pixel-by-pixel spatial-temperature correlations. In a real aircraft cargo hold environment, the cargo loading unit may undergo slight displacement due to air turbulence during flight. However, since the extrinsic transformation matrix describes the fixed installation geometry between the two cameras rather than the position of the cargo itself, as long as the relative installation positions of the two cameras remain stable during flight, the extrinsic transformation matrix remains effective throughout the entire flight mission cycle.

[0036] In some embodiments, when the target aircraft corresponding to the cargo hold is grounded, the operator places a dedicated multimodal calibration plate at a preset calibration position in the cargo hold. The bottom layer of the multimodal calibration plate is a metal substrate with a surface long-wave infrared emissivity lower than a first emissivity threshold, the middle layer is a heat insulation pad with a thermal conductivity lower than a preset thermal conductivity threshold and cut out in a preset checkerboard pattern, and the top layer is a coating that fills and covers the opening side of the cutout area, has a surface long-wave infrared emissivity higher than a second emissivity threshold, and has absorption characteristics for near-infrared light, wherein the first emissivity threshold is lower than the second emissivity threshold. For example, this multimodal calibration board can be composed of three layers of functional materials. The bottom layer is an aerospace-grade aluminum alloy substrate with a mechanically polished surface. The long-wave infrared emissivity of the polished aluminum alloy is extremely low, approximately 0.05, forming a low emissivity background for the calibration board. Simultaneously, the aluminum alloy is lightweight and has high thermal conductivity. This high thermal conductivity ensures uniform temperature throughout the calibration board, thus avoiding emissivity measurement deviations caused by uneven temperature distribution. The middle layer is a heat-insulating pad with a thermal conductivity lower than a preset thermal conductivity threshold and perforated according to a preset checkerboard pattern (e.g., 3M series optical black coating, with a long-wave infrared emissivity of approximately 1.00; black epoxy resin paint, with an emissivity of approximately 0.94-0.97). The calibration plate consists of black electrical tape (emissivity approximately 0.93-0.96). The perforated and unperforated areas of this thermal insulation layer form a structured depth pattern. The depth of the perforated areas is typically 3 to 5 millimeters, allowing the near-infrared depth camera to detect the corner positions of the calibration pattern through discontinuous depth edges. The top layer is a high-emissivity coating filling the openings of the perforated areas. This coating is made of a carbon-based or oxide-based optical black material with a long-wave infrared emissivity higher than 0.94. Simultaneously, this coating has near-infrared light absorption properties, causing the coated area to appear as a low-reflection dark area in the near-infrared intensity map, creating a contrast with the high-reflection bright area of ​​the uncoated aluminum alloy. The core advantage of this three-layer structure design lies in using the natural contrast of emissivity rather than temperature contrast to produce a clear pattern in thermal infrared images. It can operate under any temperature conditions without additional heating or cooling. Because emissivity is an inherent material property rather than a function of temperature, the calibration plate provides consistent pattern contrast across the entire flight envelope, from high ground temperatures to low high-altitude temperatures.

[0037] The operator places the multimodal calibration board at a preset calibration position in the cargo hold, simultaneously acquiring thermal infrared images from a long-wave thermal infrared camera and near-infrared depth and infrared intensity images from a near-infrared depth camera. Then, based on the infrared radiation grayscale jump between the top high-emissivity coating area and the bottom low-emissivity aluminum alloy substrate area in the thermal infrared image, the first corner point coordinate set is determined. Simultaneously, based on the difference in reflectance caused by the coating's absorption of near-infrared light in the infrared intensity image and the depth jump between the hollowed-out and unhollowed-out areas in the near-infrared depth image due to differences in physical thickness, the second corner point coordinate set corresponding to the checkerboard pattern is jointly determined. The two-dimensional pixel coordinates in the second corner point coordinate set are then back-projected into three-dimensional spatial coordinates using the depth values. Finally, a two-dimensional to three-dimensional registration constraint is constructed using the corresponding corner points in the spatial positions of the first and second corner point coordinate sets. The extrinsic parameter transformation matrix of the thermal infrared camera relative to the depth camera is solved using a perspective n-point algorithm, and this transformation matrix is ​​refined using a nonlinear optimization algorithm to minimize the reprojection error, achieving sub-pixel accuracy.

[0038] In another embodiment, while the aircraft is grounded, temperature-controlled calibration bodies with known geometric dimensions are placed at multiple different locations within the cargo hold. Each calibration body is simultaneously heated until its surface temperature is significantly higher than the ambient temperature. A long-wave thermal infrared camera identifies the high-temperature contour center of each calibration body in the thermal infrared image as a thermal infrared feature point. Simultaneously, a near-infrared depth camera extracts the geometric center of each calibration body in the 3D point cloud through geometric shape fitting as a depth feature point. Then, based on the spatial correspondence between multiple sets of thermal infrared feature points and depth feature points, an optimization algorithm minimizing reprojection error is used to solve for the extrinsic parameter transformation matrix. It is understood that other methods can also be used to achieve extrinsic parameter calibration between two infrared cameras of different wavelengths, such as using fixed structural features at known spatial locations within the cargo hold as natural calibration references; this is not limited here. It should be added that the calibration plate can be permanently installed in a specific location in the cargo hold as part of the airborne equipment to support the system's online self-calibration. At the same time, the aluminum alloy substrate has the aerospace environmental tolerance to vibration and temperature cycling, and the high emissivity coating is wear-resistant and corrosion-resistant after curing, ensuring the stability of the calibration plate in the long-term aviation operation environment.

[0039] S102. Using the intrinsic parameter matrix of the long-wave thermal infrared camera, project each three-dimensional point in the aligned three-dimensional point cloud onto the pixel plane corresponding to the two-dimensional radiation temperature map synchronously acquired by the long-wave thermal infrared camera, and construct a four-dimensional data field containing three-dimensional spatial components and temperature components.

[0040] The intrinsic parameter matrix of the long-wave thermal infrared camera refers to the parameter matrix describing the camera's own optical imaging geometry. This matrix includes horizontal focal length components, vertical focal length components, and horizontal and vertical offset components of the optical center on the pixel plane. It represents the internal geometric transformation parameters required to map three-dimensional spatial points in the long-wave thermal infrared camera coordinate system to the corresponding pixel coordinates on the camera's two-dimensional pixel plane according to perspective projection. The two-dimensional radiation temperature map refers to the two-dimensional temperature distribution matrix output by the long-wave thermal infrared camera after performing thermal radiation detection on the cargo hold interior scene in a single sampling period. The value of each pixel in this matrix represents the apparent radiation temperature of the object surface in the field of view corresponding to that pixel. The pixel plane refers to the two-dimensional imaging plane of the long-wave thermal infrared camera. This plane uses pixels as basic units, and each pixel has unique row and column coordinates. The three-dimensional spatial components refer to the three spatial coordinate values ​​of each three-dimensional point in the aligned three-dimensional point cloud in the long-wave thermal infrared camera coordinate system. These three coordinate values ​​are obtained by the coordinate transformation in step S101. The temperature component refers to the radiation temperature value carried by the pixel corresponding to the projection position of the three-dimensional point in the two-dimensional radiation temperature map. A four-dimensional data field refers to a fused data structure formed by associating three-dimensional spatial components with temperature components pixel by pixel. Each data element in this data structure contains a three-dimensional spatial coordinate vector and a temperature scalar value, thus carrying both spatial geometric information and thermal radiation temperature information of the cargo hold interior scene within a unified data framework.

[0041] Specifically, after obtaining the aligned 3D point cloud in step S101, the intrinsic parameter matrix of the long-wave thermal infrared camera is read from the pre-stored calibration parameters. This intrinsic parameter matrix has been obtained and stored through the intrinsic parameter calibration process during the camera's factory calibration or system deployment phase. Each 3D spatial point in the aligned 3D point cloud is traversed, and a perspective projection operation is performed on each point. This involves normalizing the horizontal and vertical components of the point's 3D coordinates in the long-wave thermal infrared camera coordinate system by dividing them by the depth component, obtaining normalized planar coordinates. Then, the normalized planar coordinates are scaled using the focal length component in the intrinsic parameter matrix, and the optical center offset component is superimposed to calculate the 2D projected pixel coordinates of the point on the long-wave thermal infrared camera's pixel plane. The physical meaning of this projection process is to simulate the imaging optical path of the long-wave thermal infrared camera, determining which pixel a surface point of an object at a certain spatial location within the cargo hold falls on in the 2D image captured by the camera. After obtaining the projected pixel coordinates of each 3D point, the corresponding radiation temperature value is searched in the synchronously acquired 2D radiation temperature map using these pixel coordinates as an index. This radiation temperature value is then used as a temperature component and bound to the 3D spatial coordinate component of the 3D point, generating a four-dimensional data vector containing three spatial coordinate values ​​and one temperature value. Once all 3D points in the aligned 3D point cloud have been projected and temperature-associated, all four-dimensional data vectors are aggregated and organized into a four-dimensional data field. During the projection process, due to the potential difference between the spatial sampling density of the aligned 3D point cloud and the pixel resolution of the long-wave thermal infrared camera, multiple 3D points may be projected to the same pixel location, or some pixel locations may not have 3D point projections. For cases where multiple 3D points are projected to the same pixel, the 3D point closest to the optical center of the long-wave thermal infrared camera is selected as the effective associative point for that pixel. This is because the closest point corresponds to the surface of the object actually visible to the camera. This processing method is equivalent to a depth buffering mechanism to resolve occlusion issues. For pixel locations without 3D point projections, the pixel is marked as a pixel without depth association; this pixel only retains radiation temperature information and does not participate in subsequent analysis processes requiring 3D spatial coordinates. Through the above pixel-by-pixel projection and association operations, the constructed four-dimensional data field simultaneously carries the three-dimensional spatial geometry and thermal radiation temperature distribution of the cargo hold interior scene within a unified data framework.

[0042] In some embodiments, the projection of the aligned 3D point cloud onto a 2D pixel plane and the construction of a 4D data field can be achieved in various ways. Optionally, for each 3D point in the aligned 3D point cloud, after normalization and intrinsic parameter matrix transformation to obtain sub-pixel precision projection coordinates, the sub-pixel coordinates are rounded to the nearest integer pixel coordinates. Then, the radiation temperature value of the corresponding pixel is directly read from the 2D radiation temperature map using the integer pixel coordinates as an index to complete the association. This method is computationally efficient and suitable for scenarios where the spatial sampling density of the aligned 3D point cloud is similar to the pixel resolution. Optionally, after obtaining the sub-pixel precision projection coordinates, instead of simple rounding, four adjacent integer pixel coordinates are selected around the sub-pixel coordinates. Based on the distance weights between the sub-pixel coordinates and the four adjacent pixel coordinates, bilinear interpolation is performed on the radiation temperature values ​​of the four adjacent pixels in the 2D radiation temperature map. The interpolated temperature value is used as the associated temperature component of the 3D point, thereby improving the spatial precision of temperature association at the sub-pixel level. This method is suitable for fine-grained monitoring scenarios with high requirements for temperature association precision. It is understandable that other projection and association methods can also be used to construct a four-dimensional data field, such as using nonlinear projection correction based on radial and tangential distortion models before pixel association, which is not limited here.

[0043] It should be noted that the intrinsic parameter matrix of a long-wave thermal infrared camera is usually obtained by the manufacturer through a blackbody radiation source array when the camera leaves the factory. Alternatively, it can be recalibrated on-site during the system deployment phase by placing a point heat source array with known temperature and known spatial location in the cargo hold. The accuracy of the intrinsic parameter matrix directly affects the accuracy of the projection of three-dimensional points onto the pixel plane, and thus affects the correlation quality of spatial information and temperature information in the four-dimensional data field.

[0044] S103. After determining the current atmospheric transmittance based on the cargo hold's internal temperature, relative humidity, and atmospheric pressure within the current sampling period, the corresponding target true temperature is separated from the radiation temperature value of each pixel in the four-dimensional data field using the current atmospheric transmittance and the basic equation for infrared radiation thermometry.

[0045] The cabin temperature refers to the actual temperature of the air inside the cargo hold during the current sampling period. This temperature is measured in real time by temperature sensors deployed inside the cargo hold. During flight, the cabin temperature changes due to the combined effects of the outside temperature and the air conditioning system, typically ranging from 5 to 35 degrees Celsius. Relative humidity refers to the percentage of water vapor content in the cargo hold air relative to the saturated water vapor content at that temperature during the current sampling period. This parameter is measured in real time by humidity sensors deployed inside the cargo hold. Cabin humidity decreases significantly with flight time, dropping below 20% on long-haul flights. Atmospheric pressure refers to the absolute air pressure value inside the cargo hold during the current sampling period. This parameter is measured in real time by pressure sensors deployed inside the cargo hold. According to airworthiness regulations, the pressure altitude of pressurized cargo holds is limited to no more than 8,000 feet (approximately 2,440 meters), corresponding to a minimum cabin pressure of approximately 75 kPa. Therefore, the dynamic range of cargo hold pressure is approximately 75 kPa to 101 kPa. Current atmospheric transmittance refers to the proportion of long-wave infrared radiation that is not absorbed by gas molecules such as water vapor and carbon dioxide in the atmosphere as it travels from the cargo surface to the long-wave thermal infrared camera detector under the cabin environmental conditions of the current sampling period. This proportion ranges from 0 to 1; the closer the value is to 1, the less infrared radiation is absorbed by the atmosphere. The fundamental equation for infrared radiation thermometry describes the quantitative relationship between the total radiation energy received by the long-wave thermal infrared camera detector and the true surface temperature of the target object, the surface emissivity of the target object, atmospheric transmittance, and the ambient reflected temperature and atmospheric temperature. This equation is based on Planck's blackbody radiation law and Kirchhoff's radiation law. Its physical meaning is to decompose the total radiation signal received by the detector into three components: the component of the target's own radiation after atmospheric attenuation, the component of the ambient reflected radiation after reflection from the target surface and then attenuated by the atmosphere, and the radiation component emitted by the atmosphere itself along the radiation transmission path. The radiation temperature value refers to the temperature component carried by each pixel in the four-dimensional data field. This temperature component comes from the two-dimensional radiation temperature map output by the long-wave thermal infrared camera and is the apparent temperature without atmospheric transmittance correction.

[0046] Specifically, after completing step S102 to construct the four-dimensional data field, the temperature component carried by each pixel in the four-dimensional data field is still the radiation temperature value directly output by the long-wave thermal infrared camera. This radiation temperature value deviates from the true temperature of the cargo surface due to the influence of the atmospheric environment inside the cargo hold on the infrared radiation transmission. The environmental conditions in aircraft cargo holds, especially Category C cargo holds, during flight differ significantly from those in general ground scenarios: during the ground phase, the cabin air pressure is close to standard atmospheric pressure, and the temperature and humidity are affected by the ground climate. However, during the cruise phase, the cabin air pressure decreases due to the adjustment of the pressurization system, and the outside temperature drops from above 50 degrees Celsius at ground level to below -55 degrees Celsius at the cruise altitude. The cabin temperature also changes due to the combined effects of the high-altitude low-temperature environment and the air conditioning system. The large-scale dynamic changes of these three environmental parameters directly affect the accuracy of infrared temperature measurement. In contrast, general ground systems typically do not need to consider air pressure changes and do not face such a large range of temperature and humidity fluctuations.

[0047] During each sampling cycle, the current cabin temperature is first read from the environmental sensor array deployed within the cargo hold. relative humidity and atmospheric pressure Then, based on the current cabin temperature and relative humidity, the saturated water vapor pressure at the current temperature is calculated using the saturated water vapor pressure equation. :

[0048]

[0049] in The unit is Celsius. The unit is hectopascals (hPa). The saturated water vapor pressure... With relative humidity The product of these factors is determined as the actual partial pressure of water vapor in the atmosphere. Furthermore, based on the actual partial pressure of water vapor and the spectral absorption characteristics of water vapor in the long-wave infrared band (8-14μm), combined with the radiation transmission path length between the long-wave thermal infrared camera and the target being measured, (In cargo hold scenarios, this distance is typically 1 to 5 meters). The basic atmospheric transmittance is calculated using a simplified empirical formula based on an atmospheric radiative transfer model. :

[0050]

[0051] in This is a band-dependent transmittance baseline coefficient. and These are empirical coefficients that depend on atmospheric conditions. The column content of water vapor in the atmosphere is determined by the actual water vapor partial pressure. Atmospheric temperature and Boltzmann constant J / K calculations show that, due to the short distance between the sensor and the target inside the cargo hold, the basic atmospheric transmittance of long-wave infrared radiation is usually high within this distance range.

[0052] To obtain the current atmospheric transmittance Then, determine the ambient reflection temperature. Take the cabin air temperature As an approximation of the ambient reflected temperature, a spatially weighted average can be taken when multiple temperature sensors are deployed inside the cargo hold. Then, each pixel in the four-dimensional data field is traversed, and temperature separation calculations are performed for each pixel using the fundamental equations of infrared radiation thermometry. The total radiation signal received by the long-wave thermal infrared camera detector... It consists of 3 components:

[0053]

[0054] in The emissivity of the surface of the object being measured. Given the current atmospheric transmittance, The Stefan-Boltzmann constant is... The true temperature of the object being measured. The ambient reflected temperature, Atmospheric temperature, Indicates temperature as The blackbody radiation power. Here, it is assumed that the object being measured is opaque, i.e., has zero transmittance. According to Kirchhoff's laws, the reflectivity of the object... Atmospheric reflection is negligible and atmospheric emissivity is... .

[0055] The true temperature of the object being measured can be obtained by solving the above equation. :

[0056]

[0057] Current atmospheric transmittance Ambient reflected temperature Atmospheric temperature and the preset emissivity of the target object surface Substituting into the above formula, the total radiated signal received from the detector Subtracting the atmospheric self-radiation component and environmental background reflected radiation components The contribution, divided by atmospheric transmittance and target surface emissivity and Stefan-Boltzmann constant The product of these factors is then taken as the fourth root to inversely calculate the true target temperature corresponding to that pixel. After all pixels in the four-dimensional data field have undergone the aforementioned temperature separation calculations, the target true temperature corresponding to each pixel is obtained. This target true temperature, compared to the original radiation temperature value, eliminates the temperature measurement deviation introduced by changes in the cabin's atmospheric environment and ambient background radiation, and can more accurately reflect the actual temperature state of each cargo surface within the cargo hold. The target true temperatures of all pixels are combined into a corrected temperature distribution matrix for subsequent anomaly detection.

[0058] In some embodiments, when obtaining the basic atmospheric transmittance Subsequently, a pressure correction is introduced. Since the collision broadening of absorbing molecules in the atmosphere decreases with decreasing pressure, leading to narrower absorption lines and a reduction in total absorption, atmospheric transmittance increases accordingly under low-pressure conditions. The current atmospheric transmittance is obtained by correcting the baseline atmospheric transmittance using a pressure correction formula. :

[0059]

[0060] in To measure the cabin pressure, kPa is the standard atmospheric pressure. The empirical correction index is the power-law scaling index of optical depth to air pressure ratio. For the short-distance cargo hold transport scenario described in this application, under normal atmospheric conditions without smoke, the extremely short radiation transmission path places water vapor absorption in the weak linear absorption region of Beer's Law. Within this linear region, the absorption is proportional to the molecular number density, which, under the ideal gas approximation, is proportional to the air pressure. Therefore, under normal atmospheric conditions, a preset first calibration value is taken, approximately equal to 1.0. At this point, the transmittance correction caused by a change in cargo hold air pressure from 101 kPa to 75 kPa is less than 0.1%, and the correction term is close to 1, so the influence of air pressure on transmittance can be approximately ignored. However, the true engineering significance of this air pressure correction formula lies in abnormal scenarios where smoke or aerosols appear in the cargo hold. Once smoke particles generated by an early fire appear in the cargo hold, particle scattering does not obey the linear relationship of Beer's Law and will deviate from 1.0. In this case, it is necessary to switch to the second calibration value obtained through experimental calibration to ensure the accuracy of temperature measurement under smoke conditions. This switching mechanism will be explained in detail in subsequent steps.

[0061] In some embodiments, the process of determining the current atmospheric transmittance and the calibration value of the empirical correction index based on environmental parameters can be implemented in various ways. Optionally, the calibration value determined by theoretical calculation is used to calculate the current atmospheric transmittance. Specifically, using gas absorption line parameters in a high-resolution molecular spectroscopy database, under multiple different pressure levels (e.g., 75 kPa, 80 kPa, 85 kPa, 90 kPa, 95 kPa, and 101 kPa) and multiple different temperature conditions, the theoretical value of atmospheric transmittance in the long-wave infrared band under the corresponding pressure and temperature conditions is calculated by line-by-line integration or a statistical band model. Then, the natural logarithm of the ratio of the theoretical transmittance value to the pressure at each pressure level is taken to obtain the current atmospheric transmittance. With the vertical axis as the y-axis, and with A linear fit is performed on the horizontal axis, and the slope of the resulting straight line is... The calibration value, for short-distance cargo hold transport scenarios, is typically given by the theoretical calculation results. Approximately equal to 0.9 to 1.0, this calibration value is stored and then substituted into the air pressure correction formula to calculate the current atmospheric transmittance in each sampling period. Optionally, it can be determined using a laboratory simulation calibration method. The calibration value is determined by simulating the cargo hold pressure range within a sealed calibration chamber with controllable pressure. The pressure in the calibration chamber is gradually reduced from 101 kPa to 75 kPa in 5 kPa increments. At each pressure level, the temperature, measurement distance, and humidity of the standard radiation source (blackbody furnace) within the calibration chamber are kept constant. The radiation output readings of the long-wave thermal infrared camera at each pressure level are recorded. The equivalent atmospheric transmittance at each pressure level is calculated from the radiation output readings using the fundamental equations for infrared radiation thermometry. Then, a graph is plotted on the logarithmic relationship between the equivalent atmospheric transmittance and the pressure ratio, and the slope is fitted to obtain the calibration value. The calibration value directly reflects the integrated system-level response characteristics of the sensor and the atmosphere, and is closer to the actual engineering accuracy requirements than theoretical calculations.

[0062] Understandably, in-flight online adaptive estimation can also be used to determine... The calibration values ​​are dynamically updated, for example, by comparing the deviation between the temperature measured by the thermal imager and the precise temperature measured by the contact sensor on a reference target with a known temperature installed in the cargo hold at different flight altitudes and under different air pressure conditions. The estimated value constitutes a closed-loop adaptive compensation mechanism, which is not limited here.

[0063] It should be noted that the emissivity parameter of the target object surface involved in the fundamental equations of infrared radiation thermometry The emissivity of cargo loading unit container walls varies depending on the material used. Common reference values ​​for cargo surface materials are approximately 0.93 for cardboard and paper, 0.87 to 0.88 for fabrics and cloth, 0.95 for plastic film, and 0.25 for alumina metal container surfaces. In practical applications, if the system has the capability to classify cargo surface materials, the corresponding emissivity parameter can be obtained from the emissivity lookup table. If no material classification information is available, the default value will be used. As an approximation for most non-metallic materials. In addition, the coefficients 17.269 and 237.29 in the saturated water vapor pressure equation are empirical constants of the Magnus formula, applicable to temperatures above 0°C. For the low-temperature environment (below 0°C) that may occur in the cargo hold, correction coefficients applicable to the saturated water vapor pressure on ice surfaces can be switched to ensure the accuracy of the water vapor partial pressure calculation.

[0064] S104. After updating the four-dimensional data field with the target true temperature, calculate the mean temperature and standard deviation of each pixel within the time window based on the updated four-dimensional data field of each sampling period within the preset time window, and mark the pixels whose target true temperature deviates from the mean temperature by more than a preset multiple of the standard deviation of the temperature in the current sampling period, or the pixels whose target true temperature changes at a rate exceeding a preset rate threshold between adjacent sampling periods, as abnormal temperature pixels.

[0065] The preset time window refers to a fixed time period that traces backward from the current sampling period. This time window contains updated four-dimensional data fields from multiple consecutive sampling periods, providing a statistical reference baseline in the time dimension for temperature anomaly detection. The mean temperature is the arithmetic mean of the target's true temperature at the same pixel location within the updated four-dimensional data field of each sampling period within the preset time window, calculated over time. This mean reflects the baseline temperature level of the pixel in the recent time period. The standard deviation temperature is a statistical measure of the dispersion of the target's true temperature at the same pixel location relative to the mean temperature in each sampling period within the preset time window. This standard deviation reflects the normal range of temperature fluctuations for the pixel in the recent time period. The preset multiple is a pre-set sensitivity coefficient used to control the strictness of temperature deviation judgment. The product of this coefficient and the standard deviation temperature constitutes the dynamic alarm deviation threshold for the pixel. An abnormal temperature pixel is a pixel determined to have a temperature state deviating from the normal baseline in the current sampling period, satisfying at least one of the temperature deviation condition or the rate of change condition.

[0066] Specifically, after obtaining the target true temperature of each pixel in the four-dimensional data field in step S103, the temperature component update operation of the four-dimensional data field is first performed. That is, each pixel in the four-dimensional data field is traversed, and the original radiation temperature value of the pixel is replaced with the corresponding target true temperature value obtained in step S103, while keeping the three-dimensional spatial coordinate components of the pixel unchanged, thus obtaining the updated four-dimensional data field. The updated four-dimensional data field of the current sampling period is stored in the time-series data buffer, which stores the updated four-dimensional data fields of each historical sampling period within the preset time window. During normal flight, the overall temperature of the cargo hold undergoes systematic changes from the ground to climb to cruise to descent and then to landing. To avoid triggering false alarms due to the slow drift of the overall temperature, a dynamic baseline tracking mechanism is used instead of a fixed threshold for anomaly detection. The updated four-dimensional data field, encompassing all sampling periods within a preset time window, is extracted from the time-series data buffer. For each pixel location within the four-dimensional data field, the target's true temperature value is collected across all sampling periods within the time window to form a temperature time series for that pixel. The arithmetic mean of this temperature time series is then calculated to obtain the pixel's temperature mean, and the standard deviation of the temperature time series relative to the temperature mean is calculated to obtain the pixel's temperature standard deviation. The temperature mean represents the baseline temperature level of the pixel in the recent time period, while the temperature standard deviation represents the temperature fluctuation range of the pixel under normal conditions. Subsequently, a dual anomaly detection is performed on each pixel in the updated four-dimensional data field for the current sampling period: The first detection is temperature deviation detection, which calculates the absolute value of the difference between the pixel's true target temperature and its mean temperature in the current sampling period, and determines whether this absolute value exceeds the product of the pixel's temperature standard deviation and a preset multiple. If it does, it indicates that the pixel's current temperature state significantly deviates from its recent normal baseline level. The second detection is rate of change detection, which calculates the difference between the pixel's true target temperature in the current sampling period and the previous sampling period, and divides this difference by the time interval between the two sampling periods to obtain the rate of temperature change. It determines whether this rate of change exceeds a preset rate threshold. If it does, it indicates that the pixel's location has experienced a sharp temperature rise that is inconsistent with the slow drift of the normal ambient temperature. Pixels that meet at least one of the above two detection conditions are marked as abnormal temperature pixels. This dynamic baseline tracking mechanism ensures that the system automatically adapts to the slow drift of the ambient temperature throughout the flight and only responds to real abnormal temperature rises.

[0067] In some embodiments, after updating the four-dimensional data field with the target true temperature, a three-dimensional point cloud temporal consistency check between adjacent sampling periods is further performed. The four-dimensional data field of the previous sampling period is used as a reference data field, which contains the reference three-dimensional spatial coordinates and reference true temperature of each pixel. Then, the three-dimensional spatial coordinates of each pixel in the four-dimensional data field of the current sampling period are registered with the reference three-dimensional spatial coordinates to solve for the spatial rigid body displacement matrix from the reference data field to the current sampling period. This spatial rigid body displacement matrix describes the minute spatial displacement and rotation changes of the entire scene inside the cargo hold between two adjacent sampling periods. The set of points formed by transforming the reference three-dimensional spatial coordinates through the spatial rigid body displacement matrix is ​​used as the transformation reference point set. Then, it is detected whether the proportion of pixels whose three-dimensional spatial coordinates are outside the preset neighborhood range of the transformation reference point set in the four-dimensional data field of the current sampling period exceeds a preset discrete proportion threshold. At the same time, it is detected whether the difference between the mean of the target true temperature of all pixels in the four-dimensional data field and the mean of the reference true temperature is less than a negative preset temperature drop threshold. If both of the above conditions are met, it is determined that a smoke scattering and masking event has occurred in the cargo hold and a correction operation on the current atmospheric transmittance is triggered. This is because smoke aerosols will scatter the structured light or time-of-flight signal emitted by the near-infrared depth camera, resulting in a large number of false anamorphic three-dimensional points in the depth measurement. At the same time, smoke will attenuate long-wave infrared radiation, resulting in a global apparent temperature drop. The combined satisfaction of the two criteria constitutes a sufficient condition for the determination of a smoke scattering and masking event.

[0068] In another embodiment, after updating the four-dimensional data field with the target's true temperature, environmental interference detection is performed using a method based on spatial gradient anomalies in the temperature field. The temperature gradient field between adjacent pixels in the updated four-dimensional data field for the current sampling period is calculated. Then, the current temperature gradient field is compared pixel-by-pixel with the temperature gradient field of the previous sampling period. If the average amplitude of the temperature gradient globally decreases significantly compared to the previous sampling period, and the decrease exceeds a preset gradient decay threshold, and simultaneously the global distribution variance of the three-dimensional spatial coordinates in the four-dimensional data field increases significantly compared to the previous sampling period, and the increase exceeds a preset variance expansion threshold, then an environmental interference event affecting infrared radiation transmission is determined to have occurred in the cargo hold, triggering a correction operation for the current atmospheric transmittance. This method detects environmental interference from the perspective of changes in the spatial gradient characteristics of the temperature field, complementing the method based on three-dimensional point cloud registration. It is understood that other methods can also be used to detect environmental interference events in the cargo hold and trigger atmospheric transmittance correction; this is not limited here.

[0069] In some embodiments, the marking of abnormal temperature pixels can be achieved in various ways. Optionally, for the temperature time series of the same pixel location in the updated four-dimensional data field for each sampling period within a preset time window, a linear trend removal operation is first performed to eliminate the systematic temperature drift component caused by flight phase switching. Then, the mean and standard deviation of the residual sequence after trend removal are calculated. The mean and standard deviation of the residual sequence are used as dynamic baseline parameters to perform temperature deviation judgment and rate of change judgment. This method can more effectively distinguish between systematic temperature drift and local abnormal temperature rise. Optionally, the temperature time series within the preset time window is calculated using an exponentially weighted moving average method, so that historical temperature values ​​closer to the current sampling period in time receive a greater weight in the mean calculation. This allows the dynamic baseline to track recent temperature change trends more quickly. At the same time, the temperature standard deviation is also calculated using a corresponding exponential weighting method to maintain the timeliness consistency of the baseline parameters. It is understood that other statistical methods or machine learning methods can also be used to detect and mark abnormal temperature pixels, which are not limited here.

[0070] It should be noted that the length of the preset time window and the preset multiple need to be calibrated by engineering based on the specific aircraft type, cargo hold type and operating scenario. If the time window is too short, it will cause the statistical baseline to be unstable and generate false alarms. If the time window is too long, it will cause the baseline to lag in tracking environmental changes and miss slowly developing temperature anomalies. If the preset multiple is too small, it will increase sensitivity but increase the false alarm rate. If the preset multiple is too large, it will reduce the false alarm rate but may miss early temperature anomalies.

[0071] S105. After obtaining the clustering results by performing spatial clustering based on the three-dimensional spatial coordinates of the abnormal temperature pixel in the updated four-dimensional data field, the clustering results are matched with the preset loading unit model to determine the target cargo loading unit to which the abnormal temperature pixel belongs, and a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixel is generated.

[0072] Here, "abnormal temperature pixels" refers to the set of pixels marked as having a temperature state deviating from the normal baseline in step S104. Each abnormal temperature pixel in the updated four-dimensional data field simultaneously possesses three-dimensional spatial coordinates and a corrected target true temperature. Pixels in this set satisfy at least one of the following conditions: temperature deviation or rate of change. Spatial clustering refers to an unsupervised grouping operation that, based on the three-dimensional spatial coordinates of abnormal temperature pixels, groups spatially adjacent abnormal temperature pixels into the same cluster according to a three-dimensional Euclidean distance metric. This results in abnormal temperature pixels belonging to the same local heat source region being aggregated together, while abnormal temperature pixels belonging to different local heat source regions are separated into different clusters. A pre-defined loading unit model refers to a set of three-dimensional spatial geometric descriptions of cargo loading units pre-established and stored in monitoring equipment. This set includes the standard geometric dimensions of cargo loading units at various loading positions within the cargo hold, their three-dimensional spatial occupancy range in the cargo hold coordinate system, and the corresponding loading unit identification number. A cargo loading unit refers to a standardized container or pallet used to carry and secure cargo within an aircraft cargo hold. Typical forms include aircraft containers (i.e., unit loading equipment) and aircraft pallets. Each loading unit has a unique identification number used to trace the cargo information it carries in the cargo loading manifest. A target cargo loading unit refers to the specific cargo loading unit to which an abnormal temperature pixel belongs, determined by matching the spatial clustering results with the spatial occupancy range of the pre-defined loading unit model. This matching process locates abstract three-dimensional spatial anomalies onto specific physical cargo containers. A temperature anomaly alarm is a structured alarm message that generates information including the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the pixel with the abnormal temperature. This alarm message is used to communicate the specific location of the temperature anomaly within the cargo hold and the cargo information to which it belongs to to the crew or ground monitoring personnel. This allows the crew to accurately identify which specific cargo loading unit the temperature anomaly occurred in, even when access to a Category C cargo hold is not permitted. The heat source loading unit point set is a three-dimensional spatial point set consisting of the three-dimensional spatial coordinates of all pixels belonging to the target cargo loading unit. This point set describes the complete surface geometry of the cargo loading unit identified as a heat source in three-dimensional space. The heat diffusion candidate point set is a three-dimensional spatial point set consisting of the three-dimensional spatial coordinates of adjacent pixels within a preset contact distance range from the heat source loading unit point set. The pixels in this point set belong to other cargo loading units different from the target cargo loading unit to which the abnormal temperature pixel belongs. This set is used to characterize the contact interface area of ​​adjacent loading units that may be affected by heat conduction from the heat source loading unit. The preset thermal failure temperature refers to the critical temperature at which the container wall of the cargo loading unit is set beforehand. When the container wall temperature reaches this critical temperature, the mechanical properties of the container wall material deteriorate significantly, leading to the loss of structural integrity. The predicted remaining time refers to the estimated time required from the current moment until the temperature of the heat source loading unit container wall reaches the preset thermal failure temperature, calculated based on thermal balance analysis.

[0073] Specifically, after marking all abnormal temperature pixels in the current sampling period in step S104, the three-dimensional spatial coordinates of all abnormal temperature pixels are extracted from the updated four-dimensional data field. These three-dimensional spatial coordinates describe the distribution of each abnormal temperature pixel in the three-dimensional space of the cargo hold. Since multiple independent local heat source areas may exist simultaneously in the cargo hold, such as different cargo loading units at different loading positions experiencing independent temperature anomalies, the abnormal temperature pixels generated by different heat source areas are distributed in different spatial locations in the three-dimensional space and have obvious spatial intervals between them. It is necessary to group spatially adjacent abnormal temperature pixels into the same group to distinguish different heat source areas. Spatial clustering operation is performed on the three-dimensional spatial coordinates of all abnormal temperature pixels. This clustering operation is based on the three-dimensional Euclidean distance metric. For any two abnormal temperature pixels, the Euclidean distance between their three-dimensional spatial coordinates is calculated. Abnormal temperature pixels with a spatial distance less than the preset clustering radius are grouped into the same cluster. At the same time, it is required that the number of abnormal temperature pixels in each cluster is not less than the preset minimum cluster size threshold to filter out isolated scattered points caused by measurement noise. Each cluster that meets the minimum cluster size requirement represents an independent local heat source area.

[0074] After obtaining the clustering results, the spatial distribution characteristics of each cluster are matched with the preset loading unit model. The preset loading unit model stores the three-dimensional spatial occupancy range of cargo loading units at various loading positions within the cargo hold. This occupancy range is typically described in the form of an axis-aligned bounding box or an oriented bounding box, including the center position coordinates, length, width, and height dimensions, and orientation information of the loading unit in the cargo hold coordinate system. The centroid coordinates and spatial envelope range of the three-dimensional spatial coordinates of all abnormal temperature pixels in each cluster are calculated. Then, the spatial occupancy range of each loading unit in the preset loading unit model is traversed to determine whether the centroid coordinates of the cluster fall within the spatial occupancy range of a certain loading unit, or whether the spatial overlap volume between the spatial envelope range of the cluster and the spatial occupancy range of a certain loading unit exceeds a preset overlap ratio threshold. If the above conditions are met, the target cargo loading unit to which the cluster belongs is determined, and the identification number of the target cargo loading unit is obtained. For each cluster of abnormal temperature pixels identified as belonging to a target cargo loading unit, a temperature anomaly alarm is generated. This alarm includes the identification number of the target cargo loading unit, the three-dimensional spatial coordinates of each abnormal temperature pixel in the cluster, and a statistical summary of the target true temperature of the abnormal temperature pixels in the cluster. This allows the crew to accurately know which specific cargo loading unit the temperature anomaly occurred on and the specific spatial location of the abnormal hot spot on the surface of the loading unit, even when they cannot enter the Category C cargo hold. This provides precise spatial location information for assessing the location of the hazard and deciding on targeted isolation or firefighting measures.

[0075] In some embodiments, after generating a temperature anomaly alarm, thermal failure prediction analysis is further performed to provide the crew with quantified time window information. Based on the current sampling period, in the updated four-dimensional data field of each sampling period within a preset time window, all pixels whose three-dimensional spatial coordinates are within a preset spatial neighborhood of the abnormal temperature pixel are extracted. The reason for using a spatial neighborhood rather than strict pixel coordinate consistency for extraction is that the cargo loading unit may undergo slight displacement due to airflow turbulence during flight, causing slight shifts in the pixel coordinates of the same physical surface point in different sampling periods. The spatial neighborhood setting can accommodate such slight displacements, ensuring that the extracted temperature observations of the same physical area at different times are obtained. The extracted pixels are arranged chronologically according to the target true temperature in each sampling period to form a temperature time series sequence of the abnormal temperature pixel. Then, a linear regression is performed on this temperature time series sequence, using the sampling time as the independent variable and the target true temperature as the dependent variable to solve for the least-squares regression line, obtaining the temperature time series slope of the abnormal temperature pixel within the preset time window.

[0076] If the temperature time-series slope exceeds a preset slope threshold, it indicates that the temperature in the region where the abnormal temperature pixel is located is showing a continuous upward trend, and the rate of increase exceeds the range that can be explained by normal ambient temperature drift. This indicates the presence of a persistent heat source in the region, and the heat diffusion analysis process is initiated. A heat source loading unit point set is constructed based on the three-dimensional spatial coordinates of all pixels belonging to the target cargo loading unit. This point set describes the observable surface geometry of the cargo loading unit identified as a heat source in three-dimensional space. All adjacent pixels in three-dimensional space whose distance to each point in the heat source loading unit point set is within a preset contact distance range. This preset contact distance is based on the fact that the physical gap between adjacent loading units in the cargo hold is typically in the range of several centimeters to over ten centimeters. From these adjacent pixels, pixels belonging to other cargo loading units different from the target cargo loading unit to which the abnormal temperature pixel belongs are selected, and their three-dimensional spatial coordinates are used to form a heat diffusion candidate point set. The pixels in the heat diffusion candidate point set represent the contact interface area of ​​adjacent loading units that may be affected by heat conduction or heat radiation from the heat source loading unit. The temperature time sequence within a preset time window is also extracted for each pixel in the heat diffusion candidate point set, and the temperature time sequence slope is calculated. At the same time, the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set is calculated. This contact area is obtained by accumulating the surface area of ​​the spatial patch formed by the paired points between the two point sets whose distance is less than the preset contact spacing.

[0077] In some embodiments, the specific process for calculating the predicted remaining time for the container wall of the heat source loading unit to reach the preset thermal failure temperature is as follows: Based on the temperature temporal slope of each pixel in the heat diffusion candidate point set within the preset time window and the heat capacity parameters of adjacent cargo loading units, the product of the temperature temporal slope of the adjacent loading unit to which the heat diffusion candidate point set belongs and the specific heat capacity and mass of the adjacent loading unit is determined as the heat absorption power absorbed by the adjacent loading unit from the heat source loading unit within the preset time window. The physical meaning of this heat absorption power is the input rate of heat required for the temperature of the adjacent loading unit to rise per unit time. Dividing this heat absorption power by the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set yields the interface heat flux density, which characterizes the heat conducted from the heat source loading unit to the adjacent loading unit per unit contact area per unit time. Based on the interfacial heat flux density and the temperature time-series slope of the heat source loading unit itself, the heat release power inside the heat source loading unit is determined through a heat balance equation. This equation expresses the physical relationship that the heat release power inside the heat source loading unit is equal to the heat power consumed by the heat source loading unit's own temperature rise (i.e., the product of the heat source loading unit's specific heat capacity, mass, and temperature time-series slope), plus the heat power conducted through the contact interface to all adjacent loading units (i.e., the product of the interfacial heat flux density and the contact area). Based on the heat release power, the difference between the current target true temperature of the heat source loading unit and the preset thermal failure temperature of the cargo loading unit container wall, and the preset specific heat capacity and wall thickness mass parameters of the cargo loading unit container wall, the net heat power is obtained by dividing the product of the container wall's specific heat capacity, wall thickness mass, and temperature difference by the heat release power minus the heat power conducted to adjacent loading units. This yields the predicted remaining time required for the heat source loading unit container wall to rise from its current temperature to its thermal failure temperature, and this predicted remaining time is added to the temperature anomaly alarm.

[0078] In another embodiment, the remaining time is calculated using an extrapolation method based on temperature time series. A nonlinear function is fitted to the temperature time series of all pixels in the heat source loading unit point set, selecting an exponential growth function or a polynomial function as the fitting model. The parameters of the fitting model are determined using the least squares method, resulting in a fitted curve of temperature changing over time. This fitted curve is then extrapolated along the time axis towards the future, and the time difference between the future time point corresponding to the first time the temperature value of the fitted curve reaches the preset thermal failure temperature and the current sampling time is calculated as the remaining time. This method does not rely on the thermal diffusion analysis and heat capacity parameters of adjacent loading units, but directly predicts based on the temperature evolution trend of the heat source itself. It is suitable for scenarios where there are no adjacent loading units around the heat source loading unit, or where the thermal diffusion information of adjacent loading units is unobservable due to viewpoint obstruction. It is understood that other thermodynamic analysis methods or data-driven methods can also be used to calculate the remaining time; this is not limited here.

[0079] In some embodiments, after adding the predicted remaining time to the temperature anomaly alarm, alarm priority ranking in multi-heat-source scenarios is further performed. When it is determined that multiple abnormal temperature pixels belong to different cargo loading units in the four-dimensional data field of the current sampling period, due to limitations in camera installation position and viewing angle, as well as mutual occlusion between cargo loading units within the cargo hold, the observability of the contact interface between different loading units varies in the four-dimensional data field. Some contact interfaces may be occluded by other loading units, and only some areas can be observed by the camera. Based on the ratio of the number of observable pixels that can actually be observed in the four-dimensional data field to the theoretical total number of pixels of the three-dimensional contact surface, this ratio is determined as the observable integrity coefficient of the contact interface of the corresponding cargo loading unit. The theoretical total number of pixels is calculated as follows: the theoretical contact surface between two loading units is determined based on the intersection region of the three-dimensional spatial envelope of the heat source loading unit point set and the three-dimensional spatial envelope of the heat diffusion candidate point set; the area of ​​the theoretical contact surface is calculated; and then this area is divided by a preset pixel spatial resolution, i.e., the physical area corresponding to a single pixel at that distance, to obtain the theoretical total number of pixels. The contact interface observable integrity coefficient ranges from 0 to 1. A smaller value indicates a greater degree of obstruction of the contact interface, resulting in less complete observable thermal diffusion information and suggesting that the actual thermal diffusion situation may be more severe than observed. Dividing the predicted remaining time for each cargo loading unit by its contact interface observable integrity coefficient yields a corrected priority score for that unit. This correction reflects the engineering principle of conservative assessment in aviation safety, where dividing by the observable integrity coefficient shortens the equivalent priority time for loading units with lower contact interface observable integrity, as their actual thermal diffusion rate may be higher than the rate estimated based on incomplete observation data. The corrected priority score sorts the temperature anomaly alarms for each cargo loading unit in ascending order of corrected priority score. A smaller score indicates a more urgent situation requiring priority handling. The sorted alarm list is transmitted to the cockpit display terminal for the crew to assess the situation of each loading unit according to priority and decide on appropriate emergency response measures.

[0080] In some embodiments, spatial clustering of abnormal temperature pixels and matching of clustering results with a preset loading unit model can be achieved in various ways. Optionally, a density-based spatial clustering method is adopted, using the three-dimensional spatial coordinates of each abnormal temperature pixel as a data point, setting two parameters: neighborhood search radius and minimum number of neighborhood points. Starting from any unvisited data point, all data points within its neighborhood search radius are searched. If the number of data points in the neighborhood is not less than the minimum number of neighborhood points, the data point is marked as a core point and all data points in its neighborhood are assigned to the same cluster. Then, the same neighborhood search and expansion operation is recursively performed on each data point in the newly assigned cluster until the cluster no longer expands. Finally, data points with fewer data points in their neighborhood than the minimum number of neighborhood points and not belonging to the neighborhood of any core point are marked as noise points and removed. This method can discover clusters of arbitrary shapes without pre-specifying the number of clusters, and is suitable for scenarios where the abnormal heat source area in the cargo hold has an irregular shape. Optionally, a hierarchical agglomerative spatial clustering method can be adopted. Initially, each anomalous temperature pixel is treated as an independent cluster. Then, the three-dimensional spatial distance between all cluster pairs is calculated iteratively, and the two closest clusters are merged into a new cluster. This merging operation is repeated until the minimum distance between all cluster pairs exceeds a preset merging distance threshold. The remaining clusters are the final clustering results. This method can generate a hierarchical clustering structure, and the merging distance threshold can be set according to the typical spacing between loading units in the cargo hold. It is understood that other spatial clustering algorithms can also be used to group anomalous temperature pixels; this is not limited here.

[0081] It should be noted that the spatial occupancy range of each loading unit in the preset loading unit model needs to be updated after each flight loading is completed based on the actual cargo loading list and loading location information to ensure that the spatial occupancy range in the model is consistent with the actual loading layout in the cargo hold. This update process can be automatically completed by the ground operating system during the ground loading phase and uploaded to the airborne monitoring system.

[0082] In some embodiments, in actual aircraft cargo hold monitoring scenarios, due to the compact interior space and small spacing between loading units, abnormal temperature pixels belonging to different loading units may be incorrectly grouped into the same cluster during spatial clustering because they are too close in space. This is especially true when temperature anomalies occur simultaneously at the contact interface of two adjacent loading units, where the abnormal temperature pixels on both sides are almost continuously distributed in three-dimensional space, making it difficult for pure distance-based clustering methods to separate them correctly. Before performing spatial clustering, all abnormal temperature pixels are pre-partitioned using the spatial occupancy range of each loading unit in the pre-defined loading unit model. That is, for each abnormal temperature pixel, it is determined which loading unit's spatial occupancy range its three-dimensional spatial coordinates fall into and it is assigned a corresponding loading unit pre-label. Then, during spatial clustering, the loading unit pre-label is used as an additional constraint condition, allowing only abnormal temperature pixels with the same pre-label to be grouped into the same cluster, thereby avoiding incorrect clustering across loading units and ensuring that the abnormal temperature pixels in each cluster belong to the same cargo loading unit.

[0083] In this embodiment, multimodal infrared fusion technology using a long-wave thermal infrared camera and a near-infrared depth camera is employed to register the three-dimensional spatial point cloud and the two-dimensional radiation temperature map pixel by pixel in a unified coordinate system and construct a four-dimensional data field. Combined with the real-time changing cabin environmental parameters during flight, atmospheric transmittance correction and dynamic baseline tracking are applied to the radiation temperature measurement results. This enables the detection of temperature anomalies to locate specific cargo loading units in three-dimensional space, effectively solving the technical problem in related technologies where it is difficult to determine the specific cargo loading unit to which the abnormal heat source belongs due to the difficulty in removing environmental interference during flight. This improves the accuracy of aircraft cargo hold temperature anomaly monitoring and spatial positioning precision.

[0084] In the above embodiments, multimodal infrared fusion and adaptive environmental compensation can achieve accurate detection and three-dimensional spatial positioning of cargo hold temperature anomalies. In practical applications, when implementing the above-mentioned aircraft cargo hold monitoring method based on infrared fusion technology, smoke aerosols are often generated in the early stages of a cargo hold fire. The smoke simultaneously scatters the structured light or time-of-flight signals emitted by the near-infrared depth camera, resulting in a large number of false three-dimensional points in the depth measurement. It also attenuates long-wave infrared radiation, causing a drop in the global apparent temperature. This causes the system to lose its temperature measurement accuracy due to smoke interference in the early stages of a fire, when accurate monitoring is most needed. The aircraft cargo hold monitoring method based on infrared fusion technology can solve this technical problem by using three-dimensional point cloud temporal registration between adjacent sampling periods and a smoke scattering masking event detection mechanism. After detecting a smoke event, it triggers a correction operation on atmospheric transmittance and re-executes temperature separation, improving the accuracy of temperature measurement in smoke environments and the reliability of early fire monitoring.

[0085] Please see Figure 2This is another flowchart illustrating an aircraft cargo hold monitoring method based on infrared fusion technology in an embodiment of this application.

[0086] S201. Based on the pre-calibrated extrinsic transformation matrix, the three-dimensional point cloud inside the cargo hold acquired by the near-infrared depth camera is transformed from the coordinate system of the near-infrared depth camera to the coordinate system of the long-wave thermal infrared camera to obtain the aligned three-dimensional point cloud.

[0087] S202. Using the intrinsic parameter matrix of the long-wave thermal infrared camera, project each three-dimensional point in the aligned three-dimensional point cloud onto the pixel plane corresponding to the two-dimensional radiation temperature map synchronously acquired by the long-wave thermal infrared camera, and construct a four-dimensional data field containing three-dimensional spatial components and temperature components.

[0088] S203. After determining the current atmospheric transmittance based on the cargo hold temperature, relative humidity and atmospheric pressure within the current sampling period, the corresponding target real temperature is separated from the radiation temperature value of each pixel in the four-dimensional data field through the current atmospheric transmittance and the basic equation of infrared radiation thermometry, and the four-dimensional data field is updated with the target real temperature.

[0089] S204. Based on the updated four-dimensional data field of each sampling period within the preset time window, calculate the mean temperature and standard deviation of temperature for each pixel within the time window, and mark pixels whose target true temperature deviates from the mean temperature by more than a preset multiple of the standard deviation of temperature in the current sampling period, or pixels whose target true temperature changes at a rate exceeding a preset rate threshold between adjacent sampling periods, as abnormal temperature pixels.

[0090] S205. After obtaining the clustering results by performing spatial clustering based on the three-dimensional spatial coordinates of the abnormal temperature pixels in the updated four-dimensional data field, the clustering results are matched with the preset loading unit model to determine the target cargo loading unit to which the abnormal temperature pixels belong, and a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixels is generated.

[0091] Steps S201 to S205 and Figure 1 Steps S101 to S105 in the illustrated embodiment are similar and can be found in the descriptions of steps S101 to S105, which will not be repeated here.

[0092] S206. Use the four-dimensional data field of the previous sampling period as the reference data field.

[0093] The previous sampling period refers to the complete data acquisition and processing unit immediately preceding the current sampling period in the time series. Within this sampling period, the near-infrared depth camera and the long-wave thermal infrared camera have completed one synchronous data acquisition, and have completed the entire processing flow from coordinate transformation, four-dimensional data field construction to target true temperature separation, storing the results in the time series data buffer. The reference data field refers to the four-dimensional data field of the previous sampling period selected as the data quality assessment benchmark for the current sampling period. This reference data field contains two types of information: reference three-dimensional spatial coordinates and reference true temperature for each pixel. The reference three-dimensional spatial coordinates refer to the three-dimensional spatial position of each pixel in the long-wave thermal infrared camera coordinate system in the previous sampling period, and the reference true temperature refers to the target true temperature value of each pixel in the previous sampling period after atmospheric transmittance correction.

[0094] Specifically, after updating the target true temperature of the four-dimensional data field for the current sampling period in step S203, a time-series comparison benchmark needs to be established to assess whether the data for the current sampling period is affected by abnormal environmental factors such as smoke. The four-dimensional data field from the previous sampling period, which has already completed all processing steps, is read from the time-series data buffer and designated as the reference data field for the current sampling period. The reference three-dimensional spatial coordinates in this reference data field reflect the positional distribution of the surfaces of various objects within the cargo hold in three-dimensional space during the previous sampling period, and the reference true temperature reflects the actual temperature state of the surfaces of various objects after environmental compensation during the previous sampling period. While designating the four-dimensional data field from the previous sampling period as the reference data field, the four-dimensional data field for the current sampling period remains unchanged. The two data fields will undergo pixel-by-pixel three-dimensional spatial registration and temperature statistical comparison in subsequent steps.

[0095] Understandably, the reason for choosing the previous sampling period rather than an earlier one as the reference benchmark is that the time interval between adjacent sampling periods is the shortest. Within this short time interval, the physical state of the cargo in the cargo hold changes the least. Under normal circumstances, the three-dimensional point cloud and temperature distribution between two adjacent sampling periods should be highly consistent, with only minor overall displacement caused by flight vibration and minor temperature changes caused by slow drift of environmental parameters. Any significant difference that exceeds this normal range of minor changes may indicate the occurrence of an abnormal environmental event.

[0096] In some embodiments, the selection and establishment of the reference data field can be achieved in several ways. Optionally, the complete four-dimensional data field of the previous sampling period stored in the time-series data buffer can be directly used as the reference data field without any preprocessing. This method is simple to implement and has minimal computational overhead, making it suitable for normal flight phases with short sampling intervals and gradual changes in the cargo hold environment. Optionally, four-dimensional data fields from the most recent consecutive sampling periods can be read from the time-series data buffer. The moving average of the three-dimensional spatial coordinates of each pixel and the target's true temperature in these four-dimensional data fields can be calculated over time. The three-dimensional spatial coordinates and the target's true temperature after the moving average can be combined to form a smoothed reference data field as a reference benchmark. This method suppresses measurement noise that may exist in a single sampling period through time smoothing, making the reference benchmark more stable. It is suitable for scenarios where severe flight turbulence causes large fluctuations in data within a single sampling period. It is understood that other methods can also be used to select or construct the reference data field, such as selecting the four-dimensional data field of the most stable sampling period within the flight phase based on the flight phase identifier as the reference benchmark. This is not limited here.

[0097] S207. Register the three-dimensional spatial coordinates of each pixel in the four-dimensional data field of the current sampling period with the reference three-dimensional spatial coordinates, and solve for the spatial rigid body displacement matrix from the reference data field to the current sampling period.

[0098] The reference 3D spatial coordinates refer to the 3D spatial positions of each pixel in the reference data field determined in step S206 within the long-wave thermal infrared camera coordinate system. This coordinate set describes the spatial position distribution of the surfaces of each object in the cargo hold during the previous sampling period. Registration refers to finding an optimal spatial transformation relationship in 3D space that minimizes the overall spatial deviation between the reference 3D spatial coordinate set and the 3D spatial coordinate set of the current sampling period after the transformation. The spatial rigid body displacement matrix is ​​a parameter matrix describing the rigid body transformation relationship between the 3D spatial coordinates of the reference data field and the 3D spatial coordinates of the current sampling period. This matrix contains a rotation component and a translation component. The rotation component describes the small rotational changes of the overall scene in the cargo hold between two sampling periods, and the translation component describes the small translational changes of the overall scene in the cargo hold between two sampling periods. Together, they characterize the changes in the overall spatial pose of the scene caused by normal physical factors such as flight vibration, airflow turbulence, or minor cargo displacement between adjacent sampling periods.

[0099] After establishing the reference data field in step S206, it is necessary to spatially register the 3D point cloud of the current sampling period with that of the previous sampling period to eliminate the slight overall displacement of the scene caused by normal physical factors between the two sampling periods. During flight, the aircraft may experience slight attitude changes due to turbulence, maneuvering, or engine vibration. These attitude changes are transmitted to the cargo loading unit in the cargo hold, causing slight rigid body displacement and rotation of the cargo as a whole. This results in slight differences in the 3D coordinates of the same object surface point measured by the near-infrared depth camera in two adjacent sampling periods. If this normal slight displacement difference is not eliminated, the subsequent step S209, when detecting out-of-position pixels, will misjudge the coordinate deviation caused by normal vibration displacement as a false depth point generated by smoke scattering, thus triggering a false judgment of smoke events.

[0100] Specifically, the three-dimensional spatial coordinates of all pixels in the four-dimensional data field of the current sampling period are extracted to form the current point cloud set, and the reference three-dimensional spatial coordinates of all pixels in the reference data field are extracted to form the reference point cloud set. Then, a three-dimensional spatial registration operation is performed on these two point cloud sets. The goal of the registration operation is to solve for a spatial rigid body displacement matrix containing rotation and translation vectors, such that the overall spatial deviation between the reference point cloud set and the current point cloud set after the rigid body transformation is minimized. The iterative nearest point method or its variant is used for registration. In each iteration, this method first searches for the nearest corresponding point in the current point cloud set for each point in the reference point cloud set to establish a point pair relationship. Then, based on all point pairs, the least squares method is used to solve for the rigid body transformation parameters that minimize the sum of squared distances between point pairs. The reference point cloud set is then updated with the transformed position, and the nearest corresponding point is searched again. This process is iterated until the update amount of the transformation parameters is less than a preset convergence threshold or the number of iterations reaches a preset upper limit. Finally, the converged spatial rigid body displacement matrix is ​​output. The spatial rigid body displacement matrix accurately describes the minute spatial pose changes of the entire scene inside the cargo hold from the previous sampling period to the current sampling period, providing transformation parameters for the subsequent step S208 to construct the transformation reference point set.

[0101] In some embodiments, the 3D point cloud registration and the solution of the spatial rigid body displacement matrix between the current sampling period and the reference data field can be achieved in various ways. Optionally, an iterative nearest-neighbor registration method is adopted. First, voxel downsampling is performed on the current point cloud set and the reference point cloud set to reduce the point cloud density and thus reduce the amount of computation. Then, iterative nearest-neighbor registration is performed with the downsampled reference point cloud set as the source point cloud and the downsampled current point cloud set as the target point cloud. In each iteration, the nearest neighbor search is accelerated by using a kd-tree to improve the point pair matching efficiency. In the point pair matching stage, a distance threshold is introduced to remove abnormal point pairs with excessively large distances to improve the registration robustness. After the iteration converges, the spatial rigid body displacement matrix is ​​output. Optionally, a registration method based on normal vector features is adopted. First, the local surface normal vector of each point in the current point cloud set and the reference point cloud set is estimated. Then, in the registration process, not only the Euclidean distance between corresponding points is minimized, but also the angular deviation between the normal vectors of corresponding points is minimized. By introducing normal vector constraints, the registration results can also obtain accurate tangential alignment in flat surface areas. Finally, the spatial rigid body displacement matrix is ​​solved by jointly optimizing the position error and the normal vector error.

[0102] It is understandable that other 3D point cloud registration algorithms can also be used to solve the spatial rigid body displacement matrix, such as a method that combines coarse registration based on global feature descriptors with local fine registration, which is not limited here.

[0103] S208. The set of points formed by transforming the reference three-dimensional spatial coordinates through the spatial rigid body displacement matrix is ​​used as the transformation reference set.

[0104] The reference 3D spatial coordinates refer to the set of 3D spatial positions of each pixel in the reference data field determined in step S206 within the long-wave thermal infrared camera coordinate system. This set describes the original spatial position distribution of each object surface within the cargo hold during the previous sampling period. The spatial rigid body displacement matrix refers to the rigid body transformation parameter matrix obtained in step S207 through 3D point cloud registration, which describes the minute spatial pose changes of the scene as a whole between the reference data field and the current sampling period. The transformation reference point set refers to the new set of 3D spatial points obtained by performing rotation and translation transformations on each 3D point in the reference 3D spatial coordinates as defined by the spatial rigid body displacement matrix. This set of points represents the expected position of each pixel in the reference data field within the spatial reference frame of the current sampling period after eliminating the normal overall displacement differences between adjacent sampling periods.

[0105] Specifically, after solving for the spatial rigid body displacement matrix in step S207, the three-dimensional spatial coordinates in the reference data field need to be transformed to the spatial reference frame of the current sampling period. This allows the subsequent step S209 to compare the deviation between the actual and expected three-dimensional coordinates of the current sampling period within the same spatial reference frame. The reference three-dimensional spatial coordinates of all pixels in the reference data field are traversed. For each reference three-dimensional spatial coordinate vector, a rotation and translation transformation defined by the spatial rigid body displacement matrix is ​​applied. This involves first left-multiplying the coordinate vector by the rotation component of the spatial rigid body displacement matrix to align the spatial orientation, and then superimposing the translation component to align the spatial position. This maps the reference three-dimensional spatial coordinates from the spatial state of the previous sampling period to the spatial state of the current sampling period. Once all the reference three-dimensional spatial coordinates of all pixels in the reference data field have undergone the above transformations, all the transformed three-dimensional spatial points are aggregated to form a transformation reference point set. The three-dimensional coordinates of each point in the transformation reference point set represent the expected spatial position of that point in the current sampling period after eliminating normal overall displacement factors. If the cargo hold scene in the current sampling period is not affected by abnormal factors such as smoke, then the actual three-dimensional spatial coordinates of each pixel in the current four-dimensional data field should be highly consistent with the expected spatial position of the corresponding point in the transformation reference point set. The deviation between the two only comes from normal factors such as measurement noise and local small deformations, and the deviation should be within the preset neighborhood range. Conversely, if smoke aerosols appear in the cargo hold, the smoke particles will scatter the structured light or time-of-flight signal emitted by the near-infrared depth camera, causing a large number of pixels to have serious deviations in depth measurement values, thus generating a large number of false three-dimensional points. These false three-dimensional points are spatially deviated from the position of the real object surface. Therefore, the actual three-dimensional spatial coordinates of these pixels in the current four-dimensional data field will significantly deviate from the expected spatial position of the corresponding point in the transformation reference point set, and the deviation will exceed the preset neighborhood range. These pixels that exceed the preset neighborhood range are the out-of-position pixels that need to be detected in the subsequent step S209.

[0106] In some embodiments, the process of constructing a transformation reference point set after transforming the reference three-dimensional spatial coordinates with a spatial rigid body displacement matrix can be implemented in various ways. Optionally, matrix multiplication and vector addition operations are performed point-by-point on the reference three-dimensional spatial coordinates of all pixels in the reference data field. The transformed coordinates are obtained by left-multiplying each reference three-dimensional spatial coordinate vector by a rotation matrix and adding a translation vector. Then, all the transformed coordinates are organized into a transformation reference point set with the same pixel index structure as the reference data field. This allows subsequent steps to directly establish a one-to-one correspondence between each pixel in the current four-dimensional data field and the corresponding point in the transformation reference point set through the pixel index. Optionally, a spatial index structure is constructed on the transformation reference point set while performing coordinate transformation. The transformed three-dimensional spatial points are inserted into a three-dimensional spatial index tree according to their spatial positions. This spatial index tree supports efficient neighborhood range queries, enabling subsequent step S209 to quickly complete the neighborhood search through the spatial index tree when determining whether the three-dimensional spatial coordinates of each pixel in the current four-dimensional data field are within the preset neighborhood range of the transformation reference point set without traversing all transformation reference points, thereby significantly reducing the computational complexity of out-of-position pixel detection.

[0107] It is understandable that other data organization methods can also be used to construct the transformation reference point set, such as storing the transformed 3D spatial points in buckets according to the spatial voxel grid to support fast neighborhood queries based on voxels, which is not limited here.

[0108] S209. In a four-dimensional data field with a current sampling period, if the proportion of pixels whose three-dimensional spatial coordinates are outside the preset neighborhood of the transformation reference point set exceeds a preset discrete proportion threshold, and the difference between the mean of the target true temperature of all pixels in the four-dimensional data field and the mean of the reference true temperature is less than a negative preset temperature drop threshold, then a smoke scattering and masking event is determined to have occurred in the cargo hold, and a correction operation on the current atmospheric transmittance is triggered.

[0109] The preset neighborhood range refers to a spherical or cubic spatial region in three-dimensional space, centered on the three-dimensional spatial coordinates of each transformation reference point in the transformation reference point set. The radius or side length of this region is a preset neighborhood distance parameter, used to define the allowable range of normal deviation between the actual three-dimensional spatial coordinates of each pixel in the current four-dimensional data field and the corresponding expected position in the transformation reference point set. Pixels falling within this range are considered to have normal spatial positions, while pixels falling outside this range are considered to have abnormal spatial positions, i.e., out-of-position pixels. The discrete proportion threshold refers to a preset upper limit for the proportion of out-of-position pixels to the total number of valid pixels. When the proportion of out-of-position pixels exceeds this threshold, it indicates that the depth measurement of a large number of pixels in the current sampling period is affected by systematic interference rather than random measurement noise of individual pixels. Smoke scattering masking events refer to abnormal environmental events in which smoke aerosols generated by early fires or other causes diffuse into the air inside the cargo hold, causing dual interference to depth measurements by near-infrared depth cameras and radiation temperature measurements by long-wave thermal infrared cameras. Smoke particles scatter near-infrared structured light or time-of-flight signals, resulting in a large number of false 3D points in depth measurements, manifesting as spatial anomalies. Smoke particles attenuate long-wave infrared radiation, leading to a reduction in the radiation energy received by thermal infrared cameras, manifesting as a global temperature drop. The simultaneous occurrence of these two interference effects constitutes the characteristic manifestation of smoke scattering masking events.

[0110] Specifically, after completing step S208 to construct the transformation reference point set, it is necessary to jointly determine whether a smoke scattering and masking event has occurred in the cargo hold based on the spatial deviation characteristics between the current four-dimensional data field and the transformation reference point set, as well as the temperature change characteristics between the current four-dimensional data field and the reference data field.

[0111] Specifically, spatial anomaly detection is first performed. This involves traversing all pixels with valid 3D spatial coordinates in the current sampling period's four-dimensional data field. For each pixel, the nearest transformation reference point to its 3D spatial coordinates is searched within the transformation reference point set. The Euclidean distance between the pixel's actual 3D spatial coordinates and the nearest transformation reference point's 3D spatial coordinates is calculated, and it is determined whether this Euclidean distance exceeds a preset neighborhood distance parameter. If it does, the pixel is marked as an out-of-place pixel. The total number of out-of-place pixels is counted, and their proportion to the total number of valid pixels is calculated to obtain the out-of-place pixel percentage. Then, it is determined whether this out-of-place pixel percentage exceeds a preset discrete percentage threshold. Under normal flight conditions, since the registration operation in step S207 has eliminated the overall rigid body displacement difference between adjacent sampling periods, the deviation between the actual 3D spatial coordinates of each pixel in the current four-dimensional data field and the corresponding expected position in the transformation reference point set only originates from depth measurement noise and minor local deformations of the cargo. Therefore, the number of out-of-place pixels should be extremely small, and their percentage should be far below the discrete percentage threshold. However, when smoke aerosols appear in the cargo hold, smoke particles diffuse in the space between the camera and the cargo surface. The structured light or time-of-flight signal emitted by the near-infrared depth camera is scattered by the smoke particles during propagation. Some signals are scattered to non-target directions, causing time delay or spatial encoding distortion of the echo signal. The depth calculation results show serious deviations, generating a large number of false depth points whose three-dimensional coordinates deviate from the actual object surface position. After the coordinate transformation in step S201 and the projection association in step S202, these false depth points enter the four-dimensional data field. Their three-dimensional spatial coordinates deviate significantly from the expected position corresponding to the transformation reference point set and are detected as out-of-position pixels, resulting in a significant increase in the proportion of out-of-position pixels and exceeding the discrete proportion threshold.

[0112] After spatial anomaly detection, temperature anomaly detection is further performed. The arithmetic mean of the target's true temperature for all pixels in the four-dimensional data field during the current sampling period is calculated to obtain the current global temperature mean. Simultaneously, the arithmetic mean of the reference true temperature for all pixels is obtained from the reference data field to obtain the reference global temperature mean. Then, the difference between the current global temperature mean and the reference global temperature mean is calculated. Under normal flight conditions, the change in the global temperature mean between two adjacent sampling periods is very small, and the difference typically fluctuates within a small positive or negative range. However, when smoke aerosols appear in the cargo hold, the smoke particles scatter and absorb long-wave infrared radiation, causing the infrared radiation emitted from the cargo surface to be partially attenuated by the smoke particles on the path to the long-wave thermal infrared camera detector. The radiation energy received by the detector is reduced, which is reflected in a systematic decrease in the radiation temperature values ​​of all pixels in the two-dimensional radiation temperature map. Even after the atmospheric transmittance correction in step S203, the target's true temperature is still too low because the atmospheric transmittance used in step S203 is calculated based on normal atmospheric conditions and does not consider the additional attenuation effect of smoke. This results in a significant decrease in the current global temperature average relative to the reference global temperature average, with a negative difference and its absolute value exceeding the preset temperature decrease threshold.

[0113] A smoke scattering and masking event in the cargo hold is determined when both conditions are met: the proportion of out-of-position pixels exceeds a preset discrete proportion threshold, and the difference between the current global average temperature and the reference global average temperature is less than a negative preset temperature drop threshold. A single spatial anomaly criterion might be triggered by non-smoke causes such as near-infrared depth camera malfunction or large-scale cargo displacement within the cargo hold. A single temperature drop criterion might be triggered by a systematic change in cabin temperature due to flight phase switching. Only when both spatial anomalies and temperature drops occur simultaneously is it highly indicative of a smoke scattering and masking event. After determining a smoke scattering and masking event, a correction operation is triggered on the current atmospheric transmittance. This correction operation aims to incorporate the additional attenuation effect of smoke aerosols on the long-wave infrared radiation transmission path into the atmospheric transmittance calculation. Based on the corrected atmospheric transmittance, the temperature separation calculation of the fundamental equations for infrared radiation thermometry is re-executed to obtain a more accurate target temperature in a smoke environment.

[0114] In one embodiment, the specific process of triggering the correction operation for the current atmospheric transmittance is as follows: First, all out-of-place pixels whose three-dimensional spatial coordinates are outside the preset neighborhood range of the transformation reference point set in the four-dimensional data field of the current sampling period are extracted. The three-dimensional spatial coordinates of these out-of-place pixels are used to form a smoke spatial sampling point set. The distribution position of each sampling point in the three-dimensional space of this smoke spatial sampling point set approximately reflects the spatial distribution state of smoke particles in the cargo hold, because the false depth value of the out-of-place pixels is generated by the near-infrared signal scattered by smoke particles, and the spatial position of the false depth point roughly corresponds to the location where the scattering of smoke particles occurs. The local concentration of smoke is characterized by the local point density of each sampling point in the three-dimensional space of the smoke spatial sampling point set. The higher the local point density, the denser the smoke particles and the higher the smoke concentration in that area. For each sampling point, the number of neighboring sampling points within the preset search radius is calculated and the number is normalized to a local point density value. Then, using the local point density of each sampling point as the node value, a continuous three-dimensional smoke concentration distribution field is constructed in the entire three-dimensional space of the cargo hold through a three-dimensional spatial interpolation method. This distribution field describes the estimated smoke concentration at each three-dimensional spatial location in the cargo hold. For each pixel in the four-dimensional data field, the three-dimensional spatial coordinates of the optical center of the long-wave thermal infrared camera are determined as the starting point of the line-of-sight path, and the corresponding three-dimensional spatial coordinates of the pixel in the four-dimensional data field are determined as the ending point of the line-of-sight path. Path integration is performed on the three-dimensional smoke concentration distribution field along the line-of-sight path, and the smoke concentration values ​​at each position along the path length are accumulated and summed to obtain the smoke optical thickness corresponding to that pixel. The optical thickness of the smoke represents the total attenuation of long-wave infrared radiation as it propagates from the cargo surface to the camera detector, caused by scattering and absorption by smoke particles. The corrected atmospheric transmittance is obtained by correcting the current atmospheric transmittance using an empirical correction formula. The empirical correction formula will adjust the current atmospheric transmittance. Multiply by the negative of the ratio of cabin pressure to standard atmospheric pressure Power of 1 and multiplied by the optical thickness of the smoke The negative exponential decay factor, where The index is a power-law scale of optical depth to pressure ratio. Under normal atmospheric conditions without smoke, this index takes a preset first calibration value, approximately equal to 1.0. This is because, under normal atmospheric conditions, water vapor absorption along the short-distance transport path in the cargo hold is in the linear weak absorption region of Beer's Law, where the absorption is proportional to the molecular number density, which in turn is proportional to the air pressure. After determining that a smoke scattering and masking event has occurred in the cargo hold, Switch to the preset second calibration value, which was obtained through experimental calibration of the cargo hold in a smoke-laden aerosol environment. This is because the scattering and absorption of smoke particles do not follow a linear relationship with Beer's Law. The specific value of the second calibration value, deviating from 1.0, was determined by simulating the cargo hold pressure range within a sealed calibration chamber with controllable air pressure and injecting a known concentration of smoke aerosol. The change in radiation output of the long-wave thermal infrared camera was measured at different pressure levels and determined through logarithmic fitting. The corrected atmospheric transmittance was substituted into the fundamental equation for infrared radiation thermometry, and temperature separation was performed again on each pixel in the four-dimensional data field. The target's true temperature, corrected for smoke, was separated from the radiation temperature value. This corrected true temperature was then used to update the four-dimensional data field, ensuring that subsequent steps S204 (abnormal temperature pixel detection) and S205 (spatial clustering localization) are based on the more accurate temperature data corrected for smoke.

[0115] In another embodiment, a simplified correction method based on global attenuation factor estimation is used when triggering the correction operation on the current atmospheric transmittance. Instead of constructing a three-dimensional smoke concentration distribution field and pixel-by-pixel path integral, the global attenuation degree of smoke on long-wave infrared radiation is directly estimated based on the difference between the current global temperature mean and the reference global temperature mean. The reference global temperature mean is regarded as the true global temperature benchmark under smoke-free conditions. The fourth power of the ratio of the current global temperature mean to the reference global temperature mean is used as the estimated value of the global radiation power attenuation ratio caused by smoke. The negative value of the natural logarithm of this attenuation ratio is then used as the estimated value of the global equivalent smoke optical thickness. This global equivalent smoke optical thickness is then uniformly applied to all pixels in the four-dimensional data field. The current atmospheric transmittance is multiplied by the negative exponential attenuation factor of this global equivalent smoke optical thickness to obtain the corrected atmospheric transmittance. Finally, the corrected atmospheric transmittance is substituted into the basic equation of infrared radiation thermometry to re-execute the temperature separation calculation. This simplified method assumes that the smoke distribution within the cargo hold is relatively uniform, eliminating the need to calculate the differentiated optical thickness of the smoke pixel by pixel. Its computational complexity is significantly lower than methods based on a three-dimensional smoke concentration distribution field, making it suitable for scenarios with limited computing power or relatively uniform smoke distribution. However, its correction accuracy may be lower than path integral-based methods when the smoke distribution height is uneven. It is understandable that other methods can be used to achieve atmospheric transmittance correction in smoke environments; these are not limited here.

[0116] In some embodiments, the determination of smoke scattering and masking events and the triggering of correction operations can be implemented in multiple ways. Optionally, when performing joint determination based on dual criteria, normalized scores are calculated for the proportion of out-of-position pixels and the global temperature drop amplitude, respectively. The proportion of out-of-position pixels is divided by a discrete proportion threshold to obtain a spatial anomaly normalized score, and the absolute value of the global temperature drop amplitude is divided by a preset temperature drop threshold to obtain a temperature anomaly normalized score. Then, the weighted sum of the two normalized scores is compared with a preset joint determination threshold. If the weighted sum exceeds the joint determination threshold, a smoke event is determined. This method achieves more flexible determination sensitivity adjustment by weighted fusion of information from the two criteria. The weighting coefficients can be engineered according to the actual smoke characteristics of different aircraft models and cargo compartment types. Optionally, the correction operation is not triggered immediately after determining a smoke event, but only after the dual criteria conditions are met for several consecutive sampling periods. This method further reduces the probability of false smoke determinations caused by transient interference through continuous confirmation in the time dimension, ensuring that the correction operation is only triggered when the smoke actually persists. It is understandable that other judgment strategies and triggering mechanisms can be used to detect and respond to smoke scattering and masking events, which are not limited here.

[0117] In this embodiment, a joint detection mechanism based on the dual criteria of three-dimensional point cloud temporal registration and smoke scattering masking events between adjacent sampling periods is adopted. By using the four-dimensional data field of the previous sampling period as the reference data field and solving the spatial rigid body displacement matrix to eliminate the overall small displacement caused by normal flight vibration, the occurrence of smoke scattering masking events is determined based on the simultaneous satisfaction of two conditions: the proportion of out-of-position pixels in the current four-dimensional data field exceeds the discrete proportion threshold and the global temperature mean is significantly lower than the reference true temperature mean. After determination, a correction operation on the current atmospheric transmittance is triggered to compensate for the additional attenuation effect of smoke aerosol on the long-wave infrared radiation transmission path. Then, based on the corrected atmospheric transmittance, the temperature separation calculation of the basic equation of infrared radiation thermometry is re-executed to obtain a more accurate target true temperature in the smoke environment. Therefore, in the early stage of a cargo hold fire when smoke is generated, the dual interference caused by smoke on near-infrared depth measurement and long-wave infrared radiation thermometry can be automatically identified and targeted compensation and correction can be performed. This achieves the technical effect of maintaining high temperature measurement accuracy and anomaly detection reliability in the early smoke dispersal stage of a cargo hold fire.

[0118] The exemplary monitoring device 300 provided in the embodiments of this application is described below. Figure 3 This is an exemplary hardware structure diagram of the monitoring device 300 provided in this application embodiment.

[0119] In some embodiments, the monitoring device 300 is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0120] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0121] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0122] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0123] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0124] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for monitoring aircraft cargo hold based on infrared fusion technology, characterized in that, Applied to monitoring equipment, the method includes: Based on the pre-calibrated extrinsic transformation matrix, the three-dimensional point cloud inside the cargo hold acquired by the near-infrared depth camera is transformed from the coordinate system of the near-infrared depth camera to the coordinate system of the long-wave thermal infrared camera to obtain the aligned three-dimensional point cloud. By using the intrinsic parameter matrix of the long-wave thermal infrared camera, each three-dimensional point in the aligned three-dimensional point cloud is projected onto the pixel plane corresponding to the two-dimensional radiation temperature map synchronously acquired by the long-wave thermal infrared camera, thus constructing a four-dimensional data field containing three-dimensional spatial components and temperature components. After determining the current atmospheric transmittance based on the cargo hold's internal temperature, relative humidity, and atmospheric pressure within the current sampling period, the corresponding target true temperature is separated from the radiation temperature value of each pixel in the four-dimensional data field using the current atmospheric transmittance and the basic equation for infrared radiation thermometry. After updating the four-dimensional data field with the target true temperature, based on the updated four-dimensional data field of each sampling period within a preset time window, calculate the temperature mean and temperature standard deviation of each pixel within the time window, and mark the pixels whose target true temperature deviates from the temperature mean by more than a preset multiple of the temperature standard deviation in the current sampling period, or the pixels whose target true temperature changes at a rate exceeding a preset rate threshold between adjacent sampling periods, as abnormal temperature pixels. After obtaining the clustering result by performing spatial clustering on the three-dimensional spatial coordinates of the abnormal temperature pixel in the updated four-dimensional data field, the clustering result is matched with the preset loading unit model to determine the target cargo loading unit to which the abnormal temperature pixel belongs, and a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixel is generated.

2. The method according to claim 1, characterized in that, The method further includes: When the target aircraft corresponding to the cargo hold is grounded, based on the multi-mode calibration plate located at a preset calibration position within the cargo hold, the thermal infrared image collected by the long-wave thermal infrared camera, and the near-infrared depth image and infrared intensity image collected by the near-infrared depth camera are acquired. The bottom layer of the multi-mode calibration plate is a metal substrate with a surface long-wave infrared emissivity lower than a first emissivity threshold, the middle layer is a heat insulation pad with a thermal conductivity lower than a preset thermal conductivity threshold and hollowed out according to a preset checkerboard pattern, and the top layer is a coating that fills and covers the opening side of the hollowed-out area, has a surface long-wave infrared emissivity higher than a second emissivity threshold, and has absorption characteristics for near-infrared light, wherein the first emissivity threshold is lower than the second emissivity threshold. The first corner point coordinate set is determined based on the infrared radiation grayscale jump between the top and bottom layers in the thermal infrared image. The second corner point coordinate set corresponding to the checkerboard pattern is determined by combining the reflection brightness difference in the infrared intensity image and the depth jump between the hollowed-out area and the non-hollowed-out area in the near-infrared depth image. Registration constraints are constructed using the corresponding corner points of the first and second corner point coordinate sets, and the extrinsic parameter transformation matrix is ​​obtained by solving.

3. The method according to claim 1, characterized in that, After the step of updating the four-dimensional data field with the target true temperature, the method further includes: The four-dimensional data field of the previous sampling period is used as the reference data field, which includes the reference three-dimensional spatial coordinates and reference real temperature of each pixel. The three-dimensional spatial coordinates of each pixel in the four-dimensional data field of the current sampling period are registered with the reference three-dimensional spatial coordinates, and the spatial rigid body displacement matrix from the reference data field to the current sampling period is solved. The set of points formed by transforming the reference three-dimensional spatial coordinates through the spatial rigid body displacement matrix is ​​used as the transformation reference point set; In the four-dimensional data field of the current sampling period, if the proportion of pixels whose three-dimensional spatial coordinates are outside the preset neighborhood range of the transformation reference point set exceeds a preset discrete proportion threshold, and the difference between the mean of the target true temperature of all pixels in the four-dimensional data field and the mean of the reference true temperature is less than a negative preset temperature drop threshold, then it is determined that a smoke scattering and masking event has occurred in the cargo hold, and a correction operation on the current atmospheric transmittance is triggered.

4. The method according to claim 3, characterized in that, The step of triggering the correction operation on the current atmospheric transmittance specifically includes: By extracting all out-of-position pixels whose three-dimensional spatial coordinates are outside the preset neighborhood range of the transformation reference point set in the four-dimensional data field of the current sampling period, a smoke spatial sampling point set is formed. Based on the local point density of each sampling point in the three-dimensional space in the smoke spatial sampling point set, the three-dimensional smoke concentration distribution field of the cargo hold is constructed by spatial interpolation, using the local point density of each sampling point as the node value. For each pixel in the four-dimensional data field, the path integral of the three-dimensional smoke concentration distribution field is performed along the line of sight from the optical center of the long-wave thermal infrared camera to the three-dimensional spatial coordinates corresponding to the pixel to obtain the smoke optical thickness corresponding to the pixel. The current atmospheric transmittance is corrected based on an empirical correction formula to obtain the corrected atmospheric transmittance. The empirical correction formula is as follows: in, The corrected atmospheric transmittance is... The current atmospheric transmittance, The cabin atmospheric pressure during the current sampling period. Standard atmospheric pressure The optical thickness of the smoke. The optical depth is a power-law scale exponent with respect to the air pressure ratio. Under normal atmospheric conditions without smoke, a preset first calibration value is taken. After it is determined that a smoke scattering and masking event has occurred in the cargo hold, the value is switched to a preset second calibration value. The second calibration value is obtained by experimental calibration of the cargo hold in a smoke-containing aerosol environment. Substitute the corrected atmospheric transmittance into the basic equation for infrared radiation thermometry, and re-execute the step of separating the corresponding target true temperature from the radiation temperature value of each pixel in the four-dimensional data field using the current atmospheric transmittance and the basic equation for infrared radiation thermometry.

5. The method according to claim 1, characterized in that, After the step of generating a temperature anomaly alarm containing the identifier of the target cargo loading unit and the three-dimensional spatial coordinates of the abnormal temperature pixel, the method further includes: Based on the current sampling period, in the updated four-dimensional data field of each sampling period within the preset time window, all pixels whose three-dimensional spatial coordinates are within the preset spatial neighborhood range of the abnormal temperature pixel are extracted, and the target real temperature of all pixels in each sampling period is arranged in chronological order to form the temperature time sequence of the abnormal temperature pixel. Perform linear regression on the temperature time series to obtain the temperature time series slope of the abnormal temperature pixel within the preset time window; If the temperature time-series slope exceeds a preset slope threshold, a heat source loading unit point set is constructed based on the three-dimensional spatial coordinates of all pixels belonging to the target cargo loading unit. Based on the three-dimensional spatial coordinates of adjacent pixels in the three-dimensional space that are within a preset contact distance of the heat source loading unit point set, a heat diffusion candidate point set is formed. The pixels in the heat diffusion candidate point set belong to other cargo loading units that are different from the target cargo loading unit to which the abnormal temperature pixel belongs. Based on the temperature time slope of each pixel in the heat diffusion candidate point set within the preset time window, and the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set, the predicted remaining time for the heat source loading unit container wall to reach the preset thermal failure temperature is calculated. The preset thermal failure temperature is the pre-set structural failure critical temperature of the cargo loading unit container wall. The predicted remaining time is appended to the temperature anomaly alarm.

6. The method according to claim 5, characterized in that, The step of calculating the predicted remaining time for the container wall of the heat source loading unit to reach the preset thermal failure temperature specifically includes: Based on the temperature time slope of each pixel in the heat diffusion candidate point set within the preset time window and the heat capacity parameters of adjacent cargo loading units, the heat absorption power of the loading unit to which the heat diffusion candidate point set belongs within the preset time window is determined. Divide the heat absorption power by the three-dimensional spatial contact area between the heat diffusion candidate point set and the heat source loading unit point set to obtain the interface heat flux density. Based on the interfacial heat flux density and the temperature time slope of the heat source loading unit itself, the heat release power inside the heat source loading unit is determined by the heat balance equation. Based on the heat release power, the difference between the current target true temperature of the heat source loading unit and the preset thermal failure temperature of the cargo loading unit container wall, and the preset thermal capacity parameters of the cargo loading unit container wall, the predicted remaining time for the heat source loading unit container wall to reach the thermal failure temperature is calculated.

7. The method according to claim 5, characterized in that, Following the step of appending the predicted remaining time to the temperature anomaly alarm, the method further includes: When it is determined that there are multiple abnormal temperature pixels belonging to different cargo loading units in the four-dimensional data field of the current sampling period, the ratio of the observable number of pixels of the three-dimensional spatial contact surface in the four-dimensional data field to the theoretical total number of pixels of the three-dimensional spatial contact surface is determined as the observable integrity coefficient of the contact interface of the corresponding cargo loading unit. The theoretical total number of pixels is calculated based on the intersection area of ​​the three-dimensional spatial envelope of the heat source loading unit point set and the three-dimensional spatial envelope of the heat diffusion candidate point set and the preset pixel spatial resolution. Divide the predicted remaining time for each cargo loading unit by the observable integrity coefficient of the contact interface to obtain the corrected priority score for that cargo loading unit. The temperature anomaly alarms corresponding to each cargo loading unit are sorted according to the correction priority score from smallest to largest.

8. A monitoring device, characterized in that, The monitoring device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the monitoring device to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on the monitoring device, the monitoring device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the monitoring device, the monitoring device performs the method as described in any one of claims 1-7.

Images