Intelligent storage environment monitoring system based on multi-modal sensor fusion

Abstract

The invention discloses an intelligent storage environment monitoring system based on multi-modal sensor fusion, and relates to the technical field of intelligent sensors, and the system comprises a reflection cone determination module which is used for generating a structure line belt mask based on a visible image of a ceiling of a warehouse, and determining the center direction of a reflection cone based on a depth map of the warehouse; the steady-state temperature setting module is used for setting three steady-state surface temperatures of the black body plate based on the structural line tape mask and the center direction of the reflection cone; the reflection confirmation module is used for generating a specular reflection controllability confirmation mark based on the three steady-state surface temperatures; and the shielding brightness module is used for generating shielding state brightness based on the structure line tape mask and the central direction of the reflection cone when the mirror reflection controllability confirmation mark is 1. According to the invention, the condensation risk can be actively identified and avoided.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensor technology, and in particular to an intelligent warehouse environment monitoring system based on multimodal sensor fusion. Background Technology

[0002] In modern large-scale warehousing scenarios, warehouses typically integrate multi-layer shelving, metal-structured ceilings, and various environmental control equipment to achieve high-density storage and efficient scheduling. However, due to the strong enclosure of the storage space and significant temperature and humidity gradients, localized condensation zones often form in the ceiling area. When humidity increases or temperature differences widen, condensation easily occurs at the metal seams and purlin surfaces of the ceiling, causing problems such as dripping, corrosion, and moisture absorption of goods. This is especially problematic for moisture-sensitive materials such as electronic components, pharmaceuticals, and chemical raw materials, where condensation can lead to batch scrapping or performance failure. With the advancement of intelligent warehousing, traditional single-point temperature and humidity monitoring methods cannot meet the spatial distribution accuracy requirements and struggle to identify microscale thermal changes and reflection behavior in local structures, making it impossible to provide early warnings of condensation risks. To achieve proactive protection of the warehousing environment, there is an urgent need for a high-precision monitoring method that can combine multi-source sensor information to perform thermal radiation analysis and risk identification at the structural line zone level, enabling the system to proactively identify potential risk areas and dynamically adjust shelving strategies before condensation forms.

[0003] Current technologies for monitoring warehouse environments primarily rely on point-based temperature and humidity sensor arrays or single infrared thermal imaging systems for status perception. While the former offers the advantage of simple deployment, its limitations in the number and spatial distribution of sensor points make it difficult to reflect temperature field changes on complex structural surfaces such as ceiling seams and purlins. The latter, although providing two-dimensional radiation information, often suffers from inaccurate temperature retrieval due to specular reflection, obstruction, and background radiation interference. Particularly in metal ceiling structures, the infrared energy received by the thermal imager includes both self-radiation and specular reflection components, leading to systematic biases in the measurements. Furthermore, existing monitoring systems generally lack multimodal data fusion and dynamic risk projection mechanisms, failing to map localized temperature and humidity risks to specific cargo location coordinates and generate executable management commands. This results in a disconnect between environmental monitoring results and the warehouse scheduling system, hindering truly automated safety management. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies in reflecting temperature field changes on complex structural surfaces such as ceiling seams and purlins, and to propose an intelligent warehouse environment monitoring system based on multimodal sensor fusion.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution: A smart warehouse environment monitoring system based on multimodal sensor fusion includes: The reflection cone determination module is used to generate a structural line mask based on the visible image of the warehouse ceiling and to determine the center direction of the reflection cone based on the depth map of the warehouse. The steady-state temperature setting module is used to set the three steady-state surface temperatures of the blackbody plate based on the structural line mask and the center direction of the reflection cone. The reflection confirmation module is used to generate a specular reflection controllability confirmation mark based on three steady-state surface temperatures. The occlusion brightness module is used to generate occlusion brightness based on the structure line mask and the center direction of the reflection cone when the specular reflection controllability confirmation mark is 1. The margin calculation module is used to calculate the emissivity difference result based on three steady-state surface temperatures, and to calculate the dew point margin based on the emissivity difference result and the shading state brightness. The instruction issuance module is used to issue instructions to the warehouse management system based on the dew point margin.

[0006] Preferably, generating a structural line mask based on a visible image of the warehouse ceiling and determining the center direction of the reflection cone based on a depth map of the warehouse includes: Images of the warehouse ceiling are captured using smart sensors; Directional filtering and connected component analysis are performed on the visible image of the ceiling to obtain the initial region with linear edges; Binary segmentation and region filtering are performed on the initial region of the linear edge to obtain the structural line band mask; Calculate surface normal vectors based on the depth map of the warehouse; The line-of-sight direction of the thermal imager is obtained by calculating the imaging pose parameters of the thermal imager. The direction of the center of the reflection cone is determined based on the line of sight of the thermal imager and the surface normal vector.

[0007] Preferably, the method for setting the three steady-state surface temperatures of the blackbody plate based on the structural line mask and the center direction of the reflection cone includes: The structural line strip mask is subjected to region screening to obtain the structural line strip region; Determine the center direction of the reflection cone of each pixel within the structural line strip area; Within the structural line zone, calculate the average direction vector in the direction of the center of the reflection cone; Install a temperature-controlled blackbody plate inside the warehouse; Alignment settings are configured for the emission surface normal vector and the average direction vector of the blackbody plate; After the alignment settings are completed, the three steady-state surface temperatures of the blackbody plate are set and denoted as the first background temperature, the second background temperature, and the third background temperature, respectively.

[0008] Preferably, a specular reflection controllability confirmation mark is generated based on three steady-state surface temperatures, including: Parallel polarization sampling was performed on the thermal image of the structural line strip region at each steady-state surface temperature to obtain a parallel polarization brightness image; At each steady-state surface temperature, the thermal image of the structural line strip region is sampled by vertical polarization to obtain a vertical polarization brightness image; Pixel-level difference operations are performed on parallel polarization brightness images and vertical polarization brightness images to obtain polarization difference values; Monotonicity verification was performed based on the temperature response relationship between steady-state surface temperature and polarization difference to obtain a confirmation mark for the controllability of specular reflection.

[0009] Preferably, when the specular reflection controllability confirmation flag is 1, the occlusion state brightness is generated based on the structural line mask and the center direction of the reflection cone, including: Based on the structure line mask and the center direction of the reflection cone, the shading path of the structure line region pointing towards the blackbody plate along the direction of the reflection cone is determined; A shielding plate is placed along the shielding path so that the specular reflection path in the structural line area is blocked by the shielding plate; While maintaining the background temperature of the blackbody plate in the first phase, the position and orientation of the occlusion plate remain unchanged, and a thermal imaging smart sensor is used to collect brightness images under the occlusion state. Pixels in the structural line band region are extracted from the brightness image under occlusion to obtain the occlusion brightness.

[0010] Preferably, the emissivity difference is calculated based on the three steady-state surface temperatures, including: The thermal imaging scene of the warehouse roof corresponding to three steady-state surface temperatures was captured, and the first frame brightness image, the second frame brightness image, and the third frame brightness image were obtained respectively. Perform a difference operation on the first frame brightness image and the second frame brightness image to obtain a brightness difference image; The difference between the fourth power of the first background temperature and the fourth power of the second background temperature is calculated to obtain the fourth power difference of the background temperature. The emissivity difference result is obtained by dividing the pixel brightness value of the brightness difference image by the product of the Stefan-Boltzmann constant and the fourth power difference of the background temperature.

[0011] Preferably, the dew point margin is calculated based on the emissivity difference result and the shading state brightness, including: The emissivity difference result is multiplied by the Stefan-Boltzmann constant and the fourth power of the steady-state surface temperature to obtain the reflection brightness compensation term; The self-radiation brightness value is obtained by calculating the difference between the pixel brightness value of the brightness image corresponding to the steady-state surface temperature and the reflection brightness compensation term. The emissivity radiation coefficient is obtained by multiplying the emissivity by the Stefan-Boltzmann constant. Divide the self-radiance value by the emissivity radiation coefficient to obtain the fourth power value of the surface temperature; The true surface temperature is obtained by taking the fourth root of the fourth power value of the surface temperature. The fourth root of the result of dividing the brightness of the shaded state by the emissivity radiative coefficient is used to obtain the surface temperature of the shaded state. The effective surface temperature is obtained by performing a minimum value selection operation on the shielded surface temperature and the true surface temperature; The dew point margin is obtained by calculating the difference between the effective surface temperature and the dew point temperature of the structural line zone.

[0012] Preferably, instructions are issued to the warehouse management system based on the dew point margin, including: Based on the dew point margin, the pixel coordinate set within the target area of ​​the structural line band is projected and transformed, and the risk area is extracted to obtain the no-view band. Instructions were issued to the warehouse management system, including prohibiting the placement of moisture-sensitive goods within the no-release zone.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention solves the technical problems of low accuracy in temperature and humidity distribution sensing and difficulty in early identification of local condensation risks in traditional warehouse environment monitoring by introducing multimodal sensor fusion technology. This invention combines visible light images, depth maps and infrared thermal imaging data to establish the structural line mask and the center direction of the reflective cone of the warehouse roof, so as to achieve accurate positioning and geometric modeling of the metal seams and purlin areas of the roof. This gives subsequent thermal radiation measurements clear physical boundaries and directional references, ensuring the spatial correspondence between temperature inversion and radiation compensation.

[0014] 2. This invention solves the problem of unavoidable interference from thermal mirror reflection in metal roof structures by using a reflection confirmation and shading brightness generation mechanism. Under steady-state background radiation conditions, a polarization difference sequence is constructed by comparing the brightness difference between parallel and perpendicular polarization. The controllability of mirror reflection is judged based on the monotonicity of temperature response. Then, the mirror reflection path is completely blocked by shading path planning and shading state brightness acquisition, thereby obtaining pure self-radiation brightness basic data. Without changing the warehouse structure, the physical elimination of reflected energy components can be achieved through controllable optical intervention. This provides high-precision, reflection-free conditions for real surface temperature recovery and dew point margin calculation, improving the stability and comparability of temperature measurement results.

[0015] 3. This invention achieves high-precision calculation of dew point margin and spatial projection of risk zones by jointly calculating the emissivity difference results and the shading state brightness. The brightness difference is normalized using the Stefan-Boltzmann law to obtain the emissivity difference results for each structural line zone region. Then, through reflection brightness compensation, self-radiation brightness recovery, and temperature inversion, the true temperature field of the ceiling is obtained. Further, the dew point margin index is obtained by differentiating it from the ambient dew point temperature. A no-deposition zone is generated through spatial projection mapping, and an execution instruction prohibiting the placement of moisture-sensitive goods on shelves is issued to the warehouse management system. This process realizes an automated closed loop from thermal radiation physical quantities to warehouse management instructions, proactively identifying and avoiding condensation risks, and improving the safety and operational efficiency of the intelligent warehousing environment. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a functional block diagram of an intelligent warehouse environment monitoring system based on multimodal sensor fusion, provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example: This example provides an intelligent warehouse environment monitoring system based on multimodal sensor fusion. See [link to example]. Figure 1 Specifically, including: The reflection cone determination module is used to generate a structural line mask based on the visible image of the warehouse ceiling and to determine the center direction of the reflection cone based on the depth map of the warehouse. In an embodiment of the present invention, generating a structural line mask based on a visible image of the warehouse ceiling and determining the center direction of the reflection cone based on a depth map of the warehouse includes: Images of the warehouse ceiling are captured using smart sensors; Directional filtering and connected component analysis are performed on the visible image of the ceiling to obtain the initial region with linear edges; Binary segmentation and region filtering are performed on the initial region of the linear edge to obtain the structural line band mask; Specifically, firstly, the visible image of the ceiling acquired by the intelligent sensor is loaded into the image processing unit. The image is then grayscaled and noise suppressed to ensure spatial continuity of the ceiling surface texture and reduce high-frequency noise. Next, multiple directional filtering templates aligned with the direction of the warehouse ceiling panel seams and purlins are selected, and convolution operations are performed pixel-by-pixel on the visible image to obtain directional filtering result images highlighting linear responses in each direction. Then, based on the similarity of pixel grayscale and the connectivity of four-neighbor or eight-neighbor domains, connectivity analysis is performed on the directional filtering result images, aggregating high-response pixels continuously distributed along the same direction into several initial linear edge regions. Subsequently, these initial linear edge regions are used as... The processing object selects an adaptive threshold based on the grayscale histogram of the entire image. Pixels with grayscale values ​​not lower than the threshold are marked as foreground pixels, and pixels with grayscale values ​​lower than the threshold are marked as background pixels, thus completing the binary segmentation of the initial region of the linear edge. Then, the obtained foreground pixels are filtered by region selection. Discrete short segments and noise patches are removed according to geometric conditions such as length, width, aspect ratio, and whether the direction is consistent with the arrangement direction of the ceiling components. Only linear strip regions that meet the continuity and direction constraints are retained. The pixel positions corresponding to these linear strip regions are uniformly marked as valid regions, and the remaining pixel positions are marked as invalid regions. This generates a structural line strip mask for subsequent calculations.

[0019] Specifically, the visible ceiling image refers to the two-dimensional optical image data obtained by intelligent sensors imaging the warehouse roof structure. This image presents the spatial texture, brightness differences, and boundaries of structural elements such as seams and purlins on the ceiling surface in the form of visible light intensity distribution. Directional filtering refers to applying a direction-selective filtering operator to the visible ceiling image. By enhancing brightness variations along specific directions, the response of linear structures is highlighted, thus facilitating the subsequent identification of metal seams and purlins with stable directional distribution in the ceiling. Connected component analysis refers to the process of adjusting the brightness based on the adjacency relationship between pixels in the directional filtering result. Continuously distributed regions are merged to identify initial linear regions that are morphologically continuous and spatially integrated. Binary segmentation involves dividing the regions obtained from connected component analysis into foreground and background categories based on brightness differences, thus distinguishing potential linear structures from complex backgrounds. Region filtering involves filtering the candidate regions after binary segmentation based on geometric indicators such as length, width, and shape, thereby retaining linear strip regions consistent with the ceiling metal components, and finally forming a structural line strip mask. This mask is used to limit the computational range in subsequent steps, ensuring that subsequent processing is only performed on the actual locations of the structural line strips.

[0020] Calculate surface normal vectors based on the depth map of the warehouse; Specifically, after acquiring a depth map of the warehouse ceiling area using a depth intelligent sensor, the processing unit first establishes a mapping relationship between the pixel coordinates of the depth map and the three-dimensional spatial coordinates. The row and column positions of each pixel in the depth map and their corresponding depth values ​​are converted into a set of three-dimensional point coordinates relative to the sensor coordinate system. Then, within the area defined by the structural line mask, several adjacent pixels in the horizontal and vertical directions are selected for each target pixel to form a local neighborhood. Based on the three-dimensional point coordinates corresponding to these adjacent pixels, two sets of non-collinear direction vectors are calculated. For example, one set of direction vectors connects adjacent points along the row direction, and the other set connects adjacent points along the column direction. The normal directions of these two sets of direction vectors are then obtained through vector cross product, and the normal directions are normalized to a length of one, thus obtaining the unit surface normal vector of the ceiling surface at the target pixel. The above three-dimensional coordinate reconstruction and normal vector calculation process is sequentially performed on all target pixels within the entire structural line area, ultimately forming a surface normal vector field with the same size as the depth map. This surface normal vector field is used to subsequently determine the center direction of the reflection cone by combining it with the line of sight of the thermal imager.

[0021] The line-of-sight direction of the thermal imager is obtained by calculating the imaging pose parameters of the thermal imager. The direction of the center of the reflection cone is determined based on the line of sight of the thermal imager and the surface normal vector.

[0022] Specifically, the imaging pose parameters obtained from the thermal imager calibration are imported into the processing unit. The spatial position of the thermal imager's optical center in the warehouse coordinate system and the initial direction of the imaging optical axis are calculated using the three-dimensional position coordinates and attitude rotation matrix from the pose parameters. This establishes a geometric mapping relationship from pixel coordinates to the viewing direction. Then, given the reconstructed three-dimensional point cloud of the ceiling and surface normal vectors, for each target pixel defined by the structural line mask, a direction vector from the thermal imager's optical center to that three-dimensional coordinate point is calculated based on the corresponding surface three-dimensional coordinates of that pixel. This direction vector is then normalized, and the normalized direction vector is recorded as the direction vector at that pixel. The thermal imager's line of sight is then used to read the surface normal vector at the same pixel. The line of sight is then symmetrically transformed around the surface normal vector using specular reflection geometry. This involves first calculating the components of the line of sight along the normal vector and the tangential component, then symmetrically mapping the tangential component along the normal vector to obtain a new direction vector. This new direction vector is then normalized again, and the normalized direction vector is recorded as the center direction of the reflection cone at that pixel. This process of obtaining the line of sight and performing symmetrical transformation is performed sequentially on all pixels covered by the structural line mask. Finally, a reflection cone center direction field with the same size as the ceiling image is formed for subsequent blackbody plate placement and occlusion path planning.

[0023] Specifically, the depth map refers to the two-dimensional matrix data formed by a depth intelligent sensor after actively or passively measuring the distance to the warehouse ceiling area. Each pixel corresponds to a distance value from the sensor to the ceiling surface, thus reflecting the spatial concavity and geometric shape of the ceiling surface. The surface normal vector is a unit direction quantity calculated based on the spatial coordinate difference between adjacent pixels in the depth map. This direction quantity is perpendicular to the micro-element of the ceiling surface and is used to describe the orientation of the surface at that location. The imaging pose parameters of the thermal imager refer to the set of parameters used to describe the spatial installation position and orientation of the thermal imager inside the warehouse, including its position in three-dimensional space. The coordinates and the direction information of the imaging optical axis are used to characterize the positional relationship of the thermal imager's field of view relative to the ceiling; the line of sight direction refers to the unit vector obtained by calculating the direction of the imaging optical axis in three-dimensional space based on the thermal imager's imaging pose parameters, which is used to represent the main viewing direction when the thermal imager observes any pixel point on the ceiling; the center direction of the reflection cone refers to the symmetrical direction of the specular reflection path determined by the surface normal vector and the thermal imager's line of sight, which represents the main energy reflection direction when the ceiling surface forms a specular reflection on the thermal imager under geometric optics conditions. This direction is subsequently used to identify and control the source of reflected energy at the metal seams.

[0024] The steady-state temperature setting module is used to set the three steady-state surface temperatures of the blackbody plate based on the structural line mask and the center direction of the reflection cone. In embodiments of the present invention, the method for setting three steady-state surface temperatures of the blackbody plate based on the structural line mask and the center direction of the reflection cone includes: The structural line strip mask is subjected to region screening to obtain the structural line strip region; Determine the center direction of the reflection cone of each pixel within the structural line strip area; Specifically, the structural line strip mask obtained in the previous steps is loaded into the processing unit. The valid locations marked in the mask are then filtered according to connectivity and geometric constraints. First, all continuous strip-shaped connected regions are identified based on the adjacency relationships between pixels. Then, each connected region is checked item by item according to indicators such as length, width, linearity, and direction consistency. Scattered short segments caused by noise or local reflections, and abnormal segments that do not meet the linear morphology requirements, are eliminated. Finally, valid structural segments that meet the requirements of continuity, direction consistency, and length are retained. The set of pixel positions of all valid segments is defined as the structural line strip region. Subsequently, the structural line strip... For each pixel within the region, the surface normal vector obtained in the depth reconstruction step is read, and combined with the thermal imager's line of sight obtained in the thermal imager pose acquisition step, the reflection cone center direction of the pixel is calculated sequentially according to the specular reflection direction acquisition method. That is, the specular reflection principal direction of the pixel is generated by performing a symmetrical transformation along the surface normal vector of the line of sight, and the obtained direction vector is normalized and used as the reflection cone center direction corresponding to the pixel. This direction acquisition process is continuously performed on all pixels in the entire structural line area to form a set of reflection cone center directions for subsequent background radiation control and occlusion path determination.

[0025] Within the structural line zone, calculate the average direction vector in the direction of the center of the reflection cone; Specifically, the reflection cone center direction, already obtained for each pixel within the structural line strip region, is loaded into the direction processing unit. First, the positions of all pixels within the structural line strip region are traversed, and the corresponding reflection cone center direction vector is read one by one. These unit direction vectors are then accumulated by vector addition to form an unnormalized direction accumulation vector, which comprehensively reflects the overall specular reflection pointing trend of the structural line strip. Subsequently, the direction accumulation vector is normalized by dividing the three-dimensional components of the accumulation vector by its magnitude, so that its magnitude is normalized to one, thus obtaining a unique unit direction quantity. This unit direction quantity is used as the average direction vector of the structural line strip region. This average direction vector is used to represent the common reflection pointing of the entire structural line strip region in specular reflection geometry, providing a unified reflection reference direction for subsequent blackbody plate radiation surface alignment and occlusion path planning.

[0026] Specifically, the structural line strip mask refers to the binary marked region formed after directional filtering, connected component analysis, binary segmentation, and region filtering of the visible light image. This marked region, in pixels, indicates the location of the linear strip-shaped component on the ceiling surface composed of metal seams and purlins, and is used to limit the calculation range of subsequent reflection direction and temperature inversion. The structural line strip region refers to the set of all pixels marked as valid in the structural line strip mask. These pixels correspond to the actual physical location on the surface of the ceiling component, and their internal optical reflection behavior and thermal characteristics are different from those of the surrounding ordinary ceiling panels. The direction of the reflection cone center refers to... For each pixel within the structure line area, a unit vector is obtained by calculating the symmetrical direction of the specular reflection path based on the pixel's surface normal vector and the thermal imager's line of sight. This direction represents the energy center vector when thermal radiation energy is emitted outward along the specular reflection path. The average direction vector of the reflection cone center direction refers to the unified direction quantity obtained by summing and normalizing the vectors of the reflection cone center directions corresponding to all pixels within the structure line area. This is used to represent the overall specular reflection pointing trend of the structure line area and to provide a unified reflection geometric reference direction for subsequent blackbody plate arrangement, occlusion path planning, and background light control.

[0027] Install a temperature-controlled blackbody plate inside the warehouse; Specifically, after the controllable temperature blackbody plate is removed from its packaging, it is placed at the installation station. The target direction that the blackbody plate needs to face inside the warehouse is determined by reading the average direction vector of the structural line area. Then, an installation point that can be connected to the center direction of the reflection cone is selected in an open space in the warehouse operation aisle or between shelves. The mounting bracket of the blackbody plate is fixed at the installation point, and the controllable radiation surface of the blackbody plate is oriented towards the structural line area through the pitch adjustment mechanism and azimuth adjustment mechanism of the bracket. This ensures that the emission surface of the blackbody plate is directly located on the specular reflection path of the structural line area, ensuring that the radiation energy emitted by it can propagate along the center direction of the reflection cone to the metal joint of the ceiling, thus establishing a stable energy input foundation for subsequent multi-mode background radiation replacement.

[0028] Alignment settings are configured for the emission surface normal vector and the average direction vector of the blackbody plate; Specifically, after the blackbody plate is mechanically installed, the attitude adjustment control unit is activated. First, the control unit reads the normal vector of the blackbody plate's emitting surface. This normal vector represents the unit direction of the controllable radiating surface of the blackbody plate in three-dimensional space. Simultaneously, the average direction vector of the structural line zone region is obtained from the step of obtaining the direction from the center of the reflection cone. This average direction vector is the unified direction of the overall specular reflection path of the structural line zone region. Then, the normal vector of the blackbody plate's emitting surface and the average direction vector are input into the attitude comparison module. The difference is determined by calculating the angle between the two direction vectors. The azimuth and pitch angles of the blackbody plate are gradually adjusted using the rotation joint of the mounting bracket, so that the normal vector of the emitting surface and the average direction vector gradually become consistent until the angle between them is reduced to a negligible level. This completes the alignment of the blackbody plate's radiating surface, allowing the radiated energy output by the controllable radiating surface of the blackbody plate to propagate stably along the specular reflection path of the structural line zone region.

[0029] After the alignment settings are completed, the three steady-state surface temperatures of the blackbody plate are set and denoted as the first background temperature, the second background temperature, and the third background temperature, respectively.

[0030] Specifically, after the emitting surface of the blackbody plate is aligned, the internal temperature control module of the blackbody plate is activated. By reading the data from the internal temperature sensor of the blackbody plate, it is confirmed that the current surface temperature is within the adjustable range. Then, according to the radiation difference required for the three-state background replacement, three different target surface temperatures are sequentially set in the control unit. The first target temperature is set as the first-stage background temperature, the second target temperature is set as the second-stage background temperature, and the third target temperature is set as the third-stage background temperature. Subsequently, the heating or cooling unit of the temperature control module is activated to automatically adjust the surface temperature of the blackbody plate to the first-stage background temperature and keep it stable. After collecting the brightness data at the first temperature state, the surface temperature of the blackbody plate is adjusted to the second-stage background temperature and kept stable to complete the second-stage acquisition. Finally, the surface temperature of the blackbody plate is adjusted to the third-stage background temperature and kept stable to complete the third-stage acquisition. This forms three stable background temperature datasets for subsequent brightness difference, emissivity calculation, and reflection reversal identification.

[0031] Specifically, a temperature-controlled blackbody plate refers to a radiation reference surface whose surface temperature can be adjusted and maintained at a constant temperature through internal heating or cooling devices. When operating stably, this reference surface emits thermal radiation outward in a manner close to that of an ideal blackbody, with its emission intensity determined solely by its own temperature. It is used to provide a controllable radiation background for the specular reflection path. The emission surface normal vector of the blackbody plate refers to the unit normal direction of the controllable radiation surface of the blackbody plate. This direction is perpendicular to the actual physical plane of the blackbody plate surface and represents the main radiation direction when radiant energy is emitted outward from the surface of the blackbody plate. The average direction vector refers to the unified direction obtained by summing and normalizing the directions of the center directions of all reflection cones within the structural line zone area. This direction represents the overall pointing trend of specular reflection from the metal seams and purlins in the ceiling, and is used to indicate... The radiating surface of the blackbody plate is aligned. Alignment setting refers to adjusting the installation posture of the blackbody plate by rotating it after installation, so that the normal vector of the blackbody plate's radiating surface is consistent with the average direction vector in the spatial direction, so that the radiated energy output by the blackbody plate can enter the mirror reflection path of the metal structure along the center direction of the reflection cone. Steady-state surface temperature refers to the three different constant surface temperatures that the blackbody plate maintains stably by a temperature control device after all alignment settings are completed. Each temperature state corresponds to a controllable background radiation intensity in the radiation law. These three steady-state surface temperatures are used as the first-stage background temperature, the second-stage background temperature, and the third-stage background temperature, respectively, to construct multi-state brightness difference in subsequent steps, thereby completing the quantitative identification of the low emissivity reflection reversal phenomenon in the metal strip region.

[0032] The reflection confirmation module is used to generate a specular reflection controllability confirmation mark based on three steady-state surface temperatures. In an embodiment of the present invention, a specular reflection controllability confirmation mark is generated based on three steady-state surface temperatures, including: Parallel polarization sampling was performed on the thermal image of the structural line strip region at each steady-state surface temperature to obtain a parallel polarization brightness image; Specifically, after the blackbody plate is set to a certain steady-state surface temperature, a parallel polarizer is installed at the front end of the imaging optical path of the thermal imager. This allows only the horizontally polarized radiation component to enter the sensor imaging surface. Then, the thermal image acquisition unit is activated, and the structural line area is used as the sampling target area. Each pixel in the structural line area is continuously measured within the thermal imager's field of view, so that the infrared radiation intensity after polarization screening forms a corresponding brightness value at the pixel level. This brightness value is recorded as parallel polarization brightness using internal calibration parameters. Subsequently, the parallel polarization brightness of all target pixels defined by the structural line mask is arranged according to their spatial positions to form a brightness matrix consistent with the thermal image resolution. This brightness matrix is ​​recorded as a parallel polarization brightness image, which reflects the response of the structural line area to the parallel polarized radiation component under the current steady-state background radiation conditions.

[0033] At each steady-state surface temperature, the thermal image of the structural line strip region is sampled by vertical polarization to obtain a vertical polarization brightness image; Specifically, under the condition that the steady-state surface temperature of the blackbody plate remains constant, the transmission direction of the polarizer is rotated by 90 degrees so that only the radiation component of vertically polarized vibration is allowed to pass through the imaging optical path of the thermal imager. Thermal images of the structural line area are acquired again. The thermal imager measures the vertical polarized radiation intensity of each pixel covered by the structural line mask and records the vertical polarized radiation intensity at each pixel as vertical polarization brightness. Then, all vertical polarization brightness values ​​are assembled into a brightness matrix according to the pixel position order to construct a vertical polarization brightness image. This brightness image reflects the specific response of the structural line area to the vertical polarization radiation component under the same steady-state background temperature, thus providing a complete data foundation for subsequent construction of polarization difference and polarization consistency verification.

[0034] Specifically, the parallel polarization brightness image refers to the brightness distribution map obtained by sampling the thermal image scene of the structural line strip area when the blackbody plate is at a certain steady-state surface temperature. This is achieved by adding a polarizer to the thermal imaging acquisition optical path, which causes the infrared radiation to vibrate only in the horizontal direction. This brightness distribution reflects the radiation response intensity of the ceiling metal seam and purlin surface to the parallel polarization component. The vertical polarization brightness image refers to the brightness distribution map obtained by rotating the transmission direction of the polarizer by 90 degrees at the same steady-state surface temperature, causing the infrared radiation to vibrate only in the vertical direction, and then sampling the structural line strip area again. This brightness distribution reflects the radiation response of the structural line strip area surface to the vertical polarization component. The steady-state surface temperature refers to the constant surface temperature state of the temperature-controlled blackbody plate during the temperature control process. At this temperature, the radiation energy emitted by the blackbody plate remains stable, allowing the parallel polarization sampling and vertical polarization sampling to form comparable brightness data at different temperature levels. This lays a stable radiation background foundation for subsequent construction of polarization difference, suppression of mirror flip, and establishment of emissivity difference.

[0035] Pixel-level difference operations are performed on parallel polarization brightness images and vertical polarization brightness images to obtain polarization difference values; Specifically, parallel polarization brightness images and vertical polarization brightness images acquired at the same steady-state surface temperature are loaded into the polarization processing unit. Taking the effective pixels defined by the structural line mask as the processing object, the parallel polarization brightness value and vertical polarization brightness value are read sequentially for each pixel position. A pixel-by-pixel subtraction operation is performed according to the brightness order. The polarization difference value of the pixel is obtained by subtracting the vertical polarization brightness value from the parallel polarization brightness value. The polarization difference values ​​of all pixels are arranged into a polarization difference matrix according to their original spatial positions. This matrix reflects the brightness difference response of the structural line region to the parallel polarization and vertical polarization radiation components under the current steady-state background radiation conditions, thereby forming the polarization difference value used for subsequent temperature response verification.

[0036] Monotonicity verification was performed based on the temperature response relationship between steady-state surface temperature and polarization difference to obtain a confirmation mark for the controllability of specular reflection.

[0037] Specifically, after obtaining the polarization difference values ​​corresponding to the three steady-state surface temperatures, the three polarization difference values ​​are input into the temperature response analysis unit according to the set order of the steady-state temperatures. For each pixel in the structural line band region, the polarization difference value corresponding to the pixel is read sequentially under the three steady-state temperature conditions to construct a numerical sequence of polarization difference values ​​changing with steady-state temperature. Then, it is determined whether the numerical sequence maintains a monotonic change trend of continuous increase or continuous decrease under the three steady-state temperatures. When the polarization difference value of the pixel meets the monotonic change condition in the entire temperature sequence, the monotonicity judgment result of the pixel is recorded as valid, and the proportion of valid judgment results in the entire structural line band region is counted. When the proportion exceeds the preset judgment requirement, the specular reflection controllability confirmation flag is set to one, indicating that the specular reflection path is mainly dominated by the radiation energy of the temperature-controlled blackbody plate, and subsequent occlusion state acquisition and emissivity calculation can be performed in a controllable reflection environment. If the statistical result does not meet the monotonicity requirement, the specular reflection controllability confirmation flag is set to zero to indicate that the current specular reflection has not yet entered a controllable state.

[0038] Specifically, the polarization difference refers to the difference obtained by subtracting the brightness of a parallel polarized brightness image from that of a vertically polarized brightness image at pixel positions under the same steady-state surface temperature. This difference reflects the different responses of the surface of the structural line strip region to radiation from different polarization directions. It can highlight the specular reflection components of metal seams and purlin surfaces and suppress background noise, thus providing a quantitative basis for further determining whether specular reflection is under control. The steady-state surface temperature refers to the constant surface temperature of the temperature-controlled blackbody plate after adjustment by the temperature control module. At this temperature, the radiation intensity output by the blackbody plate is stable, ensuring the comparability of brightness data collected by different polarization methods. The temperature response relationship refers to... Within the structural line zone, the polarization difference value changes with the steady-state surface temperature of the blackbody plate. By calculating the corresponding polarization difference value at three steady-state surface temperatures and analyzing the direction of its change with temperature, it is possible to verify whether the polarization difference value exhibits monotonic variation characteristics, thereby determining whether the specular reflection path is completely dominated by the temperature-controlled blackbody plate. The specular reflection controllability confirmation mark refers to the identifier obtained after verification based on the monotonic relationship between steady-state surface temperature and polarization difference value. When the polarization difference value exhibits a stable monotonic change with steady-state temperature, this identifier value is set to one, indicating that specular reflection has entered a controllable state, which can be used for subsequent occlusion state acquisition, emissivity calculation, and true temperature inversion.

[0039] The occlusion brightness module is used to generate occlusion brightness based on the structure line mask and the center direction of the reflection cone when the specular reflection controllability confirmation mark is 1. In an embodiment of the present invention, when the specular reflection controllability confirmation flag is 1, the occlusion state brightness is generated based on the structural line mask and the center direction of the reflection cone, including: Based on the structure line mask and the center direction of the reflection cone, the shading path of the structure line region pointing towards the blackbody plate along the direction of the reflection cone is determined; Specifically, after obtaining the structural line mask and the center direction of the reflection cone of each pixel within the structural line mask area, the structural line mask is used as a spatial constraint. The center direction vector of the reflection cone is read pixel by pixel within the mask area, and the starting point of each direction vector is used as the starting point of the direction line. This direction line is extended along the center direction of the reflection cone to the pre-set temperature-controlled blackbody plate mounting plane in the three-dimensional space of the warehouse, and the intersection point of the direction line and the plane where the blackbody plate is located is recorded. By connecting the starting point of each pixel with the corresponding intersection point, multiple reflection paths from the structural line mask area to the blackbody plate are formed. Then, spatial geometric analysis is performed on all paths, and the shortest connected path that can stably cover the structural line mask area and can be physically blocked by the shielding plate is selected. This path is defined as the shielding path from the center direction of the reflection cone to the blackbody plate, which is used to guide the placement of the shielding plate so that the mirror reflection path is accurately cut off in the subsequent shielding state acquisition.

[0040] A shielding plate is placed along the shielding path so that the specular reflection path in the structural line area is blocked by the shielding plate; Specifically, after determining the spatial location of the obstruction path, the obstruction path is imported into the obstruction plate arrangement unit. An installation baseline for the obstruction plate is generated based on the spatial location and orientation of the obstruction path within the warehouse. The obstruction plate is then positioned along this baseline, with its obstruction surface facing the structural line area and perpendicular to the center of the reflection cone. This ensures the obstruction plate forms a completely covering blocking interface along the path. After positioning, the translational position and rotation angle of the obstruction plate are fixed, maintaining a stable geometric posture along the obstruction path. This prevents the reflected radiation energy from propagating along the specular path towards the blackbody plate from the structural line area. In the obstructed state, the structural line area only receives diffuse reflection and its own radiation contribution, providing a reliable, interference-free condition for subsequent obstructed brightness sampling and specular reflection compensation.

[0041] Specifically, the structural line strip mask refers to the binary labeled region obtained after performing directional filtering, connected component analysis, binary segmentation, and region filtering on the visible ceiling image. This region is used to clearly define the exact spatial location of the linear strip-shaped component formed by the ceiling metal seams and purlins. The center direction of the reflection cone refers to the unit direction quantity obtained by using the surface normal vector of each pixel within the structural line strip region and the thermal imager's line of sight through specular reflection geometry. This direction quantity represents the center direction of the energy emitted outward along the specular reflection path after the ceiling metal component receives radiation from the blackbody plate. The occlusion path refers to the path between the known structural line strip mask and the center direction of the reflection cone. Under the premise of extending the direction vector of the reflection cone center in the structural line area outward along its pointing direction and establishing a connection with the three-dimensional spatial position of the blackbody plate, the resulting path line represents the energy propagation path of the metal seam or purlin in specular reflection state. The shielding plate refers to the physical blocking component installed in the line space. This component has fixed geometric dimensions and shielding capabilities. When the shielding plate is placed on the shielding path, its physical structure blocks the reflection energy from continuing to propagate along the specular path, so that the specular reflection in the structural line area in the shielding state is completely cut off, thereby providing a reference condition without specular reflection interference for subsequent shielding state brightness sampling.

[0042] While maintaining the background temperature of the blackbody plate in the first phase, the position and orientation of the occlusion plate remain unchanged, and a thermal imaging smart sensor is used to collect brightness images under the occlusion state. Specifically, the temperature-controlled blackbody plate is switched to the first-stage background temperature, and after the temperature control unit outputs a stable output, the surface temperature field of the blackbody plate is confirmed to be uniform. At the same time, the position and attitude of the shielding plate that has been installed in the previous step are locked, so that the shielding plate maintains a stable geometric state between the structure line area and the temperature-controlled blackbody plate without any translation or rotation. Subsequently, the blackbody plate temperature and shielding plate attitude are continuously maintained throughout the entire shielding acquisition process, so that the shielding plate remains on the shielding path along the direction of the reflection cone center, thereby ensuring that the specular reflection path in the structure line area is always completely blocked, providing a fixed physical environment for the next step of acquiring brightness images free from specular reflection interference.

[0043] Specifically, after the shielding plate remains in a stable blocking state, the thermal imaging smart sensor is aimed at the structural line area, and the field of view is locked according to its imaging pose parameters, so that the sensor can receive the radiation energy of the structural line area under the first background temperature conditions; the sensor converts the actual radiation intensity in this area into pixel-by-pixel brightness values ​​in the form of infrared imaging, and generates a shading state brightness image in the form of a two-dimensional matrix, so that the brightness image fully reflects the brightness distribution of the structural line area composed only of its own radiation and diffuse reflection components without specular reflection contribution, providing reference brightness data for subsequent specular reflection compensation calculation.

[0044] Pixels in the structural line band region are extracted from the brightness image under occlusion to obtain the brightness under occlusion.

[0045] Specifically, after obtaining the occluded state brightness image, the structure line mask is loaded into the pixel extraction unit. The pixel positions marked as structure line regions in the mask are mapped one by one onto the occluded state brightness image. Then, the brightness values ​​of these corresponding positions are read point by point. All the brightness values ​​selected by the mask are recombined according to their original spatial positions to form the occluded state brightness matrix of the structure line region. This occluded state brightness represents the basis of the true brightness of the structure line region after specular reflection is completely blocked. It is used for subsequent differential derivation with the unoccluded brightness, thereby separating the specular reflection contribution and realizing reflection compensation.

[0046] Specifically, the first-phase background temperature refers to the initial fixed surface temperature value of the temperature-controlled blackbody plate after steady-state heating. This temperature value directly corresponds to the blackbody plate's radiation capability and is used to provide a unified and repeatable radiation reference in occluded sampling. The position and orientation of the occluder plate refer to its translational position and spatial orientation in three-dimensional space. This position and orientation remain constant during the occlusion process, ensuring that the occluder plate forms a continuous physical blocking path between the structural line area and the blackbody plate, thereby stably isolating the incident specular reflection component. The thermal imaging intelligent sensor refers to a sensor with infrared imaging capabilities that can convert scene radiation energy into brightness signals. The device, after the shielding plate is stably in place, is used to collect the brightness distribution of the structural line area under the influence of no specular reflection. The shielded brightness image refers to the two-dimensional brightness matrix obtained by the thermal imaging smart sensor under shielded conditions. This matrix reflects the true brightness distribution of the structural line area when it only receives diffuse reflection radiation components. Pixel extraction of the structural line area refers to reading the corresponding brightness value inside the structural line area pixel by pixel from the shielded brightness image using the structural line mask as a sampling constraint. This constitutes the shielded brightness, which is used for subsequent differential calculation with the unshielded brightness to complete specular reflection compensation and separation of true radiation.

[0047] The margin calculation module is used to calculate the emissivity difference result based on three steady-state surface temperatures, and to calculate the dew point margin based on the emissivity difference result and the shading state brightness. In an embodiment of the present invention, the emissivity difference is calculated based on three steady-state surface temperatures, including: The thermal imaging scene of the warehouse roof corresponding to three steady-state surface temperatures was captured, and the first frame brightness image, the second frame brightness image, and the third frame brightness image were obtained respectively. Perform a difference operation on the first frame brightness image and the second frame brightness image to obtain a brightness difference image; Specifically, the three steady-state surface temperatures are three fixed temperature values ​​formed by the temperature-controlled blackbody plate during stable heating. Each temperature value corresponds to different intensities of infrared radiation energy radiated by the blackbody plate to the warehouse ceiling, used to excite the brightness response of the structural line strip region under different incident radiation conditions. The warehouse ceiling thermal image scene is a two-dimensional radiation brightness distribution obtained after the thermal imaging intelligent sensor images the infrared radiation energy received by the ceiling region at the observation position. This scene reflects the effective radiation emitted by each position of the ceiling to the sensor. The first frame brightness image is the brightness matrix collected and converted by the thermal imaging sensor when the blackbody plate maintains the first steady-state surface temperature. The second and third frame brightness images correspond to the brightness matrices at the second and third steady-state surface temperatures, respectively. Together, they constitute the brightness sequence of the ceiling under different incident radiation conditions. The brightness difference image is a brightness difference matrix generated by performing pixel-by-pixel subtraction on the second frame brightness image based on the first frame brightness image. By reflecting the brightness difference at the same spatial pixel position under two different steady-state surface temperatures, it is used to reveal the brightness variation law of the structural line strip region under different radiation conditions, providing basic data for subsequent emissivity calculation and reflection compensation.

[0048] The difference between the fourth power of the first background temperature and the fourth power of the second background temperature is calculated to obtain the fourth power difference of the background temperature. The emissivity difference result is obtained by dividing the pixel brightness value of the brightness difference image by the product of the Stefan-Boltzmann constant and the fourth power difference of the background temperature.

[0049] Specifically, the fourth power of the first and second phase background temperatures represent the baseline values ​​of radiant energy corresponding to the blackbody plate at two different steady-state surface temperatures. Since thermal radiation power has a monotonic relationship with the fourth power of absolute temperature, the fourth power of temperature can directly reflect the amplitude difference of thermal radiation intensity. The fourth power difference of background temperature is the result of subtracting the fourth power of the second phase background temperature from the fourth power of the first phase background temperature. This value corresponds to the quantitative difference in radiant power change between the two steady-state temperature states and is used as the energy scale when normalizing the brightness difference. The brightness difference image is formed by performing pixel-by-pixel difference between the first and second frame brightness images. The brightness difference matrix is ​​formed, where the pixel brightness difference reflects the brightness change of the same location under two background radiation conditions. The Stefan-Boltzmann constant is a constant value that describes the proportional relationship between the radiant power per unit area of ​​an ideal blackbody and the fourth power of its temperature. This constant can be used to convert the fourth power difference of thermal radiation temperature into a radiant power difference. The emissivity difference result is a dimensionless result obtained by dividing the brightness difference of each pixel in the brightness difference image by the product of the Stefan-Boltzmann constant and the fourth power difference of the background temperature. This result is used to characterize the difference in the proportion of radiation contribution of the corresponding pixels in the structural line band region under different background temperatures, and is an important intermediate quantity for subsequent calculation of the true emissivity.

[0050] Specifically, the above calculation process derives the quantitative relationship between an object's radiance and surface emissivity based on the law of thermal radiation. According to the Stefan-Boltzmann law, the radiant power of an object at a given temperature is proportional to the fourth power of the absolute temperature, and the pixel brightness value in the brightness image is precisely the sensor's response to this radiant power. When the blackbody plate switches from the first background temperature to the second background temperature, the radiant energy incident on the structural line band region changes measurably with the increase or decrease of the background temperature. Therefore, the brightness difference at the corresponding location can reflect the magnitude of the change in radiant power under the two temperature states. To eliminate the influence of temperature itself, the brightness difference is divided by the product of the Stefan-Boltzmann constant and the fourth power difference between the two background temperatures. This normalizes the brightness change to the energy ratio difference caused by the difference in surface emissivity, thus the resulting dimensionless quantity completely corresponds to the change in the effective emissivity ratio of the pixels in the structural line band region under the two temperature states. Therefore, the calculation result can directly represent the emissivity difference, which is the basic parameter for subsequent calculation of the true emissivity and specular reflection compensation.

[0051] In an embodiment of the present invention, the dew point margin is calculated based on the emissivity difference result and the occlusion state brightness, including: The emissivity difference result is multiplied by the Stefan-Boltzmann constant and the fourth power of the steady-state surface temperature to obtain the reflection brightness compensation term; The self-radiation brightness value is obtained by calculating the difference between the pixel brightness value of the brightness image corresponding to the steady-state surface temperature and the reflection brightness compensation term. The emissivity radiation coefficient is obtained by multiplying the emissivity by the Stefan-Boltzmann constant. Specifically, after obtaining the emissivity difference result, the emissivity difference result is first multiplied by the Stefan-Boltzmann constant and the fourth power of the steady-state surface temperature. This product is used as a reflection brightness compensation term, which quantifies the additional radiance superimposed on the pixels of the structural line band region by the specular reflection path under the current steady-state background temperature conditions. Subsequently, the brightness value of each pixel in the brightness image corresponding to the steady-state surface temperature is calculated pixel-by-pixel with the reflection brightness compensation term to offset the contribution of specular reflection. This ensures that the self-radiation brightness value obtained by the difference only contains the true brightness components of the structural line band region's own radiation and diffuse reflection radiation. Next, the emissivity is multiplied by the Stefan-Boltzmann constant to obtain the emissivity radiation coefficient. This radiation coefficient constructs the energy mapping relationship between the self-radiation brightness value and the fourth power of the temperature. Through the continuous execution of the above three steps, the system realizes a complete link from brightness difference to reflection compensation to self-radiation brightness recovery and energy coefficient construction, making the subsequent calculation of the fourth power of the surface temperature an executable physical operation. This lays a continuous and interpretable computational foundation for the final calculation of the true surface temperature.

[0052] Specifically, the fourth power of the steady-state surface temperature is the energy scale obtained by powering the absolute temperature of the controllable temperature blackbody plate under steady-state conditions, and its magnitude determines the order of magnitude of the radiated power. The reflection brightness compensation term is the quantity obtained by multiplying the emissivity difference result with the Stefan-Boltzmann constant and the fourth power of the steady-state surface temperature. This quantity reflects the additional brightness of the structural line band region superimposed by the reflection channel when specular reflection is present, and it needs to be subtracted in subsequent processing. The self-radiation brightness value is the brightness obtained by subtracting the reflection brightness compensation term from the pixel brightness value of the brightness image corresponding to the steady-state surface temperature. It reflects the true brightness of the structural line band region under the current background temperature, which is contributed only by its own emission and diffuse reflection. The emissivity radiation coefficient is the quantity obtained by multiplying the emissivity with the Stefan-Boltzmann constant. This coefficient establishes the energy mapping relationship between the surface temperature of the structural line band region and the self-radiation brightness, providing a basic parameter for subsequent calculation of the true surface temperature.

[0053] Divide the self-radiance value by the emissivity radiation coefficient to obtain the fourth power value of the surface temperature; The true surface temperature is obtained by taking the fourth root of the fourth power value of the surface temperature. The fourth root of the result of dividing the brightness of the shaded state by the emissivity radiative coefficient is used to obtain the surface temperature of the shaded state. The effective surface temperature is obtained by performing a minimum value selection operation on the shielded surface temperature and the true surface temperature; The dew point margin is obtained by calculating the difference between the effective surface temperature and the dew point temperature of the structural line zone.

[0054] Specifically, after obtaining the self-radiance value and emissivity coefficient, the self-radiance value is first divided pixel by the emissivity coefficient, thus converting the radiance into a fourth-power temperature value. This allows the fourth-power surface temperature value to characterize the true radiation intensity of the structural line band region in a manner consistent with the law of thermal radiation. Then, the fourth root of the fourth-power surface temperature value is calculated to obtain the true surface temperature, restoring it to an absolute temperature value with practical physical meaning. Next, the shading state radiance is divided by the emissivity coefficient to obtain the fourth-power surface temperature of the shading state, and the fourth root calculation is performed similarly. This process ensures that the surface temperature in the shading state accurately reflects the temperature state after the specular reflection is blocked by the shading path. After obtaining the true surface temperature and the surface temperature in the shading state, the system selects the minimum value of each pixel in the two temperature fields to eliminate abnormal temperatures caused by reflection compensation errors or local noise, thereby forming a stable and reliable effective surface temperature. Finally, the difference between the effective surface temperature and the dew point temperature of the structural line area is calculated to obtain the dew point margin, which can be used as an indicator to determine whether there is a risk of condensation in the current area. This realizes a complete chain calculation logic from brightness inversion to temperature and humidity risk determination.

[0055] Specifically, the fourth-power surface temperature is obtained by dividing the self-radiative brightness of the structural line zone region by the emissivity coefficient. This value corresponds to the relationship between the fourth power of temperature and radiant energy in the law of thermal radiation, and is an intermediate physical variable describing the surface radiation intensity. The true surface temperature is the absolute temperature obtained by taking the fourth root of the fourth-power surface temperature value, and is used to characterize the true temperature state formed by the self-radiative energy of the structural line zone region after deducting the effect of specular reflection. The shading state surface temperature is the temperature value obtained by dividing the shading state brightness by the emissivity coefficient and then taking the fourth root. This temperature reflects the actual temperature distribution after the shading path blocks specular reflection and can be used as a reference for the structural line zone region. The true thermal state when there is no specular reflection interference; the effective surface temperature is the composite temperature obtained by selecting the minimum value between the occluded surface temperature and the true surface temperature at the corresponding pixel position, used to eliminate abnormal temperatures caused by local reflection compensation errors and ensure the physical rationality of the final temperature field; the dew point temperature is the condensation temperature when the air reaches saturation, calculated based on the humidity and temperature in the warehouse environment, used to determine whether there is a risk of condensation; the dew point margin is the safety margin calculated by the difference between the effective surface temperature and the dew point temperature of the structural line area, representing the safe temperature distance between the current surface temperature and the condensation condition, and is an important indicator for determining whether to trigger the no-release strategy.

[0056] The instruction issuance module is used to issue instructions to the warehouse management system based on the dew point margin.

[0057] In an embodiment of the present invention, issuing instructions to the warehouse management system based on dew point margin includes: Based on the dew point margin, the pixel coordinate set within the target area of ​​the structural line band is projected and transformed, and the risk area is extracted to obtain the no-view band. Specifically, after calculating the dew point margin of each pixel within the target area of ​​the structural line strip, the set of pixel coordinates of the target area of ​​the structural line strip is used as a spatial index. The dew point margin of each pixel position is bound to its corresponding coordinate point one by one, forming a set of coordinate markers with quantitative information on temperature and humidity risks. Then, this set of coordinate markers is loaded into the projection transformation unit. Based on the geometry of the warehouse ceiling and the camera imaging model, the image coordinates are mapped point by point to the actual spatial location inside the warehouse, so that each pixel with a dew point margin less than or close to zero has a clear physical location in three-dimensional space. After completing the spatial positioning, the risk area extraction unit performs connectivity analysis, neighborhood merging, and shape closure processing on all spatial points that meet the condensation risk conditions, aggregating discrete risk points into continuous risk areas. Combined with the structural line strip mask, the area range is clipped and constrained, so that the final no-deposition zone strictly covers the structural line strip spatial area with condensation risk, thereby forming a highly reliable risk prohibition zone that can be used for warehouse management decisions.

[0058] Instructions were issued to the warehouse management system, including prohibiting the placement of moisture-sensitive goods within the no-release zone.

[0059] Specifically, after generating the no-deal zone, the spatial range of the no-deal zone and the corresponding structural line area location are loaded into the warehouse management interface unit. Based on the warehouse's location coding system, the codes of all actual locations within the no-deal zone are listed one by one, forming a list of prohibited moisture-sensitive goods. Subsequently, the control logic synchronizes this prohibited list with the scheduling module of the warehouse management system and automatically generates an information instruction containing the prohibited area range, prohibited goods type, prohibited duration, and execution strategy. After this instruction is pushed to the warehouse management system, the system will automatically check whether the target location falls within the no-deal zone when executing the shelving scheduling. When the location is within the no-deal zone, the system will reject any shelving application for moisture-sensitive goods and indicate the risk reason. In this way, the risk area is not misused through management strategies, realizing automatic risk avoidance capabilities in the intelligent warehousing environment.

[0060] Specifically, the target area of ​​the structural line zone is the actual monitored area determined by the structural line zone mask. This area, consisting of the metal seams of the ceiling and the purlin positions, is the controlled space where the potential condensation risk is most concentrated. The pixel coordinate set is the set of all image coordinate points constituting the target area of ​​the structural line zone. This set is used to map the dew point margin from the temperature domain back to the image space to form a regionalized risk marker. Projection transformation is the process of corresponding the dew point margin value with the pixel position of the structural line zone area, so that each temperature point with risk significance forms a spatial region on the image plane, thereby extracting the risk area. Risk area extraction is the process of connecting, clustering, or geometrically merging the pixels with dew point margins less than or close to zero, ultimately forming a no-release zone. This no-release zone is used to indicate the specific spatial range where there is a potential condensation risk. The warehouse management system is a management platform responsible for warehouse cargo scheduling, shelving rule formulation, and environmental constraint enforcement. By issuing instructions to this system, it can be prohibited from shelving moisture-sensitive goods within the no-release zone, thereby avoiding damage to the goods caused by condensation and realizing proactive safety control of the intelligent warehousing environment.

[0061] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An intelligent warehouse environment monitoring system based on multimodal sensor fusion, characterized in that, include: The reflection cone determination module is used to generate a structural line mask based on the visible image of the warehouse ceiling and to determine the center direction of the reflection cone based on the depth map of the warehouse. The steady-state temperature setting module is used to set the three steady-state surface temperatures of the blackbody plate based on the structural line mask and the center direction of the reflection cone. The reflection confirmation module is used to generate a specular reflection controllability confirmation mark based on three steady-state surface temperatures. The occlusion brightness module is used to generate occlusion brightness based on the structure line mask and the center direction of the reflection cone when the specular reflection controllability confirmation mark is 1. The margin calculation module is used to calculate the emissivity difference result based on three steady-state surface temperatures, and to calculate the dew point margin based on the emissivity difference result and the shading state brightness. The instruction issuance module is used to issue instructions to the warehouse management system based on the dew point margin.

2. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 1, characterized in that, A structural line mask is generated based on the visible image of the warehouse ceiling, and the center direction of the reflection cone is determined based on the depth map of the warehouse, including: Images of the warehouse ceiling are captured using smart sensors; Directional filtering and connected component analysis are performed on the visible image of the ceiling to obtain the initial region with linear edges; Binary segmentation and region filtering are performed on the initial region of the linear edge to obtain the structural line band mask; Calculate surface normal vectors based on the depth map of the warehouse; The line-of-sight direction of the thermal imager is obtained by calculating the imaging pose parameters of the thermal imager. The direction of the center of the reflection cone is determined based on the line of sight of the thermal imager and the surface normal vector.

3. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 2, characterized in that, Three steady-state surface temperatures of the blackbody plate are used to set the direction of the structured strip mask and the center of the reflection cone, including: The structural line strip mask is subjected to region screening to obtain the structural line strip region; Determine the center direction of the reflection cone of each pixel within the structural line strip area; Within the structural line zone, calculate the average direction vector in the direction of the center of the reflection cone; Install a temperature-controlled blackbody plate inside the warehouse; Alignment settings are configured for the emission surface normal vector and the average direction vector of the blackbody plate; After the alignment settings are completed, the three steady-state surface temperatures of the blackbody plate are set and denoted as the first background temperature, the second background temperature, and the third background temperature, respectively.

4. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 3, characterized in that, The controllability verification markers for specular reflection are generated based on three steady-state surface temperatures, including: Parallel polarization sampling was performed on the thermal image of the structural line strip region at each steady-state surface temperature to obtain a parallel polarization brightness image; At each steady-state surface temperature, the thermal image of the structural line strip region is sampled by vertical polarization to obtain a vertical polarization brightness image; Pixel-level difference operations are performed on parallel polarization brightness images and vertical polarization brightness images to obtain polarization difference values; Monotonicity verification was performed based on the temperature response relationship between steady-state surface temperature and polarization difference to obtain a confirmation mark for the controllability of specular reflection.

5. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 3, characterized in that, When the specular reflection controllability confirmation flag is 1, the occlusion state brightness is generated based on the structured strip mask and the center direction of the reflection cone, including: Based on the structure line mask and the center direction of the reflection cone, the shading path of the structure line region pointing towards the blackbody plate along the direction of the reflection cone is determined; A shielding plate is placed along the shielding path so that the specular reflection path in the structural line area is blocked by the shielding plate; While maintaining the background temperature of the blackbody plate in the first phase, the position and orientation of the occlusion plate remain unchanged, and a thermal imaging smart sensor is used to collect brightness images under the occlusion state. Pixels in the structural line band region are extracted from the brightness image under occlusion to obtain the occlusion brightness.

6. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 3, characterized in that, The emissivity difference was calculated based on three steady-state surface temperatures, including: The thermal imaging scene of the warehouse roof corresponding to three steady-state surface temperatures was captured, and the first frame brightness image, the second frame brightness image, and the third frame brightness image were obtained respectively. Perform a difference operation on the first frame brightness image and the second frame brightness image to obtain a brightness difference image; The difference between the fourth power of the first background temperature and the fourth power of the second background temperature is calculated to obtain the fourth power difference of the background temperature. The emissivity difference result is obtained by dividing the pixel brightness value of the brightness difference image by the product of the Stefan-Boltzmann constant and the fourth power difference of the background temperature.

7. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 6, characterized in that, Dew point margin is calculated based on emissivity difference results and occlusion state brightness, including: The emissivity difference result is multiplied by the Stefan-Boltzmann constant and the fourth power of the steady-state surface temperature to obtain the reflection brightness compensation term; The self-radiation brightness value is obtained by calculating the difference between the pixel brightness value of the brightness image corresponding to the steady-state surface temperature and the reflection brightness compensation term. The emissivity radiation coefficient is obtained by multiplying the emissivity by the Stefan-Boltzmann constant. Divide the self-radiance value by the emissivity radiation coefficient to obtain the fourth power value of the surface temperature; The true surface temperature is obtained by taking the fourth root of the fourth power value of the surface temperature. The fourth root of the result of dividing the brightness of the shaded state by the emissivity radiative coefficient is used to obtain the surface temperature of the shaded state. The effective surface temperature is obtained by performing a minimum value selection operation on the shielded surface temperature and the true surface temperature; The dew point margin is obtained by calculating the difference between the effective surface temperature and the dew point temperature of the structural line zone.

8. The intelligent warehouse environment monitoring system based on multimodal sensor fusion according to claim 3, characterized in that, Instructions are issued to the warehouse management system based on dew point margin, including: Based on the dew point margin, the pixel coordinate set within the target area of ​​the structural line band is projected and transformed, and the risk area is extracted to obtain the no-view band. Instructions were issued to the warehouse management system, including prohibiting the placement of moisture-sensitive goods within the no-release zone.

Images