Video multi-level accurate display method and system based on sight distance segmentation

Abstract

The application provides a video multi-level accurate display method and system based on a sight distance segmentation, and relates to the technical field of image processing. The method comprises the following steps: performing adaptive boundary scanning to identify an object boundary, taking one boundary point in the object boundary as a seed point, and cutting out a first heat source area and a second heat source area on a spatial topological structure corresponding to a low light scene through a region growing algorithm driven by a decay component in a thermal gradient matrix; marking depth levels corresponding to the first heat source area and the second heat source area according to intensity mutation characteristics of a mutation component and exponential decay characteristics of the decay component in the thermal gradient matrix; performing corresponding adjustment on the first heat source area and the second heat source area respectively based on the depth level marking, and fusing the adjusted first heat source area and the second heat source area to generate an optimized display image. The application can realize multi-level accurate display of a video in a low light environment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for accurate multi-layer video display based on line-of-sight segmentation. Background Technology

[0002] In infrared thermal imaging video applications in low-light environments, the core technical problem that needs to be solved in order to achieve effective observation of both nearby key targets and distant interfering heat sources at the same time is: how to accurately distinguish heat source objects at different distances and perform targeted image enhancement processing on each one.

[0003] Current mainstream technical solutions typically employ a combination of physical models and frequency domain analysis. This method first decomposes the thermal image into low-frequency background components and high-frequency edge components through frequency domain transformation, assuming that the low-frequency components correspond to distant heat diffusion areas and the high-frequency components correspond to near-field object contours. Subsequently, an atmospheric scattering model is used to compensate for the low-frequency components to recover distant details, while techniques such as neural networks are employed to enhance the high-frequency components to improve near-field texture. Finally, the processed components are fused to output an optimized image.

[0004] However, the above-mentioned schemes have the following significant limitations: First, their physical modeling process is relatively simplified and fails to fully consider the physical directionality differences in the gradient changes of thermal radiation during propagation, resulting in inaccurate differentiation between heat sources in the foreground and background, and making them prone to misjudgment under complex weather conditions. Second, the frequency domain decomposition-based processing method is prone to introducing spectral aliasing in areas with complex object edges or multiple heat sources, thus producing jagged artifacts in the enhanced image, affecting the clarity of foreground textures and the coherence of background thermal distribution. Third, such methods typically rely on a dual-channel parallel processing architecture and computationally complex neural network models, requiring high hardware computing power and making it difficult to implement real-time video processing on resource-constrained edge devices. Summary of the Invention

[0005] This application provides a method and system for precise multi-layer video display based on line-of-sight segmentation, which solves the problem of poor image display in existing technologies due to the inability to balance processing accuracy, visual quality and system efficiency in low-light complex scenes.

[0006] Firstly, this application provides a method for precise multi-layer video display based on line-of-sight segmentation, including:

[0007] Convert the infrared thermal radiation data of the image to be processed in the target video in a low-light scene into the corresponding temperature mapping data;

[0008] Wide dynamic range synthesis is performed on the temperature mapping data to compensate for the temperature difference loss between the dark and bright areas of the target in the low-light scene, and a dynamic heat distribution map is generated based on the compensation result of the temperature difference loss.

[0009] Construct the corresponding thermal gradient matrix based on the dynamic heat distribution map;

[0010] An adaptive boundary scan is performed to identify the object boundary. Using a boundary point in the object boundary as a seed point, a region growing algorithm driven by the attenuation component in the thermal gradient matrix is ​​used to segment the first heat source region and the second heat source region on the spatial topology corresponding to the low-light scene.

[0011] Based on the intensity abrupt change characteristics of the abrupt change component and the exponential decay characteristics of the decay component in the thermal gradient matrix, the depth levels corresponding to the first heat source region and the second heat source region are marked.

[0012] Based on the deep hierarchical labeling, corresponding adjustments are performed on the first heat source region and the second heat source region respectively, and the adjusted first heat source region and second heat source region are merged to generate an optimized display image.

[0013] Optionally, an adaptive boundary scan is performed to identify object boundaries. Using a boundary point within the object boundaries as a seed point, a region growing algorithm driven by the decay component in the thermal gradient matrix is ​​used to segment a first heat source region and a second heat source region on the spatial topology corresponding to the low-light scene, including:

[0014] The boundary determination threshold is adaptively determined by scanning the gradient distribution of the thermal gradient matrix. All positions in the thermal gradient matrix that exceed the boundary determination threshold are connected as boundary points to obtain the object boundary.

[0015] For each boundary point in the object boundary, as a seed point, the difference between the maximum and minimum values ​​of the attenuation component is calculated based on the attenuation components of all positions in the neighborhood of the seed point using a region growing algorithm driven by the attenuation component, and is used as a dynamic similarity threshold.

[0016] Traverse each adjacent position directly connected to the seed point in the spatial topology and calculate the difference between the attenuation component of the adjacent position and the attenuation component of the seed point.

[0017] The adjacent positions with a difference less than or equal to the dynamic similarity threshold are included in the second heat source region, and the adjacent positions with a difference greater than the dynamic similarity threshold are included in the first heat source region.

[0018] Optionally, based on the intensity abrupt change characteristics of the abrupt change components and the exponential decay characteristics of the decay components in the thermal gradient matrix, the depth levels corresponding to the first heat source region and the second heat source region are marked, including:

[0019] Traverse each location in the first heat source region, calculate the maximum absolute difference between the abrupt change component of the location and the abrupt change component of the adjacent location in the spatial topology, and use it as the change amplitude value. Based on the statistical distribution characteristics of all abrupt change components in the thermal gradient matrix, determine the intensity abrupt change threshold.

[0020] The locations where the change amplitude value is greater than the intensity abruptness threshold are marked as the first depth level, and the locations where the change amplitude value is less than or equal to the intensity abruptness threshold are marked as the second depth level;

[0021] Traverse each location in the second heat source region, obtain the attenuation component stored at the location in the thermal gradient matrix as the actual component value, and calculate the shortest topological distance from the location to the object boundary.

[0022] Based on the shortest topological distance and the preset attenuation coefficient, the expected attenuation value is calculated, and a floating range is set with the expected attenuation value as the center as the expected attenuation range;

[0023] The positions where the actual component values ​​are within the expected attenuation range are marked as the third depth level, and the positions where the actual component values ​​are not within the expected attenuation range are marked as the fourth depth level.

[0024] Optionally, based on depth-level marking, corresponding adjustments are performed on the first heat source region and the second heat source region respectively, and the adjusted first heat source region and second heat source region are merged to generate an optimized display image, including:

[0025] Based on the depth level marker, the saturation component of the pseudo-color image corresponding to the first heat source region is subjected to nonlinear stretching to generate a pseudo-color sub-image.

[0026] Based on the depth level marker, a piecewise linear transformation is performed on the brightness component of the brightness image corresponding to the second heat source region to generate an initial brightness sub-image;

[0027] Perform a Gaussian weighted average calculation on the initial brightness submap to generate the target brightness submap;

[0028] The pseudo-color sub-image and the target brightness sub-image are merged through spatial registration and fusion operations to generate an optimized display image.

[0029] Secondly, this application provides a video multi-layer precise display system based on line-of-sight segmentation, including:

[0030] The conversion module is used to convert the infrared thermal radiation data of the image to be processed in the target video in a low-light scene into the corresponding temperature mapping data.

[0031] The synthesis module is used to perform wide dynamic range synthesis on the temperature mapping data to compensate for the temperature difference loss between the dark area and the bright area of ​​the target in the low-light scene, and to generate a dynamic heat distribution map based on the compensation result of the temperature difference loss.

[0032] The construction module is used to construct the corresponding thermal gradient matrix based on the dynamic heat distribution map;

[0033] The segmentation module is used to perform adaptive boundary scanning to identify object boundaries. Using a boundary point in the object boundary as a seed point, the first heat source region and the second heat source region are segmented on the spatial topology corresponding to the low-light scene by a region growing algorithm driven by the attenuation component in the thermal gradient matrix.

[0034] The marking module is used to mark the depth levels corresponding to the first heat source region and the second heat source region based on the intensity abruptness characteristics of the abruptness component and the exponential decay characteristics of the decay component in the thermal gradient matrix.

[0035] The fusion module is used to perform corresponding adjustments on the first heat source region and the second heat source region based on the depth level label, and then merge the adjusted first heat source region and the second heat source region to generate an optimized display image.

[0036] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to realize a method for precise multi-layer video display based on line-of-sight segmentation as described in the first aspect above.

[0037] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a method for precise multi-layer video display based on line-of-sight segmentation as described in the first aspect.

[0038] The video multi-layer accurate display method based on line-of-sight segmentation provided in this application has the following beneficial effects:

[0039] This application constructs a thermal gradient matrix based on a dynamic thermal distribution map, accurately quantifying the temperature change characteristics of a scene. Using the attenuation component in this matrix to drive a region growing algorithm with object boundary points as seeds, it achieves physical separation of the first and second heat source regions in terms of spatial topology, completing the core line-of-sight segmentation. Furthermore, based on the intensity characteristics of the abrupt change component and the exponential characteristics of the attenuation component in the thermal gradient matrix, the segmented first and second heat source regions are marked with a depth hierarchy, constructing a hierarchical heat source model. Finally, based on the depth hierarchy marking, the first and second heat source regions are adjusted and merged, ultimately outputting an optimized display image that achieves accurate distance separation and clear multi-layered display of heat source targets in low-light environments.

[0040] Furthermore, this application accurately identifies temperature abrupt change edges through adaptive boundary scanning, establishing an objective physical boundary basis for distance segmentation; then, it dynamically calculates the region growth threshold using the attenuation component in the thermal gradient matrix, enabling the segmentation process to adapt to the attenuation characteristics of thermal radiation at different distances, thereby achieving accurate separation of near-field and far-field heat sources in spatial topology and completing basic line-of-sight segmentation; on this basis, by comparing the differences in attenuation components pixel by pixel, not only is high-precision spatial segmentation achieved, but also a quantitative basis is provided for subsequent differentiation of multi-level depth layers.

[0041] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart of a method for precise multi-layer video display based on line-of-sight segmentation provided in this application is shown;

[0044] Figure 2 This paper presents a schematic diagram of the structure of a video multi-layer precision display system based on line-of-sight segmentation provided in this application;

[0045] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0047] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0048] Existing technologies rely on frequency domain decomposition and simplified physical models, making it difficult to directly and accurately distinguish heat sources at different distances in the spatial domain, resulting in inaccurate line-of-sight segmentation and coarse layer display.

[0049] To address the aforementioned issues, this application proposes a method for accurate multi-layer video display based on line-of-sight segmentation. The core idea of ​​this method is to achieve physical separation of near and far heat sources in the image spatial topology by using an adaptive region growing algorithm driven by the attenuation component in the thermal gradient matrix, and to mark multiple depth levels by combining abrupt change and attenuation features, thereby completing accurate line-of-sight segmentation and multi-layer optimized display.

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Figure 1 A flowchart of a method for precise multi-layer video display based on line-of-sight segmentation is provided in this application embodiment, as follows: Figure 1 As shown, the method includes:

[0052] 101. Convert the infrared thermal radiation data of the image to be processed in the target video under low light scene into the corresponding temperature mapping data.

[0053] Infrared thermal radiation data refers to the raw electrical signal sequence captured by the sensor array, while temperature mapping data can be a two-dimensional temperature distribution map.

[0054] In one specific implementation, step 101 includes:

[0055] 1011. The infrared thermal radiation data of each image to be processed in a low-light scene is parsed into a corresponding first thermal response set, wherein each thermal response value in the first thermal response set corresponds to the spatial position of a detection unit in the sensor array.

[0056] The first thermal response set refers to the spatial distribution of the output values ​​of all detection units in the sensor array at a single moment.

[0057] In step 1011, the original binary data stream is acquired, and the length of a single frame data is calculated according to preset parameters. Then, the original binary data stream is divided according to this length to obtain independent single frame data blocks. The preset parameters include the width and height of the sensor array and the number of bytes of data for each detection unit.

[0058] For each spatial location, the data offset is calculated based on its coordinates in the sensor array, and multiple bytes of data corresponding to the offset are extracted. Then, the byte data is converted into the thermal response value of that location, and finally a single-frame thermal response matrix F is output, where the element F(i,j) represents the instantaneous thermal response value of the detection unit in the i-th row and j-th column of the sensor array.

[0059] 1012. Based on multiple first thermal response sets, perform a superposition operation on multiple thermal response values ​​at the same spatial location to generate a second thermal response set, wherein the second thermal response set includes numerical fluctuation characteristics of adjacent locations.

[0060] Among them, the numerical fluctuation feature reflects the signal stability of adjacent units, and the second thermal response set refers to the dataset enhanced by superimposing multiple frames.

[0061] In step 1012, based on multiple first thermal response sets, a superposition operation is performed on multiple thermal response values ​​corresponding to each spatial location to generate a second thermal response set; at the same time, based on the thermal response value sequence of each location in multiple frames, its mean is calculated, and a standard deviation matrix is ​​generated based on the mean to reflect the signal stability of adjacent units, so as to characterize the numerical fluctuation characteristics contained in the second thermal response set; wherein, the superposition operation and the noise statistics calculation can both be performed according to a preset number of superposition frames.

[0062] 1013. Based on the numerical fluctuation characteristics of the adjacent positions, generate an adjustment threshold to distinguish between valid signals and noise, and filter the cumulative thermal response values ​​in the second thermal response set that are lower than the adjustment threshold.

[0063] The adjustment threshold refers to the filtering threshold that is adaptive based on the local noise level.

[0064] In step 1013, based on the numerical fluctuation characteristics of adjacent positions represented by the standard deviation matrix, the average fluctuation level in the local neighborhood of each position is calculated, and an adaptive adjustment threshold is generated accordingly. Subsequently, the cumulative thermal response values ​​in the second thermal response set are filtered based on the adjustment threshold, and response values ​​below the threshold are filtered out, and the dynamically filtered data is output.

[0065] 1014. Based on the preset correspondence rules between response intensity and temperature, each cumulative thermal response value is converted into a corresponding physical temperature value. All physical temperature values ​​are sorted according to the original arrangement order of the sensor array to generate a two-dimensional temperature distribution map as temperature mapping data.

[0066] The correspondence between response intensity and temperature refers to the mapping relationship between sensor voltage and physical temperature, which can be achieved through a calibration function. This calibration function can be a quadratic polynomial, and this embodiment does not specifically limit the specific expression of the function. The two-dimensional temperature distribution map refers to the temperature matrix arranged according to the sensor space.

[0067] In step 1014, based on the preset correspondence rules between response intensity and temperature, the cumulative thermal response value after dynamic filtering is converted into the corresponding physical temperature value; then, the physical temperature value is integrated according to the original spatial arrangement order of the detection units in the sensor array to generate a two-dimensional temperature distribution map as temperature mapping data; wherein, the correspondence rules between response intensity and temperature are implemented through a calibration function, and the specific function form is determined according to the sensor characteristics.

[0068] In the data preprocessing stage, this application effectively enhances signal strength in low-light environments by generating a second thermal response set through multi-frame overlay. Then, based on adaptive threshold adjustment, the accumulated response values ​​after overlay are dynamically filtered, thereby suppressing random noise and local interference while preserving the true heat source signal. On this basis, a preset calibration function converts the filtered response values ​​into physically meaningful temperature values, and a two-dimensional temperature distribution map is generated according to their spatial location. This series of operations achieves effective signal purification and accurate temperature conversion, while providing a reliable data foundation with sufficient noise suppression and high spatial fidelity for subsequent line-of-sight segmentation and multi-level analysis.

[0069] 102. Perform wide dynamic range synthesis on the temperature mapping data to compensate for the temperature difference loss between the dark area and the bright area of ​​the target in the low-light scene, and generate a dynamic heat distribution map based on the compensation result of the temperature difference loss.

[0070] Construct the corresponding thermal gradient matrix based on the dynamic heat distribution map.

[0071] In one specific implementation, step 102 includes:

[0072] 1021. Acquire multiple sets of temperature mapping data for the same low-light scene, where each set of temperature mapping data is generated based on a different signal acquisition duration.

[0073] The signal acquisition duration refers to the duration of a single temperature mapping data acquisition, which can be used as the integration duration. Different integration durations are used to capture different brightness details.

[0074] In step 1021, the control sensor array collects data on the same scene at three different integration times: short, medium, and long, generating three sets of temperature mapping data matrices with the same matrix dimension as the sensor resolution. The short integration time data is used to preserve weak signals in dark areas, the medium integration time data is used to record basic thermal distribution, and the long integration time data is used to preserve high dynamic details in bright areas. In this embodiment, the specific values ​​of each time period are not specifically limited.

[0075] 1022. Perform a weighted superposition operation on the physical temperature values ​​with the same spatial location in each group of temperature mapping data to generate a composite temperature value.

[0076] In step 1022, the total integration time is first calculated, and the weight of each group is determined according to the ratio of the integration time of each group to the total time. Then, each spatial location is traversed, and the temperature value of the location in the three matrices is multiplied by the corresponding weight and then added together to obtain the composite temperature value of the location. Finally, a composite temperature value matrix of the same size as the input data is formed.

[0077] 1023. The composite temperature value and sensitivity component of each spatial location in the dark area of ​​the target are superimposed, and the composite temperature value and saturation component of each spatial location in the bright area of ​​the target are superimposed to obtain the superimposed composite temperature value.

[0078] The sensitivity component is provided by short integration time data and is used to enhance details in dark areas; the saturation component is provided by long integration time data and is used to restore details in bright areas. The target dark area refers to the region where the synthesis temperature value is lower than the first preset threshold, and the target bright area refers to the region where the synthesis temperature value is higher than the second preset threshold. In this embodiment, the numerical value of each preset threshold is not specifically limited.

[0079] In step 1023, the statistical characteristics of each position in the synthetic temperature value matrix in the local neighborhood are first calculated, such as the standard deviation, to characterize the temperature fluctuation of that position. For each position, if its synthetic temperature value is lower than the first preset threshold and the local statistical characteristics are greater than the noise threshold, it is determined to be a dark area. The synthetic temperature value of that position is then weighted and fused with the corresponding value in the data collected in a short period of time to enhance the sensitivity of the dark area.

[0080] If the synthesized temperature value is higher than the second preset threshold and the local statistical characteristics are greater than the noise threshold, it is determined to be a bright area. The synthesized temperature value at this location is then weighted and fused with the corresponding value in the data collected over a long period of time to suppress oversaturation; the synthesized temperature value at other locations remains unchanged.

[0081] 1024. Integrate the superimposed composite temperature values ​​corresponding to all spatial locations according to their spatial positions to generate a dynamic heat distribution map.

[0082] In step 1024, the combined temperature values ​​of all locations processed in step 1023 are recombined into a complete two-dimensional matrix according to their original row and column coordinates, which serves as the final dynamic heat distribution map.

[0083] This application effectively solves the imaging bottleneck of simultaneous loss of dark details and oversaturation of bright areas under single exposure by acquiring multiple sets of temperature mapping data with different integration times, and performing weighted superposition and zonal compensation. The method enhances the identification of low-temperature targets and the preservation of high-temperature gradient information in a unified thermal distribution map, thus providing a data foundation with complete dynamic range and spatial continuity for subsequent line-of-sight segmentation and multi-level analysis.

[0084] 103. Construct the corresponding thermal gradient matrix based on the dynamic heat distribution map.

[0085] In one specific implementation, the thermal gradient matrix includes abrupt change components and attenuation components; step 103 includes:

[0086] 1031. Calculate the temperature gradient between adjacent locations in the dynamic heat distribution map and generate an initial matrix.

[0087] Here, the temperature gradient is a vector representing the rate and direction of temperature change between adjacent pixels. In step 1031, based on the input dynamic heat distribution map, the temperature difference between each pixel location and its adjacent pixels in the horizontal and vertical directions is calculated as the gradient component of that location. Subsequently, the two gradient components of each location are combined and arranged according to their original spatial coordinates to form an initial thermal gradient matrix, which fully records the spatial characteristics of temperature change at each location in the scene.

[0088] 1032. Taking each position in the initial matrix as the center position, based on the local mutation detection technology, calculate the absolute difference between the gradient value of the center position and the average gradient value in the local neighborhood. When the absolute difference exceeds the preset mutation threshold, mark the gradient data of the center position as a mutation component.

[0089] Among them, the mutation component refers to the feature point where the gradient value changes drastically in the local space, which usually corresponds to the physical edge of the object or the outline of the near heat source.

[0090] In step 1032, the gradient magnitude at each location in the initial thermal gradient matrix is ​​first calculated. Then, for each location, the average gradient magnitude of all locations within a local neighborhood of a preset size, such as a 3×3 pixel range, is calculated. Subsequently, the gradient magnitude at that location is compared with the aforementioned local average. If the absolute value of the difference between the two exceeds a preset abrupt change threshold, it is determined that there is a temperature abrupt change at that location, and the gradient data corresponding to it in the initial thermal gradient matrix is ​​marked as an abrupt change component.

[0091] 1033. Identify the center location of the heat source in the thermal gradient matrix and determine at least one analysis path along the direction of thermal radiation propagation; extract the gradient data at each location on each analysis path to form a gradient data sequence; when the gradient data sequence exhibits an exponential decay characteristic, mark the gradient data in the gradient data sequence as decay components.

[0092] Among them, the attenuation component refers to the characteristic data sequence in which the gradient amplitude value exhibits a regular exponential decay along the direction of thermal radiation propagation, corresponding to the physical law of thermal radiation spreading with distance in space.

[0093] In step 1033, one or more heat source center locations are first identified in the initial thermal gradient matrix. These locations typically have significantly higher cumulative gradient values. Starting from each heat source center, multiple analysis paths are defined outward along multiple radiation directions, such as horizontal, vertical, or diagonal. Along each analysis path, the gradient magnitude of each location in the initial thermal gradient matrix is ​​extracted sequentially to form a gradient data sequence. The sequence is fitted and analyzed. If its trend conforms to a preset exponential decay model, the gradient data corresponding to all locations on the path are marked as decay components in the initial thermal gradient matrix.

[0094] Step 1034: Combine the initial matrix, the mutation component, and the decay component to obtain the thermal gradient matrix.

[0095] This application constructs a thermal gradient matrix and simultaneously extracts gradient abrupt change features representing the object's contour and gradient attenuation features representing heat propagation, providing a dual-dimensional discrimination basis that combines structural and physical characteristics for subsequent line-of-sight segmentation and multi-level analysis.

[0096] 104. Perform adaptive boundary scanning to identify object boundaries. Using one of the boundary points of the object boundary as a seed point, use the region growing algorithm driven by the decay component in the thermal gradient matrix to segment the first heat source region and the second heat source region on the spatial topology corresponding to the low-light scene.

[0097] The second heat source region refers to the distant heat target region; the first heat source region refers to the nearby heat target region.

[0098] In one specific implementation, step 104 includes:

[0099] 1041. By adaptively determining the boundary determination threshold through scanning the gradient distribution of the thermal gradient matrix, all positions in the thermal gradient matrix that exceed the boundary determination threshold are connected as boundary points to obtain the object boundary.

[0100] Here, the object boundary refers to the set of pixel locations in the image where the temperature gradient changes significantly, forming a clear dividing line. In step 1041, based on the thermal gradient matrix, the minimum, maximum, and average values ​​of all gradient amplitudes in the entire image are statistically analyzed. Then, based on these global statistical features, a boundary determination threshold is dynamically calculated, such as the weighted sum of the differences between the average value and the maximum and minimum values. Finally, all pixel locations with gradient amplitudes exceeding this dynamic threshold are marked as object boundary points, and the set of boundary points is output.

[0101] 1042. For each boundary point in the object boundary, as a seed point, a region growing algorithm driven by the decay component is used to calculate the difference between the maximum and minimum values ​​of the decay component based on the decay components of all positions in the neighborhood of the seed point, which is used as a dynamic similarity threshold.

[0102] Among them, the attenuation component is the feature value in the thermal gradient matrix that represents the exponential decay of thermal radiation with distance. The dynamic similarity threshold is used to determine whether adjacent positions belong to the same line-of-sight region in region growing. The region growing algorithm refers to the technique of gradually expanding outward from the starting point to identify similar regions.

[0103] In step 1042, each object boundary point is used as a seed point for region growth. For each seed point, the attenuation component value corresponding to the position of all pixels in the surrounding set neighborhood in the thermal gradient matrix is ​​extracted. Then, the difference between the maximum and minimum values ​​of the attenuation component values ​​in the neighborhood is calculated, and this difference is multiplied by a preset scaling factor to generate a dynamic similarity threshold that matches the local thermal radiation attenuation characteristics of the current seed point. This threshold will be used as the standard for subsequent judgment of whether adjacent pixels belong to the same viewing distance region.

[0104] 1043. Traverse each adjacent position directly connected to the seed point in the spatial topology and calculate the difference between the attenuation component of the adjacent position and the attenuation component of the seed point.

[0105] Among them, the attenuation component value refers to the feature value used to distinguish between near and far distances; the direct connection position refers to the position adjacent to the seed point.

[0106] 1044. The adjacent positions with a difference less than or equal to the dynamic similarity threshold are included in the second heat source region, and the adjacent positions with a difference greater than the dynamic similarity threshold are included in the first heat source region.

[0107] In step 1044, the difference in attenuation components between each adjacent location is compared with the dynamic similarity threshold of the corresponding seed point. If the difference is less than or equal to the dynamic similarity threshold, the adjacent location is considered to have similar attenuation characteristics to the seed point and is classified into the second heat source region. If the difference is greater than the dynamic similarity threshold, the attenuation characteristics are considered to be significantly different and is classified into the first heat source region. By traversing all seed points and their adjacent locations, the above judgment and classification are completed, and finally, the pixel sets of the divided first and second heat source regions are output.

[0108] This application accurately identifies object contours through adaptive boundary scanning, and uses boundary points as seeds to dynamically calculate local growth thresholds using attenuation components. By comparing the attenuation differences between neighboring pixels and seed points, it achieves high-precision automated separation of near-field and far-field heat sources, providing an accurate line-of-sight segmentation basis for subsequent depth-level labeling and multi-layer display.

[0109] 105. Based on the intensity abrupt change characteristics of the abrupt change component and the exponential decay characteristics of the decay component in the thermal gradient matrix, mark the depth levels corresponding to the first heat source region and the second heat source region.

[0110] In one specific implementation, step 105 includes:

[0111] 1051. Traverse each position in the first heat source region, calculate the maximum absolute difference between the abrupt change component of the position and the abrupt change component of the adjacent position in the spatial topology, and use it as the change amplitude value. Based on the statistical distribution characteristics of all abrupt change components in the thermal gradient matrix, determine the intensity abrupt change threshold.

[0112] Among them, spatial topology refers to the connection model that describes the adjacency relationship between locations, the change magnitude value refers to the largest absolute value among the differences between the mutation components of this location and all its neighboring locations, and the intensity mutation threshold is a global threshold determined based on the statistical distribution of all mutation components, such as taking the top 95 percentile, which is used to distinguish between drastic mutations and ordinary changes.

[0113] In step 1051, each pixel position in the first heat source region is traversed, and the absolute difference between it and each adjacent position in the thermal gradient matrix is ​​calculated. The maximum value among all differences is taken as the change amplitude value of that position. At the same time, based on the statistical distribution of all the abrupt change component values ​​in the entire thermal gradient matrix, the intensity abrupt change threshold as the boundary standard is calculated.

[0114] 1052. The positions where the change amplitude value is greater than the intensity mutation threshold are marked as the first depth level, and the positions where the change amplitude value is less than or equal to the intensity mutation threshold are marked as the second depth level.

[0115] Each depth level is used to characterize the relative position of the heat source in the direction perpendicular to the imaging direction. The first depth level can correspond to the shallow layer and the region with drastic surface changes, while the second depth level can correspond to the middle layer and the region with gentle subsurface changes.

[0116] 1053. Traverse each position in the second heat source region, obtain the attenuation component stored in the thermal gradient matrix at the position as the actual component value, and calculate the shortest topological distance from the position to the object boundary.

[0117] The topological distance can be calculated using Manhattan distance, which represents the minimum number of pixel steps required to reach the nearest object boundary from that location.

[0118] 1054. Based on the shortest topological distance and the preset attenuation coefficient, calculate the expected attenuation value, and set a floating interval centered on the expected attenuation value as the expected attenuation range.

[0119] The expected attenuation value refers to the ideal attenuation value calculated based on the distance; the floating range refers to the reasonable fluctuation range of the attenuation value allowed.

[0120] In step 1054, the following formula can be used: Calculate the expected decay value, where The expected attenuation value is given by γ, the preset attenuation coefficient is given by dt, and the floating range is set to [Qmin, Qmax]. For example, Qmin = 0.9 × Ed and Qmax = 1.1 × Ed.

[0121] 1055. Mark the positions where the actual component values ​​are within the expected attenuation range as the third depth level, and mark the positions where the actual component values ​​are not within the expected attenuation range as the fourth depth level.

[0122] Among them, the third depth level can correspond to a deep, distant heat source that conforms to the physical attenuation law, while the fourth depth level can correspond to a medium-deep heat source with an abnormal attenuation law.

[0123] Based on the intensity and attenuation of abrupt change components, this application automatically distinguishes between shallow and intermediate layers within the first heat source region according to the magnitude of local abrupt changes. Within the second heat source region, it automatically distinguishes between deep and mid-deep layers by comparing measured attenuation values ​​with distance-based physical prediction models. This method achieves multi-layered, physically interpretable depth-level labeling of thermal scenes, providing a structured and distinguishable depth feature foundation for subsequent line-of-sight perception and precise target localization.

[0124] 106. Based on the depth-level labeling, perform corresponding adjustments on the first heat source region and the second heat source region respectively, and merge the adjusted first heat source region and the second heat source region to generate an optimized display image.

[0125] In one specific implementation, step 106 includes:

[0126] 1061. Based on the depth level marker, the saturation component of the pseudo-color image corresponding to the first heat source region is subjected to nonlinear stretching to generate a pseudo-color sub-image.

[0127] Among them, depth level label is a label corresponding to shallow, middle, medium-deep, and deep layers, pseudo-color image refers to a visual image that maps temperature values ​​to colors, and saturation component refers to the color vividness index.

[0128] In step 1061, for each position marked as a shallow layer corresponding to a marker S or a middle layer corresponding to a marker M, it is assigned to the first heat source region set; subsequently, for the pseudo-color image corresponding to the first heat source region, a nonlinear transformation, such as exponential or piecewise function stretching, is specifically performed on its saturation component; this transformation aims to increase the color saturation difference between adjacent temperature ranges, thereby highlighting the texture details of the near-field target surface, generating a texture-enhanced pseudo-color sub-image that can perform visual optimization for the characteristics of the near-field region, enhancing the target recognition.

[0129] 1062. Based on the depth level marker, perform piecewise linear transformation on the brightness component of the brightness image corresponding to the second heat source region to generate an initial brightness sub-image.

[0130] Among them, the brightness image is a grayscale image generated by mapping temperature values, and its brightness component directly reflects the intensity of thermal radiation.

[0131] In the specific implementation of step 1062, the positions of markers D corresponding to deep layers or MD corresponding to mid-deep layers are identified and assigned to the second heat source region set. For the brightness image corresponding to the region, a preset piecewise linear transformation is performed on its brightness components. The core of this transformation is to expand the dynamic range of the main thermal radiation intensity range in the image, while appropriately compressing the extremely dark and extremely bright parts, so as to enhance the overall contrast of the distant thermal scene without distortion, and generate an initially optimized brightness sub-image for adaptive adjustment to address the problem of weak contrast and limited dynamic range in the distant region.

[0132] 1063. Perform a Gaussian weighted average calculation on the initial brightness sub-map to generate the target brightness sub-map.

[0133] Gaussian weighted average is a spatial filtering operation that uses a Gaussian kernel function to perform a weighted average of pixels in the neighborhood, which can effectively smooth noise and preserve edges.

[0134] In the specific implementation of step 1063, in order to suppress noise in the distant area, a local neighborhood window, such as 3×3 pixels, is first defined for each pixel position in the initial brightness sub-image. Then, the Gaussian weighted average of the brightness values ​​of all pixels in the window is calculated, and this smoothed value is used to replace the original brightness value at the center position. This operation can effectively suppress random noise caused by atmospheric attenuation, sensor noise, etc., while maintaining the overall contour and trend of the heat distribution, and finally generating a target brightness sub-image with significantly suppressed noise.

[0135] 1064. The pseudo-color sub-image and the target brightness sub-image are merged through spatial registration and fusion operations to generate an optimized display image.

[0136] Spatial registration and fusion refers to the precise alignment and synthesis of two separately optimized sub-images based on their original spatial coordinates.

[0137] In the specific implementation of step 1064, the pixel source in the final fused image is determined according to the depth level label of each pixel. For example, for the position labeled S or M, the corresponding color pixel is extracted from the texture-enhanced pseudo-color sub-image; for the position labeled D or MD, the corresponding brightness and grayscale pixels are extracted from the noise-suppressed target brightness sub-image; then, all pixels are integrated according to the unified spatial coordinates to generate a complete optimized display image.

[0138] The image features rich color and texture details in the foreground area and high-contrast, low-noise thermal distribution in the background area, achieving a multi-layered, high-definition thermal imaging visualization effect.

[0139] This application performs differentiated enhancements based on the different characteristics of near-field and far-field heat sources. Specifically, saturation stretching is applied to the near-field region to enhance texture details, while piecewise linear transformation is used to expand the dynamic range of the far-field region and Gaussian filtering is used to suppress noise. Then, the processed sub-images are precisely fused according to the depth markers to output an optimized thermal image with clear near-field textures, high far-field contrast, and low noise, achieving a multi-layered, high-fidelity visualization effect.

[0140] Figure 2 This application provides a schematic diagram of the structure of a video multi-layer precise display system based on line-of-sight segmentation, as shown in the embodiment of the present application. Figure 2 As shown, the system includes:

[0141] The conversion module 21 is used to convert the infrared thermal radiation data of the image to be processed in the target video in a low-light scene into the corresponding temperature mapping data.

[0142] The synthesis module 22 is used to perform wide dynamic range synthesis on the temperature mapping data to compensate for the temperature difference loss between the dark area and the bright area of ​​the target in the low-light scene, and to generate a dynamic heat distribution map based on the compensation result of the temperature difference loss.

[0143] Module 23 is used to construct a corresponding thermal gradient matrix based on the dynamic heat distribution map.

[0144] The segmentation module 24 is used to perform adaptive boundary scanning to identify object boundaries. Using a boundary point in the object boundary as a seed point, the first heat source region and the second heat source region are segmented on the spatial topology corresponding to the low-light scene by a region growing algorithm driven by the attenuation component in the thermal gradient matrix.

[0145] The marking module 25 is used to mark the depth level corresponding to the first heat source region and the second heat source region based on the intensity abrupt change characteristics of the abrupt change component and the exponential decay characteristics of the decay component in the thermal gradient matrix.

[0146] The fusion module 26 is used to perform corresponding adjustments on the first heat source region and the second heat source region based on the depth level marker, and to fuse the adjusted first heat source region and the second heat source region to generate an optimized display image.

[0147] Figure 2 The aforementioned video multi-layer precise display system based on line-of-sight segmentation can perform... Figure 1 The implementation principle and technical effects of the video multi-layer precise display method based on line-of-sight segmentation described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs operations in the video multi-layer precise display system based on line-of-sight segmentation in the above embodiments have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0148] In one possible design, Figure 2 The video multi-layer precise display system based on line-of-sight segmentation shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0149] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0150] The processing component 32 is used for the above Figure 1The embodiment describes a method for precise multi-layered video display based on line-of-sight segmentation.

[0151] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits, digital signal processors, digital signal processing devices, programmable logic devices, field-programmable gate arrays, controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0152] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk.

[0153] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0154] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0155] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0156] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0157] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a method for precise multi-layer video display based on line-of-sight segmentation.

[0158] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including several instructions to cause a computer device to execute the methods described in the various embodiments or some parts of the embodiments.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A video multi-level accurate presentation method based on range division, characterized in that, include: Convert the infrared thermal radiation data of the image to be processed in the target video in a low-light scene into the corresponding temperature mapping data; Wide dynamic range synthesis is performed on the temperature mapping data to compensate for the temperature difference loss between the dark and bright areas of the target in the low-light scene, and a dynamic heat distribution map is generated based on the compensation result of the temperature difference loss. Construct the corresponding thermal gradient matrix based on the dynamic heat distribution map; An adaptive boundary scan is performed to identify the object boundary. Using a boundary point in the object boundary as a seed point, a region growing algorithm driven by the attenuation component in the thermal gradient matrix is ​​used to segment the first heat source region and the second heat source region on the spatial topology corresponding to the low-light scene. Based on the intensity abrupt change characteristics of the abrupt change component and the exponential decay characteristics of the decay component in the thermal gradient matrix, the depth levels corresponding to the first heat source region and the second heat source region are marked. Based on the depth-level labeling, corresponding adjustments are performed on the first heat source region and the second heat source region respectively, and the adjusted first heat source region and second heat source region are merged to generate an optimized display image.

2. The method of claim 1, wherein, The infrared thermal radiation data of the image to be processed in the target video under low-light scenes is converted into corresponding temperature mapping data, including: The infrared thermal radiation data of each image to be processed in a low-light scene is analyzed into a corresponding first thermal response set, wherein each thermal response value in the first thermal response set corresponds to the spatial position of a detection unit in the sensor array. Based on multiple first thermal response sets, a superposition operation is performed on multiple thermal response values ​​at the same spatial location to generate a second thermal response set, wherein the second thermal response set includes numerical fluctuation characteristics of adjacent locations. Based on the numerical fluctuation characteristics of the adjacent positions, an adjustment threshold is generated to distinguish between valid signals and noise, and the cumulative thermal response values ​​in the second thermal response set that are lower than the adjustment threshold are filtered out. By using a preset correspondence rule between response intensity and temperature, each cumulative thermal response value is converted into a corresponding physical temperature value. All the physical temperature values ​​are then sorted according to the original arrangement order of the sensor array to generate a two-dimensional temperature distribution map as temperature mapping data.

3. The method of claim 1, wherein, A wide dynamic range synthesis is performed on the temperature mapping data to compensate for the temperature difference loss between the dark and bright areas of the target in the low-light scene, and a dynamic heat distribution map is generated based on the compensation result of the temperature difference loss, including: Acquire multiple sets of temperature mapping data for the same low-light scene, where each set of temperature mapping data is generated based on a different signal acquisition duration; The physical temperature values ​​with the same spatial location in each set of temperature mapping data are weighted and superimposed to generate a composite temperature value. The composite temperature value and sensitivity component of each spatial location within the dark area of ​​the target are superimposed, and the composite temperature value and saturation component of each spatial location within the bright area of ​​the target are superimposed to obtain the superimposed composite temperature value. A dynamic heat distribution map is generated by integrating the superimposed composite temperature values ​​corresponding to all spatial locations.

4. The method of claim 1, wherein, The thermal gradient matrix includes abrupt change components and attenuation components; Based on the dynamic heat distribution map, a corresponding thermal gradient matrix is ​​constructed, including: Calculate the temperature gradient between adjacent locations in the dynamic heat distribution map to generate an initial matrix; Taking each position in the initial matrix as the center position, based on the local mutation detection technology, the absolute difference between the gradient value of the center position and the average gradient value in the local neighborhood is calculated. When the absolute difference exceeds the preset mutation threshold, the gradient data of the center position is marked as a mutation component. Identify the center location of the heat source in the thermal gradient matrix and determine at least one analysis path along the direction of thermal radiation propagation; extract the gradient data at each location on each analysis path to form a gradient data sequence; when the gradient data sequence exhibits an exponential decay characteristic, mark the gradient data in the gradient data sequence as decay components. By combining the initial matrix, the mutation component, and the decay component, a thermal gradient matrix is ​​obtained.

5. The method of claim 1, wherein, An adaptive boundary scan is performed to identify object boundaries. Using a boundary point within the object boundaries as a seed point, a region growing algorithm driven by the decay component of the thermal gradient matrix is ​​used to segment a first heat source region and a second heat source region on the spatial topology corresponding to the low-light scene, including: The boundary determination threshold is adaptively determined by scanning the gradient distribution of the thermal gradient matrix. All positions in the thermal gradient matrix that exceed the boundary determination threshold are connected as boundary points to obtain the object boundary. For each boundary point in the object boundary, as a seed point, the difference between the maximum and minimum values ​​of the attenuation component is calculated based on the attenuation components of all positions in the neighborhood of the seed point using a region growing algorithm driven by the attenuation component, and is used as a dynamic similarity threshold. Traverse each adjacent position directly connected to the seed point in the spatial topology and calculate the difference between the attenuation component of the adjacent position and the attenuation component of the seed point. The adjacent positions with a difference less than or equal to the dynamic similarity threshold are included in the second heat source region, and the adjacent positions with a difference greater than the dynamic similarity threshold are included in the first heat source region.

6. The method of claim 1, wherein, Based on the intensity abrupt change characteristics of the abrupt change components and the exponential decay characteristics of the decay components in the thermal gradient matrix, the depth levels corresponding to the first heat source region and the second heat source region are marked, including: Traverse each location in the first heat source region, calculate the maximum absolute difference between the abrupt change component of the location and the abrupt change component of the adjacent location in the spatial topology, and use it as the change amplitude value. Based on the statistical distribution characteristics of all abrupt change components in the thermal gradient matrix, determine the intensity abrupt change threshold. The locations where the change amplitude value is greater than the intensity abruptness threshold are marked as the first depth level, and the locations where the change amplitude value is less than or equal to the intensity abruptness threshold are marked as the second depth level; Traverse each location in the second heat source region, obtain the attenuation component stored at the location in the thermal gradient matrix as the actual component value, and calculate the shortest topological distance from the location to the object boundary. Based on the shortest topological distance and the preset attenuation coefficient, the expected attenuation value is calculated, and a floating range is set with the expected attenuation value as the center as the expected attenuation range; The positions where the actual component values ​​are within the expected attenuation range are marked as the third depth level, and the positions where the actual component values ​​are not within the expected attenuation range are marked as the fourth depth level.

7. The method of claim 1, wherein, Based on depth-level labeling, corresponding adjustments are performed on the first heat source region and the second heat source region respectively, and the adjusted first heat source region and second heat source region are merged to generate an optimized display image, including: Based on the depth level marker, the saturation component of the pseudo-color image corresponding to the first heat source region is subjected to nonlinear stretching to generate a pseudo-color sub-image. Based on the depth level marker, a piecewise linear transformation is performed on the brightness component of the brightness image corresponding to the second heat source region to generate an initial brightness sub-image; Perform a Gaussian weighted average calculation on the initial brightness submap to generate the target brightness submap; The pseudo-color sub-image and the target brightness sub-image are merged through spatial registration and fusion operations to generate an optimized display image.

8. A video multi-level accurate presentation system based on range division, characterized in that, include: The conversion module is used to convert the infrared thermal radiation data of the image to be processed in the target video in a low-light scene into the corresponding temperature mapping data. The synthesis module is used to perform wide dynamic range synthesis on the temperature mapping data to compensate for the temperature difference loss between the dark area and the bright area of ​​the target in the low-light scene, and to generate a dynamic heat distribution map based on the compensation result of the temperature difference loss. The construction module is used to construct the corresponding thermal gradient matrix based on the dynamic heat distribution map; The segmentation module is used to perform adaptive boundary scanning to identify object boundaries. Using a boundary point in the object boundary as a seed point, the first heat source region and the second heat source region are segmented on the spatial topology corresponding to the low-light scene by a region growing algorithm driven by the attenuation component in the thermal gradient matrix. The marking module is used to mark the depth levels corresponding to the first heat source region and the second heat source region based on the intensity abruptness characteristics of the abruptness component and the exponential decay characteristics of the decay component in the thermal gradient matrix. The fusion module is used to perform corresponding adjustments on the first heat source region and the second heat source region based on the depth level label, and then merge the adjusted first heat source region and the second heat source region to generate an optimized display image.

9. A computing device, comprising: It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a method for precise multi-layer video display based on line-of-sight segmentation as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a method for precise multi-layer video display based on line-of-sight segmentation as described in any one of claims 1 to 7.

Images