A three-dimensional temperature field rendering method, system and storage medium

Abstract

The application discloses a three-dimensional temperature field rendering method, system and storage medium, and relates to the technical field of temperature three-dimensional visualization. The three-dimensional temperature field rendering method comprises the following steps: synchronously acquiring a two-dimensional thermal imaging image, a three-dimensional grid model of a to-be-measured object and camera internal parameters; extracting the contour of the to-be-measured object from the two-dimensional thermal imaging image, and updating the camera optical center parameter; performing optimal pose estimation on the three-dimensional grid model based on the extracted contour, so that the projection contour of the three-dimensional grid model is aligned with the contour in the thermal imaging image; generating dynamic UV coordinates, and establishing the mapping relationship between the model vertexes and the thermal imaging image pixels; sampling the temperature from the thermal imaging image according to the dynamic UV coordinates; and performing enhanced rendering on the sampled temperature by using an anisotropic heat diffusion algorithm, so as to generate a three-dimensional temperature field visualization model of the to-be-measured object. The whole three-dimensional rendering process can be driven by a single two-dimensional thermal imaging image, and the system complexity and implementation cost are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of temperature three-dimensional visualization, and in particular to a three-dimensional temperature field rendering method and system and a storage medium. BACKGROUND

[0002] In the industrial manufacturing process, especially in high-temperature forming processes such as die casting and forging, the temperature field distribution of the mold and the casting is a key factor affecting product quality and production safety. Temperature abnormalities are prone to cause defects such as porosity, cracks, and cold separation, so it is of great significance to accurately perceive and visualize the temperature field.

[0003] At present, an infrared thermal imager is generally used in industrial sites to obtain a two-dimensional thermal imaging image, but it can only provide a single perspective of the surface temperature distribution, and it is difficult to fully reflect the three-dimensional thermal state of complex structures (such as mold cavities and cooling channels). To achieve three-dimensional visualization, existing technologies mainly rely on the following schemes:

[0004] Surface temperature mapping technology: two-dimensional temperature data collected by a thermocouple or a thermal imager is mapped to the surface of a three-dimensional model through predefined UV (U-V Coordinate, texture) coordinates; this method relies on manual calibration, requires reconfiguration when changing production, has a long debugging period, and cannot handle occluded areas.

[0005] Volume rendering technology: based on CT (Computed Tomography, computed tomography) scanning or simulation-generated voxelized temperature data, light projection, and other methods are used for rendering; although it can show internal structures, it has high computational complexity and cannot meet real-time requirements.

[0006] Physical simulation rendering: solve the heat conduction equation by combining material thermal conductivity, specific heat capacity, and other parameters to generate a physically realistic temperature field; however, this method relies on high-performance GPUs (Graphics Processing Unit, graphics processing unit) and is difficult to deploy in embedded systems due to complex modeling.

[0007] Dynamic UV generation technology: automatically generate UV coordinates through pose estimation and deformation compensation algorithms to adapt to model changes; however, when there is occlusion or high vertex density, it still faces problems such as coordinate drift and memory overflow. SUMMARY

[0008] To solve the above problems, the present application provides a three-dimensional temperature field rendering method that can drive the entire three-dimensional rendering process through a single two-dimensional thermal imaging image, reducing system complexity and implementation cost. A corresponding three-dimensional temperature field rendering system is provided, which can be applied to different three-dimensional temperature field rendering methods.

[0009] The first technical solution adopted in this application is: providing a three-dimensional temperature field rendering method, including the following steps:

[0010] Simultaneously acquire two-dimensional thermal imaging images, a three-dimensional mesh model of the object under test, and camera internal parameters;

[0011] The contour of the object under test is extracted from the two-dimensional thermal imaging image, and the camera optical center parameters are updated based on the extracted contour.

[0012] The optimal pose of the 3D mesh model is estimated based on the extracted contour, so that the projected contour of the 3D mesh model is aligned with the contour in the thermal imaging image.

[0013] Based on the estimated optimal pose, dynamic UV coordinates are generated for the 3D mesh model, and a mapping relationship is established between the model vertices and the pixels of the thermal imaging image.

[0014] Temperature is sampled from the thermal imaging image based on the dynamic UV coordinates, and the sampling results are corrected.

[0015] An anisotropic thermal diffusion algorithm is used to enhance the rendering of the sampled temperature, generating a three-dimensional temperature field visualization model of the object under test.

[0016] In an optional embodiment, the step of extracting contours from a two-dimensional thermal imaging image includes:

[0017] The thermal imaging image is preprocessed, including distortion correction based on camera internal parameters, and image enhancement processing to improve contrast and suppress noise.

[0018] Edge features and temperature gradient features are extracted from the preprocessed image, respectively;

[0019] The edge features and temperature gradient features are weighted and fused according to configurable weights to generate enhanced fused features;

[0020] A morphological closing operation is performed on the fused features to connect the broken edges, and the largest connected region is extracted as the contour of the object under test.

[0021] In an optional embodiment, the weighted fusion is achieved by linearly superimposing weight coefficients assigned to edge features and temperature gradient features respectively, in order to improve the contour recognition capability for blurred edges and weak texture regions.

[0022] In an optional embodiment, the optimal pose estimation employs a staged strategy:

[0023] First, a coarse estimation is performed to calculate the similarity of various geometric features between the projected contour of the 3D model and the thermal imaging contour. The geometric feature similarity includes chamfer distance, contour region area similarity, and contour moment feature similarity. The comprehensive similarity is obtained by weighted summation, and the initial pose is selected from the candidate poses.

[0024] If the coarse estimation result does not reach the set confidence threshold, a fine estimation is performed. Starting from the initial pose, the optimal pose is calculated iteratively by optimizing a loss function that prioritizes contour alignment error and supplements pose smoothness.

[0025] In an optional embodiment, the step of generating dynamic UV coordinates includes:

[0026] Transform the vertices of the 3D mesh model from the model coordinate system to the camera coordinate system;

[0027] The image coordinates are obtained by projecting the vertices in the camera coordinate system onto the image coordinate system using perspective projection.

[0028] The image coordinates are normalized to a standard continuous range of zero to one to obtain the UV coordinates. Based on the normalization process, the UV coordinates are made independent of the specific resolution of the thermal imaging image.

[0029] In an optional embodiment, the temperature sampling employs a bicubic interpolation method, which achieves sub-pixel accuracy temperature sampling through a weighted average of surrounding pixels, in order to suppress the temperature step effect caused by the resolution limitation of the thermal imager.

[0030] In an optional embodiment, for vertices with invalid UV coordinates, the temperature data is supplemented by finding the vertex’s multiple nearest neighbor vertices in 3D space and taking a weighted average of their temperature values ​​with the inverse of the distance as the weight.

[0031] In an optional embodiment, the anisotropic thermal diffusion algorithm preferentially propagates temperature along the contour tangent direction of the object under test during the diffusion process, while suppressing non-physical diffusion behavior perpendicular to the contour direction.

[0032] The second technical solution adopted in this application is: providing a three-dimensional temperature field rendering system for implementing the three-dimensional temperature field rendering method as described in any of the preceding claims, including:

[0033] The data input module is used to acquire thermal imaging images, 3D models, and camera parameters;

[0034] A contour extraction module is used to extract object contours from the thermal imaging image;

[0035] The pose estimation module is used to estimate the optimal pose of the 3D model relative to the camera.

[0036] The UV mapping module is used to generate the dynamic UV coordinates of the 3D model.

[0037] The temperature processing module is used for temperature sampling, invalid value completion, and physical value correction.

[0038] The rendering output module is used to perform anisotropic heat diffusion calculations and output the rendered three-dimensional temperature field model.

[0039] The third technical solution adopted in this application is: providing a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the three-dimensional temperature field rendering method as described in any of the above.

[0040] Due to the adoption of the above technical solution, this application has at least one of the following beneficial effects compared with the prior art:

[0041] 1. The entire 3D rendering process can be driven by a single 2D thermal imaging image, reducing system complexity and implementation costs.

[0042] 2. Through dynamic optical center correction and normalized UV coordinates, one-click model change is achieved, shortening the production line debugging cycle and improving production efficiency.

[0043] 3. By using staged pose estimation and normalized UV coordinates, computational efficiency is optimized while ensuring accuracy.

[0044] 4. Temperature completion is achieved by using three-dimensional spatial K-nearest neighbor interpolation, which is more consistent with physical reality than simple two-dimensional image interpolation; in addition, an anisotropic heat diffusion algorithm is used to simulate the physical characteristics of heat preferentially propagating along the tangent direction of the mold contour. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0046] Figure 1 A flowchart illustrating a three-dimensional temperature field rendering method provided in an embodiment of this application;

[0047] Figure 2 A schematic diagram of a pose estimation process provided in an embodiment of this application;

[0048] Figure 3 A schematic diagram of the framework of a three-dimensional temperature field rendering system provided in an embodiment of this application;

[0049] Figure 4This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0051] The terms "first," "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0052] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] Existing technologies for rendering 3D temperature fields generally suffer from problems such as reliance on preset UV coordinates and manual calibration, the need for multi-view images or high-computational physical simulation, poor ability to restore temperature in occluded areas, and difficulty in balancing rendering accuracy and speed. These issues result in long system changeover and debugging cycles, high hardware costs, poor real-time performance, and an inability to adapt to the dynamic changes in complex industrial environments. This application achieves one-click model changeover without manual intervention and efficient, high-precision rendering that can be completed on a regular CPU (Central Processing Unit) by automatically extracting contours from a single thermal image and dynamically generating UV coordinates, and by employing a staged pose estimation and anisotropic thermal diffusion algorithm. It also reasonably restores the temperature distribution in occluded areas.

[0054] like Figure 1 As shown, Figure 1 A flowchart illustrating a three-dimensional temperature field rendering method provided in an embodiment of this application includes the following steps:

[0055] The system simultaneously acquires a two-dimensional thermal imaging image, a three-dimensional mesh model of the object under test, and camera internal parameters. The two-dimensional thermal imaging image is captured by an infrared thermal imager, which obtains a temperature distribution map of the current mold surface. The two-dimensional thermal imaging image contains a clear outline of the mold and temperature information. In this embodiment, the resolution of the infrared thermal imager is 320×420. In other embodiments, the resolution of the infrared thermal imager can be adaptively adjusted based on different objects under test, and no limitation is made in this regard.

[0056] The three-dimensional mesh model of the object under test is obtained based on the CAD (Computer-Aided Design) model of the object under test. The three-dimensional mesh model accurately reflects the geometry of the object under test. In this embodiment, the three-dimensional mesh model is a triangular mesh model.

[0057] The camera parameters are updated using the checkerboard calibration method to obtain the camera's intrinsic parameters, which include the camera intrinsic parameter matrix and distortion coefficients. The camera intrinsic parameter matrix includes the focal length and optical center coordinates. The distortion coefficients include radial distortion coefficients and tangential distortion coefficients.

[0058] The focal length in the camera intrinsic parameter matrix determines the scaling ratio of the image, while the optical center coordinates define the center point of the image plane; together, they constitute the core of the pinhole imaging model, enabling the system to establish a precise mathematical relationship between three-dimensional spatial points and two-dimensional image points; ensuring that the projected contour of the three-dimensional model can be accurately aligned with the real contour in the thermal image, avoiding overall misalignment caused by imaging scale or center offset, and significantly improving the spatial accuracy of temperature mapping.

[0059] Actual lenses inherently possess optical defects, leading to radial distortion (such as barrel and pincushion distortion) and tangential distortion (caused by the lens not being parallel to the sensor). Without correction, the contours of image edges and temperature points will be distorted, resulting in systematic errors in subsequent contour extraction and temperature sampling. By introducing radial and tangential distortion coefficients, the system can accurately correct distortion in the original thermal image during the preprocessing stage, restoring the distorted image to a geometry conforming to the ideal pinhole model, thus ensuring the accuracy of contour extraction.

[0060] Extracting the contour of an object from a two-dimensional thermal imaging image involves the following steps:

[0061] Preprocessing of thermal imaging images includes distortion correction based on camera intrinsic parameters. The distortion correction steps are described in detail below:

[0062] First, the pixel coordinates in the original thermal image are transformed into normalized coordinates by the inverse of the camera intrinsic parameter matrix. The normalization process eliminates the influence of imaging scales such as focal length. The image is then precisely corrected for distortion in the normalized coordinate system. The severity of distortion is positively correlated with the distance from the point to the optical center of the image; that is, the farther the point is from the image center, the stronger the distortion effect.

[0063] Distortion correction of normalized images includes radial distortion correction and tangential distortion correction. Radial distortion correction is used to correct the nonlinear scaling problem between the image center and the edges caused by lens curvature, such as common barrel distortion or pincushion distortion. During correction, the scaling factor is calculated based on the distance from the point to the optical center of the image, and the image is stretched or compressed in the radial direction to obtain the coordinates after the initial distortion.

[0064] Tangential distortion correction is used to compensate for trapezoidal or tilting deformation caused by non-parallel mounting of the lens and image sensor. The positional effect is evaluated using the same distance from the optical center of the image. The additional offset is calculated using the tangential distortion coefficient and added to the radially corrected coordinates to obtain the actual point coordinates after full correction.

[0065] The entire distortion correction process ensures the accurate restoration of the geometric structure in the thermal imaging image, eliminates shape distortion caused by lens defects, and provides a reliable geometric basis for the subsequent accurate extraction of edge features and temperature gradient features.

[0066] Image enhancement processing improves contrast and suppresses noise. Specifically, it addresses common problems in infrared images such as low resolution, blurred edges, low contrast, non-uniform stripes, and Gaussian noise by applying CLAHE (Contrast Limited Adaptive Histogram Equalization) to stretch low-contrast areas, especially enhancing details in dark areas and improving the overall contrast level of the image. At the same time, guided filtering combined with a non-local means algorithm effectively preserves the true edges of the image while suppressing non-uniform stripes and Gaussian noise, significantly improving the signal-to-noise ratio in weak texture areas.

[0067] Edge features and temperature gradient features are extracted from the preprocessed image. In this embodiment, the Canny Edge Detector is used to analyze the grayscale changes of the image to identify the physical boundaries of the object, thereby obtaining edge features. At the same time, the Sobel Operator is used to calculate the temperature gradient of each point in the image, capture the areas where the temperature changes drastically, and obtain the temperature gradient features. The areas where the temperature changes drastically usually correspond to the thermal boundaries near the contact between the mold and the casting or the cooling channel.

[0068] Edge features and temperature gradient features are weighted and fused according to configurable weights to generate enhanced fused features. Weighted fusion is achieved by linearly superimposing weight coefficients assigned to edge features and temperature gradient features respectively, in order to improve the contour recognition ability of blurred edges and weak texture regions.

[0069] Specifically, weighted fusion is achieved by assigning a weight coefficient (called the edge sharpness factor) to edge features and another weight coefficient (called the temperature contrast factor) to temperature gradient features, and then linearly superimposing the two. It should be noted that this application does not limit the specific values ​​of the edge sharpness factor and the temperature contrast factor. By adjusting the specific values ​​of the edge sharpness factor and the temperature contrast factor, the contour recognition effect can be optimized for blurred edges and weak texture areas. This not only improves the representation ability of these difficult-to-process areas, but also enhances the robustness of contour recognition under complex working conditions by combining traditional edge information and temperature features.

[0070] Morphological closing operations are performed on the fusion features to connect the broken edges. The morphological closing operation, through a process of first dilation and then erosion, can effectively connect broken edges caused by noise or low contrast, fill small gaps inside the contour, and thus enhance the continuity and integrity of the contour.

[0071] The system updates the camera's optical center parameters based on the extracted contours and extracts the largest connected component as the contour of the object to be measured. That is, all connected components are extracted from the image after morphological closure processing, and the connected component with the largest area is selected as the final mold contour. At the same time, the system records the geometric center coordinates of the contour in the current fused feature map and uses the coordinates to dynamically update the camera's optical center parameters, providing a more accurate reference benchmark for the subsequent projection alignment and coordinate mapping of the 3D model.

[0072] In another embodiment, a gradient-based optimization strategy is employed to improve the reliability of contour extraction. Specifically, gradients are first calculated on the fused feature map to highlight regions of dramatic change in the image, which typically correspond to the true boundaries of objects.

[0073] The magnitude of the gradient is used to quantify the intensity of change at each pixel. Based on this, a Gaussian difference operator is introduced to filter and enhance the gradient information; the Gaussian difference operator can effectively suppress high-frequency noise caused by casting spatter, surface oxidation, or environmental interference while preserving significant edges.

[0074] The final contour is determined by finding the closed boundary with the maximum response in the enhanced gradient map, that is, selecting the contour that maximizes the convolution result of the gradient magnitude and the difference of Gaussian operator; this significantly improves the stability and robustness of contour extraction in complex industrial environments, and can still output continuous, complete and accurate mold contours even in the presence of various visual interferences.

[0075] like Figure 2 As shown, Figure 2 This is a schematic diagram of the pose estimation process provided in an embodiment of this application; the optimal pose estimation of the 3D mesh model is performed based on the extracted contour, so that the projected contour of the 3D mesh model is aligned with the contour in the thermal imaging image; in this embodiment, the optimal pose estimation adopts a phased strategy, which includes a model preprocessing stage, a coarse estimation stage, and a fine estimation stage.

[0076] Model preprocessing stage: The system performs targeted optimization of the 3D mesh model based on the accuracy requirements of temperature rendering. Specifically, it identifies key areas sensitive to temperature changes, such as gates, overflow channels, and cooling channels where temperature gradients change drastically, as well as areas with sparse or unreasonable vertex distribution. For these areas, a bilateral splitting method is used to refine the local mesh to more accurately represent spatial temperature changes. At the same time, edge folding simplification technology is applied to non-critical areas to reduce the overall number of vertices and computational load of the model while ensuring the rendering accuracy of the core areas, thereby improving the efficiency of subsequent processing.

[0077] Bilateral splitting is a locally adaptive mesh subdivision technique. Its core idea is to subdivide only in areas requiring high precision, avoiding the generation of redundant cells in geometrically smooth regions, thus saving computational resources. Compared to global uniform subdivision, this method is more efficient. Compared to global reconstruction methods, it starts from the original triangle and gradually completes the subdivision through an iterative process of "splitting the long side—subdivision—refinement," achieving simplicity while preserving the key geometric features of the original model to the greatest extent and preventing shape distortion caused by reconstruction. Furthermore, this method helps reduce the generation of elongated triangles, improving mesh quality.

[0078] The implementation of the bilateral separatist law consists of three steps:

[0079] 1. Initial splitting and division: For the target triangle, find the two longest sides among its three sides, insert dividing points on the sides according to the preset splitting interval, and use these points to divide the original triangle into multiple smaller triangles and quadrilaterals.

[0080] 2. Repeat the splitting process: Check the side length of each generated small shape. If there is a side length that is greater than the preset splitting interval, repeat the above splitting process for that shape until all side lengths do not exceed the preset splitting interval.

[0081] 3. Final processing: Keep all small triangles; for the remaining quadrilaterals, divide them in half by connecting the diagonals, delete the original faces and replace them with newly generated triangles to complete the local encryption.

[0082] Edge folding simplification is the reverse operation, which simplifies the mesh in non-critical areas by merging vertices and folding edges, and will not be elaborated on here.

[0083] The similarity of various geometric features between the projected contour of the 3D model and the thermal imaging contour is calculated by coarse estimation. The geometric feature similarity includes chamfer distance similarity, contour region area similarity, and contour moment feature similarity. Chamfer distance similarity measures the degree of geometric matching between the edge points of the two contours, contour region area similarity reflects the overall size consistency, and contour moment feature similarity evaluates whether the global shape distribution of the contour is consistent.

[0084] The overall similarity is obtained by weighted summation, and the initial pose, i.e., the coarsely estimated pose, is selected from the candidate poses. If the overall similarity is greater than a set confidence threshold, the coarsely estimated pose is directly output, and the fine estimation stage is skipped. In this embodiment, the confidence threshold is set to 95%. In other embodiments, the confidence threshold can be selected differently, and there is no limitation on it.

[0085] If the coarse estimation result does not reach the set confidence threshold (i.e., the overall similarity is less than or equal to the set confidence threshold), then a fine estimation is performed. The steps of the fine estimation stage are described in detail below:

[0086] Starting with the initial pose, i.e. the coarsely estimated pose, we optimize a loss function that prioritizes contour alignment error and supplements pose smoothness. The loss function includes chamfer distance, regularization coefficient, and the square norm of pose change. The chamfer distance directly quantifies the edge difference between the projected contour and the true contour. The regularization coefficient is used to balance contour alignment error and pose change smoothness. The square norm of pose change serves as a smoothness constraint to prevent severe jitter caused by noise interference during the optimization process.

[0087] The loss function is minimized by an iterative optimization algorithm (such as gradient descent), and the final optimal pose is output when the loss converges or the maximum number of iterations is reached.

[0088] Based on the estimated optimal pose, dynamic UV coordinates are generated for the 3D mesh model. The steps for generating dynamic UV coordinates include:

[0089] The vertices of the 3D mesh model are transformed from the model coordinate system to the camera coordinate system. Based on the previously obtained estimated optimal pose, the corresponding rotation matrix and translation vector are calculated. Using this transformation parameter, each vertex in the 3D model is transformed from its original model coordinate system to the camera coordinate system. The transformation formula is: the vertex coordinates in the camera coordinate system are equal to the transpose of the rotation matrix multiplied by (model vertex coordinates minus the translation vector). Here, the translation vector represents the positional offset of the origin of the model coordinate system relative to the origin of the camera coordinate system, while the transpose of the rotation matrix is ​​used to eliminate the difference in direction between the two coordinate systems, completing the alignment of spatial poses. After the transformation, the set of visible vertices located within the camera's view frustum and not occluded is further selected, and only these vertices are subjected to subsequent calculations, reducing unnecessary computation and improving processing efficiency.

[0090] The vertex in the camera coordinate system is projected onto the image coordinate system by perspective projection to obtain the image coordinates. After obtaining the vertex coordinates in the camera coordinate system, the perspective projection model is used to map them onto the two-dimensional image plane. In this embodiment, the perspective projection method simulates the pinhole imaging model to achieve a geometric mapping from three dimensions to two dimensions.

[0091] When a 3D model contains a large number of vertices, a block-based processing strategy can be adopted to avoid excessive memory usage or even overflow caused by processing all at once. This involves dividing the model vertices into multiple blocks according to their spatial location or index order, and performing coordinate transformation and projection calculations on each block. This method significantly reduces the peak memory usage of a single operation and improves the system's stability on low-configuration devices.

[0092] Image coordinates are normalized to a standard continuous interval of zero to one to obtain UV coordinates. After obtaining the image coordinates of all visible vertices, the system calculates the actual range of these coordinates, namely the minimum and maximum values ​​in the u direction and the v direction. Subsequently, the image coordinates (u, v) of each vertex are linearly mapped to the standard continuous interval [0,1] to obtain normalized UV coordinates. Normalization is a conventional technique for those skilled in the art and will not be elaborated upon here.

[0093] The normalization process makes the UV coordinates independent of the specific resolution of the thermal imaging image. Furthermore, regardless of the size of the mold, the normalized UV coordinates are always mapped to a unified [0,1] space, eliminating the need for recalibration or parameter adjustment for new molds. This significantly improves the system's adaptability and ease of use in production change scenarios.

[0094] Establish a mapping relationship between model vertices and thermal imaging image pixels to accurately project the temperature information of the two-dimensional thermal image onto the surface of the three-dimensional model.

[0095] Temperature is sampled from thermal images based on dynamic UV coordinates, and the sampling results are corrected. Bicubic interpolation is used for temperature sampling, achieving sub-pixel accuracy through a weighted average of surrounding pixels to suppress the temperature step effect caused by the thermal imager's resolution limitations. The temperature sampling steps are described in detail below:

[0096] The corresponding heatmap pixel coordinates are calculated based on the normalized UV coordinates. Through the above transformation, a spatial mapping relationship between the vertices of the 3D model and the heatmap pixels is established.

[0097] Bicubic interpolation sampling is performed using the Keys cubic convolution kernel function to obtain the raw temperature value for each vertex. This process involves weighting the pixel values ​​within a 3x3 neighborhood on the heatmap, with the weights determined by the relative position to the current point.

[0098] The temperature value at the location corresponding to the vertex of the 3D model is obtained from the thermal imaging image. Due to the limited resolution of the thermal imager, the position of the model vertex projected onto the image often does not fall exactly at the center of a pixel, but is located at a sub-pixel position between pixels; directly taking the nearest pixel will result in coarse temperature information, with a "stepped" distribution.

[0099] A weighted average-based interpolation algorithm is adopted, which comprehensively considers the temperature values ​​of multiple neighboring pixels around the target point and assigns different weights to them according to their distance from the target point: the closer the pixel is, the greater its influence, and the farther away the pixel is, the smaller its influence. This weight relationship is defined by a smooth mathematical function that changes continuously within a local range to ensure that the interpolation result transitions naturally.

[0100] Even with low resolution thermal imagers, sub-pixel-level temperature sampling accuracy can be achieved, effectively mitigating temperature jumps caused by pixelation. This results in a smoother, more continuous temperature field mapped onto the 3D model, more realistically reflecting the temperature distribution on the object's surface.

[0101] A vertex whose UV coordinates are invalid is one whose normalized UV coordinates exceed the range of [0,1]. This indicates that the vertex is outside the camera's field of view or is occluded by other objects. Therefore, its UV coordinates are considered invalid and marked as an occluded point.

[0102] The temperature data is supplemented by finding the nearest neighbor vertices of a given vertex in 3D space and then weighting their temperature values ​​using the inverse of the distance. Specifically, for these occluded points, firstly, the K nearest vertices to the occluded point are found in the 3D mesh model; these vertices are called the K nearest neighbors of the occluded point. Then, the actual temperature of the occluded point is estimated using the temperature values ​​of these nearest neighbors.

[0103] During the calculation, the contribution of each nearest neighbor vertex to the temperature of the shading point is weighted according to their distance from the shading point. The closer the vertex is, the greater its temperature value has on the temperature estimation of the shading point; conversely, the more distant the vertex is, the smaller its influence.

[0104] The temperature values ​​of occluded points are filled by using a weighted average of the temperatures of spatially adjacent vertices. In this way, even if some areas cannot be directly observed due to occlusion, a reasonable temperature distribution can be calculated using information from the surrounding environment, resulting in a more complete, continuous, and realistic three-dimensional temperature field that more closely resembles the actual heat distribution.

[0105] In some mold scenarios with blackbody radiation sources, physical temperature value correction can be performed to improve the accuracy of temperature measurement. Because the raw temperature data acquired by the thermal imager may be affected by factors such as emissivity, environmental reflection, and lens attenuation, resulting in a systematic deviation from the true physical temperature, correction is necessary.

[0106] This step involves collecting multiple calibration points on a blackbody radiation source at a known temperature to establish a correspondence between the original measured temperature and the actual physical temperature, and then fitting a calibration model based on this relationship. Using this model to adjust the original temperature values ​​of all vertices effectively compensates for system errors, making the final output temperature value closer to the true temperature distribution of the object's surface.

[0107] In complex structures such as molds, heat does not spread uniformly in all directions. For example, heat diffuses more easily along the tangential direction of the mold surface (i.e., the direction of the surface extension), while it diffuses more slowly in the direction perpendicular to the surface. The traditional isotropic diffusion model incorrectly assumes that heat "overflows" uniformly in all directions, resulting in a distorted temperature distribution.

[0108] An anisotropic thermal diffusion algorithm is used to enhance the rendering of the sampled temperature, generating a three-dimensional temperature field visualization model of the object under test. During the diffusion process, the anisotropic thermal diffusion algorithm preferentially propagates the temperature along the contour tangent direction of the object under test, while suppressing non-physical diffusion behavior perpendicular to the contour direction.

[0109] An anisotropic heat diffusion algorithm simulates the physical property of heat preferentially propagating along the surface contour of a 3D model. This algorithm enhances heat conduction along the surface tangential direction by analyzing the model's geometry, while suppressing non-physical diffusion perpendicular to the surface, making the evolution of the temperature field more consistent with real heat flow paths. By adaptively adjusting the calculation step size, it automatically reduces the iteration step size in regions of drastic temperature changes, ensuring computational stability and avoiding spurious temperature oscillations or divergences.

[0110] In order to transform the calculated physical temperature values ​​into an intuitive visual effect, the system first counts the highest and lowest temperatures among all valid vertices (i.e., non-occluded vertices with acquired temperature values) to determine the dynamic temperature range of the current scene.

[0111] Then, the temperature value of each vertex is linearly mapped based on this range, normalized to the standard interval [0,1]. Subsequently, using a preset thermal color map (such as blue-green-red or black-red-yellow), the normalized temperature values ​​are converted into corresponding RGB (Red Green Blue) color values. Lower temperature areas are displayed as cool colors (such as blue), and higher temperature areas are displayed as warm colors (such as red or yellow), achieving an intuitive mapping between "temperature and color".

[0112] This application also provides a three-dimensional temperature field rendering system for implementing the three-dimensional temperature field rendering method of any of the above embodiments, such as... Figure 3 As shown, Figure 3 A schematic diagram of the framework of a three-dimensional temperature field rendering system provided in an embodiment of this application includes:

[0113] The data input module is used to acquire thermal imaging images, 3D models, and camera parameters;

[0114] A contour extraction module is used to extract object contours from the thermal imaging image;

[0115] The pose estimation module is used to estimate the optimal pose of the 3D model relative to the camera.

[0116] The UV mapping module is used to generate the dynamic UV coordinates of the 3D model.

[0117] The temperature processing module is used for temperature sampling, invalid value completion, and physical value correction.

[0118] The rendering output module is used to perform anisotropic heat diffusion calculations and output the rendered three-dimensional temperature field model.

[0119] The methods described in the above embodiments can be implemented as computer programs; therefore, this application proposes a computer-readable storage medium. Please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium stores program data that can be executed by a processor to implement the steps of any embodiment of the three-dimensional temperature field rendering method described above.

[0120] In this embodiment, the computer-readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium capable of storing program data. Alternatively, it can be a server storing the program data, which can send the stored program data to other devices for execution or run the stored program data itself.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A three-dimensional temperature field rendering method, characterized in that, Includes the following steps: Simultaneously acquire two-dimensional thermal imaging images, a three-dimensional mesh model of the object under test, and camera internal parameters; The contour of the object under test is extracted from the two-dimensional thermal imaging image, and the camera optical center parameters are updated based on the extracted contour. The optimal pose of the 3D mesh model is estimated based on the extracted contour, so that the projected contour of the 3D mesh model is aligned with the contour in the thermal imaging image. Based on the estimated optimal pose, dynamic UV coordinates are generated for the 3D mesh model, and a mapping relationship is established between the model vertices and the pixels of the thermal imaging image. Temperature is sampled from the thermal imaging image based on the dynamic UV coordinates, and the sampling results are corrected. An anisotropic thermal diffusion algorithm is used to enhance the rendering of the sampled temperature, generating a three-dimensional temperature field visualization model of the object under test; The steps for extracting contours from a two-dimensional thermal imaging image include: The thermal imaging image is preprocessed, including distortion correction based on camera internal parameters, and image enhancement processing to improve contrast and suppress noise. Edge features and temperature gradient features are extracted from the preprocessed image, respectively; The edge features and temperature gradient features are weighted and fused according to configurable weights to generate enhanced fused features; A morphological closing operation is performed on the fused features to connect the broken edges, and the largest connected region is extracted as the contour of the object under test.

2. The three-dimensional temperature field rendering method according to claim 1, characterized in that, The weighted fusion is achieved by linearly superimposing weight coefficients assigned to edge features and temperature gradient features respectively, in order to improve the contour recognition capability of blurred edges and weak texture regions.

3. The three-dimensional temperature field rendering method according to claim 1, characterized in that, The optimal pose estimation adopts a phased strategy: First, a coarse estimation is performed to calculate the similarity of various geometric features between the projected contour of the 3D model and the thermal imaging contour. The geometric feature similarity includes chamfer distance, contour region area similarity, and contour moment feature similarity. The comprehensive similarity is obtained by weighted summation, and the initial pose is selected from the candidate poses. If the coarse estimation result does not reach the set confidence threshold, a fine estimation is performed. Starting from the initial pose, the optimal pose is calculated iteratively by optimizing a loss function that prioritizes contour alignment error and supplements pose smoothness.

4. The three-dimensional temperature field rendering method according to claim 1, characterized in that, The step of generating dynamic UV coordinates includes: Transform the vertices of the 3D mesh model from the model coordinate system to the camera coordinate system; The image coordinates are obtained by projecting the vertices in the camera coordinate system onto the image coordinate system using perspective projection. The image coordinates are normalized to a standard continuous range of zero to one to obtain the UV coordinates. Based on the normalization process, the UV coordinates are made independent of the specific resolution of the thermal imaging image.

5. The three-dimensional temperature field rendering method according to claim 1, characterized in that, The temperature sampling employs a bicubic interpolation method, which achieves sub-pixel accuracy temperature sampling through a weighted average of surrounding pixels, in order to suppress the temperature step effect caused by the resolution limitations of the thermal imager.

6. The three-dimensional temperature field rendering method according to claim 5, characterized in that, For vertices with invalid UV coordinates, the temperature data is supplemented by finding the vertex's multiple nearest neighbor vertices in 3D space and taking a weighted average of their temperature values ​​with the reciprocal of the distance as the weight.

7. The three-dimensional temperature field rendering method according to claim 1, characterized in that, The anisotropic thermal diffusion algorithm preferentially propagates temperature along the contour tangent of the object under test during the diffusion process, while suppressing non-physical diffusion behavior perpendicular to the contour direction.

8. A three-dimensional temperature field rendering system for implementing the three-dimensional temperature field rendering method as described in any one of claims 1-7, characterized in that, include: The data input module is used to acquire thermal imaging images, 3D models, and camera parameters; A contour extraction module is used to extract object contours from the thermal imaging image; The pose estimation module is used to estimate the optimal pose of the 3D model relative to the camera. The UV mapping module is used to generate the dynamic UV coordinates of the 3D model. The temperature processing module is used for temperature sampling, invalid value completion, and physical value correction. The rendering output module is used to perform anisotropic heat diffusion calculations and output the rendered three-dimensional temperature field model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the three-dimensional temperature field rendering method as described in any one of claims 1-7.

Images