A mapping method for three-dimensional temperature field images

Abstract

This invention discloses a method for mapping three-dimensional temperature field images. The method involves sampling temperature field images from different viewpoints; dividing the surface of a three-dimensional digital model into several small surface regions based on the total number of sampled temperature field images, with each small surface region corresponding to a temperature field image from a specific viewpoint; assigning a temperature field image to each side of the three-dimensional digital model; and establishing a mapping relationship between each grid on each side and its corresponding temperature field image, thus mapping the temperature field image onto the surface of the three-dimensional digital model. This invention provides a method for mapping three-dimensional temperature field images to address problems in existing technologies such as insufficient continuity in mapping relationships, unsmooth transitions in temperature fields, and susceptibility to erroneous mapping relationships. It aims to achieve a more continuous mapped temperature field, improving observability and mapping accuracy.

Description

Technical Field

[0001] This invention relates to the field of thermal mapping, and more specifically to a method for mapping three-dimensional temperature field images. Background Technology

[0002] Based on the target 3D digital surface model and the 2D temperature field distribution image, a thermal mapping of the 3D model can be performed, that is, the temperature distribution image is pasted on the digital surface model. This process is called temperature field mapping, which establishes a mapping relationship between temperature distribution information and the 3D digital surface model.

[0003] Establishing a mapping relationship between each mesh of a 3D object and a temperature field image requires first finding the corresponding temperature field image. In existing technologies, temperature field mapping algorithms commonly find a temperature field image approximately perpendicular to the normal direction of each mesh. This reduces mapping errors caused by viewpoint mistakes. However, this approach requires a large number of temperature field images from multiple views, and sometimes a single mesh matches multiple temperature field images. This necessitates extracting texture blocks from these multiple images, resulting in a discontinuous mapping relationship and uneven temperature field transitions. Furthermore, as... Figure 1 As shown, suppose there is a mesh V1V2V3 in the 3D model, whose projection on the temperature field image is p1p2p3, and a point V on the opposite surface of this mesh. a The projection onto the temperature field image is p. a But actually p a The temperature field pixel of the point will not appear in V a The location is determined by point V within the grid V1V2V3. m Because the image is occluded, an incorrect mapping will occur. The example above shows that determining the mapping between a 3D object model and a 2D temperature field image solely through projection will result in an incorrect correspondence where multiple 3D vertices are projected to a single pixel location. Summary of the Invention

[0004] This invention provides a mapping method for three-dimensional temperature field images to solve problems such as insufficient continuity of mapping relationships, unsmooth transitions in temperature fields, and susceptibility to erroneous mapping relationships in existing technologies. The goal is to make the mapped temperature field more continuous, improve observability, and enhance mapping accuracy.

[0005] This invention is achieved through the following technical solution:

[0006] A method for mapping a three-dimensional temperature field image includes the following steps:

[0007] Sample temperature field images from different perspectives;

[0008] Based on the total number of sampled temperature field images, the surface of the 3D digital model is divided into several small surface regions, each of which corresponds to a temperature field image from a specific viewpoint.

[0009] Assign a temperature field image to the surface of each small region of the 3D digital model;

[0010] Establish a mapping relationship between each grid on the surface of each small region and the corresponding temperature field image, and map the temperature field image onto the surface of the three-dimensional digital model.

[0011] To address the problems of insufficient continuity in mapping relationships, uneven temperature field transitions, and susceptibility to erroneous mapping relationships in existing technologies, this invention proposes a mapping method for three-dimensional temperature field images. Considering that the visibility of texture images varies across different viewpoints for a three-dimensional model, this application adopts a viewpoint-based method for dividing the surface of a three-dimensional object. Based on the total number of sampled temperature field images, the surface of the three-dimensional model is divided into several small surface regions. Each small surface region corresponds to the temperature field image under that viewpoint. This method can ensure the visibility of the small surface region while avoiding multi-region remapping of the temperature field image, and makes the mapped temperature field more continuous and observable.

[0012] Furthermore, it also includes verifying the temperature field image assigned to a small area of ​​the surface of the 3D digital model. The verification methods include:

[0013] Given a temperature field image, determine the division angle of the surface of the small region corresponding to the temperature field image as 2β;

[0014] Using the direction of the plane normal vector of the temperature field image as the center direction, calculate the angle α between the normal vector of each grid on the side of the three-dimensional digital model and the center direction;

[0015] Compare α and β: If α ≤ β, then the temperature field mapping pixel block corresponding to the grid is considered to exist in the temperature field image.

[0016] Furthermore, methods for establishing the mapping relationship between each grid on each side and the corresponding temperature field image include:

[0017] Based on the camera pose parameters used in the temperature field image, the 3D vertex coordinates of the corresponding region in the temperature field image are converted from the world coordinate system to the camera coordinate system;

[0018] Based on the camera intrinsic parameters, the temperature field coordinates p of the 3D vertex are calculated using the following formula. ij (u,v):

[0019]

[0020] In the formula, [X,Y,Z] are the coordinates of the 3D vertex in the camera coordinate system, w is the width of the image, h is the height of the image, and f is the height of the image. x ,f y ,c x ,c y This refers to the camera's internal parameters.

[0021] After determining the temperature field image corresponding to each grid, specific pixel coordinates are assigned to the vertices of the grid. First, based on the camera pose parameters determined by calibration, the vertex coordinates of the corresponding region in the temperature field image are converted from the world coordinate system to the camera coordinate system. Then, using intrinsic parameters such as the camera's focal length and image size, the temperature field coordinates of the 3D vertex are calculated. A pixel block is determined by the 3D vertex, and the corresponding pixel block is pasted onto the grid, thus realizing the temperature mapping of the 3D object model.

[0022] Furthermore, the method for mapping the temperature field image onto the surface of the 3D digital model includes the following mapping steps:

[0023] Write the point, surface, and grouping information of the 3D digital model into the obj file;

[0024] Specify the temperature field image of the corresponding group in the MTL material description file;

[0025] Load the prepared OBJ file, MTL material description file, and temperature field images from various viewpoints into the 3D display software.

[0026] Furthermore, during the mapping process of the temperature field image, the target contour boundary is adjusted using the following method:

[0027] Determine the outline of the object;

[0028] For two adjacent temperature field images, the contour lines and mapping boundaries of both images are mapped onto one of the images. The contour lines that intersect with the mapping boundary are then identified to determine if there is any misalignment.

[0029] If the judgment result is misalignment, adjust the temperature field mapping boundary to correct the misalignment.

[0030] In existing technologies, when the same contour line is mapped onto a 3D model through temperature field images from different angles, misalignment may occur. This is mainly caused by camera pose estimation errors and geometric errors in the 3D model. The temperature field mapping relationship calculated by the pinhole imaging model has certain deviations due to these two errors. The uncertainty of the errors means that the same contour line in different temperature field images may be mapped to different positions in the 3D model.

[0031] To address this, this solution proposes the following approach: First, detect the internal contour lines of the object. For two adjacent temperature field images, map their contour lines and mapping boundaries onto one of the images. Identify the contour lines that intersect with the mapping boundary; the locations where the distance between the two intersection points is larger are the positions requiring boundary adjustment. Finally, find the intersection points of contour lines from different temperature field images; these are the target positions for boundary adjustment. Determine if there is misalignment. If misalignment is found, adjust the temperature field mapping boundary to correct it.

[0032] Furthermore, methods for determining the outline of an object include:

[0033] The Canny operator is used to perform edge detection on the temperature field image to obtain the approximate contour information in the image.

[0034] Large-scale edge block information in temperature field images is obtained using a superpixel segmentation algorithm.

[0035] The edge block information and the general outline information are combined;

[0036] Remove the approximate outline information that is located within the same region segmentation as the edge block information, and use the retained boundaries as the outline lines of each region.

[0037] Identifying the main contour lines within an object is fundamental to boundary adjustment. Only by finding these main contour lines can the distances between them in images from different temperature fields be calculated, thereby identifying the locations where boundary adjustments are needed and the target locations for boundary adjustments. Since minimizing the temperature field cost function in existing techniques already allows the mapped boundary to bypass complex regions of the temperature field, the mapped boundary obtained using existing techniques generally does not pass through fine contour lines. Therefore, this approach only needs to detect contour lines in larger regions, such as the contour lines of hair, face, neck, clothing, etc. This approach combines the Canny operator and superpixel segmentation algorithm to obtain the contour lines in each temperature field image.

[0038] Furthermore, during the adjustment of the target contour boundary, the method for determining whether the contour line is misaligned is as follows: extract the intersection points of the contour line and the mapped boundary, and determine the distance between any two adjacent intersection points. If the distance between any two intersection points is greater than a set threshold, then it is considered misaligned. The threshold value is adaptively set according to the actual application conditions.

[0039] Furthermore, methods for obtaining large-scale edge patch information in temperature field images using superpixel segmentation algorithms include:

[0040] The entire image is covered with square blocks of fixed size that overlap each other. The square blocks are then stitched together so that each pixel is assigned to one of the square blocks.

[0041] Extract two adjacent pixels belonging to different square blocks, and reassemble the square blocks using a concatenation penalty function to make the boundaries of the square blocks match the boundaries of the two adjacent pixels as closely as possible, thus obtaining edge block information.

[0042] Superpixel segmentation is a graph-cut optimization-based segmentation algorithm. It first covers the entire image with fixed-size, overlapping square blocks. Each pixel may be covered by multiple square blocks. Then, these square blocks are stitched together, or in other words, each pixel is assigned one of all the square blocks. If two adjacent pixels are assigned to the same square block, there is no penalty function. However, if they belong to different square blocks, a stitching penalty function exists, which is inversely proportional to the grayscale difference between the two pixels. In other words, the above process re-stitches the square blocks so that their boundaries match the boundaries in the image as closely as possible, because this minimizes the stitching cost.

[0043] Furthermore, the process of mapping the temperature field image onto the surface of the 3D digital model includes the following texture generation methods:

[0044] In a three-dimensional digital model, a set of meshes that are contiguous and located in the same temperature field image is defined as a patch.

[0045] Extract the range of this set from the corresponding temperature field image to form a new smaller image;

[0046] Based on the position of the set in the temperature field image, calculate the image coordinates of each vertex contained in the set in the smaller image;

[0047] Create a new texture map, put the small images corresponding to the set into the new texture map for binning, calculate the position of the set in the new image, obtain the texture coordinates of each vertex in the texture map, and generate the texture map.

[0048] This solution specifically defines the method for generating textures.

[0049] Furthermore, it also includes processing the seams between adjacent textures, the processing method being:

[0050] Optimize the objective function J1 using the following formula:

[0051]

[0052] Optimize the objective function J2 using the following formula:

[0053]

[0054] In the formula, f vleft f vrightThese represent the colors of the same vertex v in two different patches; g vleft g vright These are the color adjustment values ​​for the same vertex v in two different patches, on the left and right sides, respectively; g vi g vj These are two adjacent vertices within the same patch;

[0055] The objective function of the texture is optimized using matrix J, where J = J1 + J2.

[0056] This solution specifies the exact methods for handling texture seams.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. The present invention provides a mapping method for a three-dimensional temperature field image, which can ensure the visibility of the surface of the small area while avoiding multi-region remapping of the temperature field image, and makes the mapped temperature field more continuous and observable.

[0059] 2. The present invention provides a mapping method for a three-dimensional temperature field image, which can divide the surface region according to the number of sampled images, and can also achieve a better actual mapping effect by changing the sampling frequency of the temperature field image.

[0060] 3. The present invention provides a mapping method for three-dimensional temperature field images, which optimizes temperature field mapping through contour adjustment to solve the problem in the prior art that the same contour line in different temperature field images may be mapped to different positions in the three-dimensional model.

[0061] 4. The present invention provides a mapping method for a three-dimensional temperature field image, which solves the defect of texture seams by means of seam processing. Attached Figure Description

[0062] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a schematic diagram of texture visibility in existing technologies;

[0064] Figure 2 This is a flowchart illustrating a specific embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of the division of a three-dimensional surface model from different perspectives in a specific embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of 2D-3D point mapping in a specific embodiment of the present invention;

[0067] Figure 5 This is a schematic diagram of the temperature field mapping optimization process in a specific embodiment of the present invention;

[0068] Figure 6 This is the original temperature field image in a specific embodiment of the present invention;

[0069] Figure 7 This is a schematic diagram of edge detection in a specific embodiment of the present invention;

[0070] Figure 8 This is a schematic diagram of superpixel segmentation in a specific embodiment of the present invention;

[0071] Figure 9 This is a schematic diagram of contour line extraction in a specific embodiment of the present invention;

[0072] Figure 10 This is a schematic diagram of the temperature field mapping results in a specific embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are for explaining the invention only and are not intended to limit the invention. In the description of this application, it should be understood that the terms "front," "rear," "left," "right," "upper," "lower," "vertical," "horizontal," "high," "low," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the scope of protection of this application.

[0074] Example:

[0075] A mapping method for three-dimensional temperature field images, such as Figure 1 As shown, it includes the following steps:

[0076] (1) Sample temperature field images from different perspectives;

[0077] (2) Based on the total number of sampled temperature field images, the surface of the three-dimensional digital model is divided into several small regions, and each small region corresponds to a temperature field image from a certain perspective.

[0078] Assign a temperature field image to the surface of each small region of the 3D digital model;

[0079] The temperature field image assigned to a small area of ​​the surface of the 3D digital model was validated. The validation methods included:

[0080] Given a temperature field image, determine the division angle of the surface of the small region corresponding to the temperature field image as 2β;

[0081] Using the direction of the plane normal vector of the temperature field image as the center direction (the plane normal vector of the image is the direction of the optical center of the camera, i.e. the Z-axis direction of the camera coordinate system), calculate the angle α between the normal vector of each grid on the side of the three-dimensional digital model and the center direction.

[0082] Compare α and β: If α ≤ β, then the temperature field mapping pixel block corresponding to the grid is considered to exist in the temperature field image.

[0083] (3) Establish the mapping relationship between each grid on each side and the corresponding temperature field image:

[0084] Based on the camera pose parameters used in the temperature field image, the 3D vertex coordinates of the corresponding region in the temperature field image are converted from the world coordinate system to the camera coordinate system;

[0085] The definition of a 3D vertex is as follows: A 3D model is composed of individual coordinate points, which are connected to form a 3D surface. A 3D vertex represents each 3D point that makes up a 3D surface.

[0086] Based on the camera intrinsic parameters, the temperature field coordinates p of the 3D vertex are calculated using the following formula. ij (u,v):

[0087]

[0088] In the formula, [X,Y,Z] are the coordinates of the 3D vertex in the camera coordinate system, w is the width of the image, h is the height of the image, and f is the height of the image. x ,f y ,c x ,c y This refers to the camera's internal parameters.

[0089] (4) Map the temperature field image onto the surface of the three-dimensional digital model.

[0090] During the mapping process, the target contour boundary is adjusted using the following method:

[0091] Determine the outline of an object:

[0092] The Canny operator is used to perform edge detection on the temperature field image to obtain the approximate contour information in the image.

[0093] Large-scale edge patch information in the temperature field image is obtained using a superpixel segmentation algorithm:

[0094] The entire image is covered with square blocks of fixed size that overlap each other. The square blocks are then stitched together so that each pixel is assigned to one of the square blocks.

[0095] Extract two adjacent pixels belonging to different square blocks, and reassemble the square blocks using a concatenation penalty function to make the boundaries of the square blocks match the boundaries of the two adjacent pixels, thus obtaining edge block information.

[0096] By combining the edge block information and the general outline information, the general outline information that is located in the same region segment as the edge block information is removed, and the retained general outline information is used as the outline of each region.

[0097] For two adjacent temperature field images, the contour lines and mapping boundaries of both images are mapped onto one of the images. The contour lines that intersect with the mapping boundary are found, and it is determined whether there is a misalignment: the intersection points of the contour lines and the mapping boundary are extracted, and the distance between any two adjacent intersection points is determined. If the distance between any intersection points is greater than a set threshold, it is considered to be misaligned.

[0098] If the judgment result is misalignment, adjust the temperature field mapping boundary to correct the misalignment.

[0099] During the mapping process, textures are generated using the following method:

[0100] In a three-dimensional digital model, a set of meshes that are contiguous and located in the same temperature field image is defined as a patch.

[0101] Extract the range of this set from the corresponding temperature field image to form a new smaller image;

[0102] Based on the position of the set in the temperature field image, calculate the image coordinates of each vertex contained in the set in the smaller image;

[0103] Create a new texture map, put the small images corresponding to the set into the new texture map for binning, calculate the position of the set in the new image, obtain the texture coordinates of each vertex in the texture map, and generate the texture map.

[0104] This embodiment also processes the seams between adjacent textures. The processing method is as follows: the objective function J of the texture is optimized using the following matrix:

[0105]

[0106] In the formula, f represents a vector storing the color difference between all seam vertices in two adjacent sets; g represents the color adjustment value vector of all vertices; A and Γ represent the sparse matrices of vertices in two adjacent sets, respectively, and T represents the transpose of the corresponding matrix.

[0107] Finally, the temperature field graphics are mapped onto the surface of the 3D digital model using the following mapping method:

[0108] Write the point, surface, and grouping information of the 3D digital model into the obj file;

[0109] Specify the temperature field image of the corresponding group in the MTL material description file;

[0110] Load the prepared OBJ file, MTL material description file, and temperature field images from various viewpoints into the 3D display software.

[0111] Example 2:

[0112] A method for mapping three-dimensional temperature field images. In this embodiment, the temperature field distribution is represented by an image (thermograph / cloud map), so the temperature field mapping is essentially a texture mapping problem.

[0113] The existing data in this embodiment includes a 3D digital model of an object composed of a mesh, texture images with temperature field distribution information from different viewpoints, and camera pose matrices (calibration data) corresponding to each viewpoint. The mapping method is as follows:

[0114] Using texture images from different perspectives and digital surface models of 3D objects as input, find the texture image corresponding to each grid in the digital surface model;

[0115] The texture coordinates corresponding to the mesh are calculated from the imaging projection relationship in the calibration results;

[0116] Texture mapping is completed by writing the texture coordinates corresponding to the mesh into the model file and writing the corresponding texture description file.

[0117] The specific implementation process is as follows:

[0118] (a) Distribution of temperature field images:

[0119] Considering that the visibility of texture images varies across different viewpoints for a 3D model, this embodiment employs a viewpoint-based 3D object surface segmentation method. Based on the total number of sampled temperature field images, the 3D model surface is divided into several small surface regions. Each small surface region corresponds to a temperature field image from that viewpoint, ensuring the visibility of the small surface region while avoiding multi-region remapping of the temperature field image. For example... Figure 3 As shown, assuming a total of 6 temperature field images are acquired, because the temperature field images are sampled at a fixed frame rate, with the turntable rotating at a constant speed, the 6 images should be evenly distributed around the 3D object. Therefore, in the figure, the 6 temperature field images IM j j = 1, 2, ..., 6, corresponding to the 6 surface regions VA of the 3D object. j Let j = 1, 2, ..., 6, and each temperature field image corresponds to a region of 60 degrees. Assume the plane normal vector of the temperature field image IM is V.IM Using this direction as the center direction, calculate the normal vector V of each mesh on the surface of the 3D object. n The angle α between the grid and the center direction, if α is less than or equal to half of the dividing angle β, then the temperature field mapping pixel block Texture(V) corresponding to the grid is... n If it exists in the temperature field image, then it does not exist; otherwise, it does not exist.

[0120] Texture(V n )∈IM j ,ifα n ≤β.

[0121] This method can automatically divide the surface region based on the number of sampled images, and can also achieve better actual mapping effects by changing the sampling frequency of the temperature field images. For example, if the temperature field of the target surface is complex and variable, and six temperature field images are no longer sufficient to cover it completely, resulting in the loss of detailed temperature field, the sampling frequency of the temperature field images can be increased to further subdivide the viewing angle range.

[0122] (ii) Determine the mapping relationship between the temperature field image and the grid:

[0123] After determining the temperature field image corresponding to each grid, the next step is to assign specific pixel coordinates to the vertices of the grid. First, based on the camera pose parameters determined by calibration, the vertex coordinates of the corresponding region in the temperature field image are converted from the world coordinate system to the camera coordinate system. Then, the temperature field coordinates of the 3D vertex are calculated using parameters such as the camera's focal length and image size. A pixel block is determined by the 3D vertex, and the corresponding pixel block is attached to the grid, thus completing the temperature mapping of the 3D object model.

[0124] like Figure 4 As shown, the texture coordinate system is based on the bottom left corner of the image. t The origin is the coordinate system, with the horizontal axis U pointing to the right and the vertical axis V pointing upwards. Pixel positions are represented by coordinates [u, v]. The texture coordinate system scales the entire image horizontally and vertically to the interval [0, 1], so u and v both take values ​​of [0, 1]. This facilitates the stretching or scaling of pixel blocks during temperature field mapping. Figure 4 In the middle, assume a three-dimensional vertex V a The coordinates in the camera coordinate system are [X,Y,Z]. Based on the imaging projection relationship in the pinhole camera model, the vertex V... a coordinate p in texture coordinate system a for:

[0125]

[0126]

[0127] Where fx ,f y ,c x ,c y Here, w and h are the camera intrinsic parameters, representing the image width and height in pixels, respectively. Using the two formulas above, the temperature field image IM corresponding to each 3D vertex in the 3D object model can be calculated. j Temperature field coordinates p ij .

[0128] (III) Temperature Field Mapping Process

[0129] The OBJ file format is a standard 3D model file format developed by Wavefront. In addition to the basic information of the model such as 3D points, normals, and faces, it also supports custom curves and surfaces. The file itself does not have color information, but materials and temperature field maps can be added to the model through the .mtl material description file.

[0130] An OBJ file contains many elements that describe the features of a 3D object, among which there are three essential elements: vertices, faces, and groups.

[0131] The first part, vertex data, mainly includes the geometric vertex (v) specifying its 3D location, the coordinates (vt) indicating the corresponding temperature field map, and the vertex normal (vn). The abbreviations in parentheses are header identifiers indicating their type. OBJ files can optionally include vertex normals or temperature field coordinates, but vertex geometric locations are mandatory. To facilitate the representation of faces (f), OBJ introduces the concept of vertex indices, using an automatically numbered sequential indexing method.

[0132] The second part is the face (f) data. Each face is described by three or more sets of vertex indices, and each set of indices has at least one geometric vertex attribute. Together with the temperature field coordinates and normals, it can contain three different attributes. If a face is represented by three vertices, it is a triangular mesh; if it is represented by four or more vertices, it is a polygonal mesh.

[0133] The third part is the concept of groups (g). OBJ files allow all faces to be grouped, with the purpose of assigning different temperature field images and material properties to different groups. The OBJ standard places the temperature field and material for each group in a material temperature field library (.mtl) file. The material temperature field library file mainly contains material lighting parameters such as ambient light reflection (Ka), diffuse reflection (Kd), specular reflection (Ks), and the weighted reflectance index (Ns) of specular reflection. Most importantly, the diffuse temperature field image file is specified using the map_Kd command.

[0134] The program writes the point, face, and grouping information of the 3D model into the OBJ file according to the above specifications, and specifies the temperature field image of the corresponding group in the MTL material description file. Then, the prepared OBJ file, MTL material description file, and temperature field images from various perspectives are loaded into the 3D display software to present a 3D model with temperature field information.

[0135] (iv) Temperature field mapping optimization based on target contour boundary adjustment

[0136] In existing technologies, when the same contour line is mapped onto a 3D model through temperature field images from different angles, misalignment may occur. This is mainly caused by camera pose estimation errors and geometric errors in the 3D model. The temperature field mapping relationship calculated by the pinhole imaging model has certain deviations due to these two errors. The uncertainty of the errors means that the same contour line in different temperature field images may be mapped to different positions in the 3D model.

[0137] The approach to solving this optimization problem in this embodiment is as follows: First, the main contour lines inside the object are detected. For two adjacent temperature field images, their contour lines and mapping boundaries are uniformly mapped onto one of the images. The contour lines intersecting with the mapping boundary are identified, and the locations where the distance between the two intersection points is larger are the locations where boundary adjustment is needed. Finally, the locations where the contour lines from different temperature field images intersect are found; these are the target locations for boundary adjustment. The basic framework is as follows: Figure 5 As shown, the specific process is described below:

[0138] Step 1: Determine the outline

[0139] Identifying the main contour lines within an object is fundamental to this process. Only by finding these main contour lines can the distances between them in different temperature field images be calculated, thereby identifying the locations where the boundaries need adjustment and the target locations for boundary adjustment. Since minimizing the temperature field cost function already allows the mapping boundary to bypass complex regions of the temperature field, after the previous adjustment process, the mapping boundary will generally not pass through fine contour lines. Therefore, we only need to detect the contour lines of larger regions, such as the contour lines of the hair region, face, neck, clothing, etc. We combine the Canny operator and superpixel segmentation algorithm to obtain the contour lines in each temperature field image, mainly consisting of the following three steps:

[0140] (1) Canny edge detection

[0141] First, the Canny operator is used to perform edge detection on the two-dimensional color image to obtain the approximate contour information of the image. The Canny edge detection operator is a multi-level edge detection algorithm, and its main steps include: smoothing the image with a Gaussian filter; calculating the magnitude and direction of the gradient using the finite difference of the first-order partial derivative; performing non-maximum suppression on extrema; and detecting and connecting edges using a double thresholding method.

[0142] (2) Superpixel segmentation

[0143] Due to the complexity of image temperature field information, the Canny operator alone often fails to achieve satisfactory contour results. The edge information obtained by the Canny operator is often too fragmented, failing to produce precise contour lines and severely impacting the optimization of the contour cost function. Therefore, additional steps are needed to further refine the edge information. In this paper, we introduce a superpixel segmentation algorithm to obtain larger-scale edge block information and combine this information with the edge information obtained by the Canny operator to achieve satisfactory target edge results. The superpixel segmentation algorithm is a graph cut optimization-based segmentation algorithm. It first covers the entire image with fixed-size, overlapping square blocks. Each pixel may be covered by multiple square blocks. Then, these square blocks are stitched together, or in other words, each pixel is assigned one of all square blocks. If two adjacent pixels are assigned to the same square block, there is no penalty function. However, if they belong to different square blocks, a stitching penalty function exists, which is inversely proportional to the grayscale difference between the two pixels. In other words, the above process is to re-stitch the square blocks so that their boundaries match the boundaries in the image as closely as possible, as this minimizes the stitching cost.

[0144] Superpixel segmentation can also obtain superpixels of corresponding sizes by changing the size of the square blocks. Setting the square blocks to be larger can be used for relatively coarse region segmentation. After segmentation, the pixels in each region have relatively similar color and temperature fields.

[0145] (3) Filter out the details to obtain the final outline.

[0146] Superpixel segmentation algorithms can perform coarse region segmentation, but the square blocks used are often large, leading to significant errors in the segmentation boundaries and making it difficult to determine accurate boundaries. Since calculating the contour cost function requires more precise segmentation boundaries, this embodiment combines the coarse region results obtained from superpixel segmentation with the contour information obtained from the Canny operator. Specifically, we first use the Canny operator to detect edges in the temperature field image more precisely, and then use the region segmentation results from the superpixel segmentation algorithm to filter out unwanted detail edges. What remains are the main contour lines of the object. These contour line results can then be used to detect contour line offsets and intersection points.

[0147] Edge detection, superpixel segmentation, and contour extraction were performed on temperature images acquired in the experiment. The experimental results are as follows: Figures 6 to 9 As shown.

[0148] Step 2: Determining the offset and intersection positions of the contour lines.

[0149] After obtaining the contour lines of each temperature field image in the first step, for each pair of adjacent temperature field images, the contour lines and temperature field mapping boundaries are mapped to the same image using pixel-to-pixel matching relationships. Contour lines intersecting the temperature field mapping boundary are identified. If the distance between their intersection points exceeds a certain threshold, the contour lines are considered to potentially be misaligned, and the temperature field mapping boundary needs to be adjusted to a more suitable position. Two contour lines will intersect at a certain point, where the intersection distance is 0, meaning the contour cost function is minimized. Therefore, the temperature field mapping boundary should be adjusted to pass through this intersection point to avoid misalignment.

[0150] (V) Implementation steps of the mapping-based texture reconstruction algorithm

[0151] Step 1: Texture Generation

[0152] After selecting an image view for each face, the faces that are connected and correspond to the same view can be regarded as a patch. Performing subsequent operations on a patch-by-patch basis can reduce computational overhead and enhance color continuity, thus avoiding the impact of seams on the final presentation.

[0153] After obtaining the patches, the next step is simply to place each patch into an atlas (texture map) and map the vertices of the mesh (3D mesh) to pixels in the atlas. Specifically, for each patch, the area containing the patch is extracted from the corresponding view to form a new small image. The image coordinates of each vertex contained in the patch within the small image are calculated using the intrinsic and extrinsic parameters of the view, i.e., the position of the patch. Next, a new image (i.e., a texture map) is created, and the small images corresponding to each patch are placed into the new image. This process can be viewed as a bin packing problem. Using a bin packing algorithm, the position of the patch in the new image can be calculated, thereby transferring the image coordinates of the vertices from the small images of the patch to the new image. These coordinates are the texture coordinates of the vertices. If a texture map is full, a new texture map is created.

[0154] Step 2: Seam Treatment

[0155] The seam forms because the left and right patches are mapped to different viewpoints. The same location of the same object, viewed from different angles, will appear in different colors due to factors such as lighting. Therefore, when the images from the two viewpoints are stitched together, a discontinuity occurs, resulting in the seam. The solution in this embodiment is to simultaneously adjust the pixel colors of the left and right patches to make them more similar, thus minimizing the discontinuity.

[0156] Let f be the colors of the same vertex v in the two different patches on the left and right sides. vleft and f vright The corresponding color adjustment values ​​are g. vleft and g vright To make the colors on the left and right sides more similar after adjustment, it is equivalent to optimizing the objective function J1:

[0157]

[0158] However, on the other hand, considering that if the color adjustment values ​​of two adjacent vertices in the same patch differ too much, it can easily lead to color anomalies within the patch, it is also necessary to constrain the color adjustment values ​​of adjacent vertices in the same patch, which means optimizing another objective function J2:

[0159]

[0160] For ease of calculation, the objective functions J1 and J2 are optimized simultaneously and written in the following matrix form:

[0161]

[0162] Where f represents a vector storing the color difference between the left and right patches for all gap vertices, and g represents the color adjustment value vector for all vertices (the same vertex is split into left and right vertices); A and Γ represent sparse matrices of vertices in two adjacent sets, respectively. A sparse matrix means that the number of elements with a value of 0 is much greater than the number of non-zero elements, and the distribution of non-zero elements is irregular; in this embodiment, A and Γ contain only 0 and ±1, where A is used to select the left and right vertices on the gap, and Γ is used to select adjacent vertices in the same patch. The specific matrix form is as follows:

[0163]

[0164] By implementing all the above steps, the desired three-dimensional temperature field mapping image can be obtained, as shown below. Figure 10 As shown.

[0165] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0166] It should be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Furthermore, the term “connection” as used herein, unless otherwise specified, can mean a direct connection or an indirect connection via other components.

Claims

1. A method for mapping a three-dimensional temperature field image, characterized in that, Includes the following steps: Sample temperature field images from different perspectives; Based on the total number of sampled temperature field images, the surface of the 3D digital model is divided into several small surface regions, each of which corresponds to a temperature field image from a specific viewpoint. Assign a temperature field image to the surface of each small region of the 3D digital model; Establish the mapping relationship between each grid on the surface of each small region and the corresponding temperature field image, and map the temperature field image onto the surface of the three-dimensional digital model; It also includes verifying the temperature field image assigned to a small area of ​​the surface of the 3D digital model. The verification methods include: Given a temperature field image, determine the division angle of the surface of the small region corresponding to the temperature field image as 2β; Using the direction of the plane normal vector of the temperature field image as the center direction, calculate the angle α between the normal vector of each grid on the surface of a small region of the three-dimensional digital model and the center direction; Compare α and β: If α ≤ β, then the temperature field mapping pixel block corresponding to the grid is considered to exist in the temperature field image; Methods for establishing the mapping relationship between each grid and the corresponding temperature field image in each small region include: Based on the camera pose parameters used in the temperature field image, the 3D vertex coordinates of the corresponding region in the temperature field image are converted from the world coordinate system to the camera coordinate system; Based on the camera intrinsic parameters, the temperature field coordinates p of the 3D vertex are calculated using the following formula. ij (u,v): In the formula, [X,Y,Z] are the coordinates of the 3D vertex in the camera coordinate system, w is the width of the image, h is the height of the image, and f is the height of the image. x ,f y ,c x ,c y For camera intrinsic parameters, among which f represents the focal length. and Indicates cell size, c x c y This represents the center coordinates in the image pixel coordinate system; During the mapping process of the temperature field image, the target contour boundary is adjusted using the following method: Determine the outline of the object; For two adjacent temperature field images, the contour lines and mapping boundaries of both images are mapped onto one of the images. The contour lines that intersect with the mapping boundary are then identified to determine if there is any misalignment. If the judgment result is misalignment, adjust the temperature field mapping boundary to correct the misalignment; The process of mapping a temperature field image onto the surface of a 3D digital model includes the following texture generation methods: In a three-dimensional digital model, a set of meshes that are contiguous and located in the same temperature field image is defined as a patch. Extract the range of this set from the corresponding temperature field image to form a new smaller image; Based on the position of the set in the temperature field image, calculate the image coordinates of each vertex contained in the set in the smaller image; Create a new texture map, put the small images corresponding to the set into the new texture map for binning, calculate the position of the set in the new image, obtain the texture coordinates of each vertex in the texture map, and generate the texture map; This also includes processing the seams between adjacent textures, the processing method is as follows: Optimize the objective function J1 using the following formula: Optimize the objective function J2 using the following formula: In the formula, f vleft f vright These represent the colors of the same vertex v in two different patches; g vleft g vright These are the color adjustment values ​​for the same vertex v in two different patches, on the left and right sides, respectively; g vi g vj These are two adjacent vertices within the same patch; The objective function of the texture is optimized using matrix J, where J = J1 + J2.

2. The mapping method for a three-dimensional temperature field image according to claim 1, characterized in that, The method of mapping temperature field images onto the surface of a 3D digital model includes the following texturing steps: Write the point, surface, and grouping information of the 3D digital model into the obj file; Specify the temperature field image of the corresponding group in the MTL material description file; Load the prepared OBJ file, MTL material description file, and temperature field images from various viewpoints into the 3D display software.

3. The mapping method for a three-dimensional temperature field image according to claim 1, characterized in that, Methods for determining the outline of an object include: The Canny operator is used to perform edge detection on the temperature field image to obtain the approximate contour information in the image. Large-scale edge block information in temperature field images is obtained using a superpixel segmentation algorithm. Combining the edge block information and the general outline information; Remove the approximate outline information that is located within the same region segmentation as the edge block information, and use the retained boundaries as the outline lines of each region.

4. The mapping method for a three-dimensional temperature field image according to claim 1, characterized in that, The method for determining whether there is misalignment is as follows: extract the intersection points of the contour line and the mapped boundary, and determine the distance between any two adjacent intersection points. If the distance between any two intersection points is greater than a set threshold, then it is considered misaligned.

5. The mapping method for a three-dimensional temperature field image according to claim 3, characterized in that, Methods for obtaining large-scale edge patch information in temperature field images using superpixel segmentation algorithms include: The entire image is covered with square blocks of fixed size that overlap each other. The square blocks are then stitched together so that each pixel is assigned to one of the square blocks. Extract two adjacent pixels belonging to different square blocks, and reassemble the square blocks using a concatenation penalty function to make the boundaries of the square blocks match the boundaries of the two adjacent pixels as closely as possible, thus obtaining edge block information.

Images