A seam alignment method and device

Abstract

The present application provides a method and device for seam alignment, which selects the misaligned area first, adjusts and corrects the misaligned area, and moves the texture pixels of the triangle mapping in the misaligned area so that the adjusted and corrected area is aligned with the area on the other side of the seam. Only simple operations such as manually picking up the seam line segment are required to automatically achieve seam alignment, without the need for manual processing that requires a lot of time and effort, and the texture alignment degree at the seam is high.

Description

Technical Field

[0001] The present application relates to the technical field of texture mapping, and in particular to a seam alignment method and device. Background Art

[0002] A 3D model mainly consists of three parts: geometry, texture, and material. Texture usually refers to the color details on the surface of an object, which is visible to the human eye. Visible light is irradiated onto the surface of an object to form diffuse reflection, which is transmitted to the human eye to form a color effect. It not only expresses the uneven grooves on the surface of an object, but also includes the color patterns on the smooth surface of the object, also known as texture mapping. Textures are generally stored in computer formats in image formats, such as JPG, PNG, TGA, etc., which is a two-dimensional data format. In order to correctly display a two-dimensional texture on a three-dimensional model, it is also necessary to define the correspondence between two-dimensional and three-dimensional, that is, the UV coordinates. The texture mapping process is the process of obtaining this correspondence.

[0003] When mapping textures, multiple texture photos need to be mapped onto a 3D model. However, at the seams where two photos are mapped on a 3D model, the textures are easily misaligned, such as a continuous stripe breaking at the seam. Misalignment of the textures at the seams will affect the effect of texture mapping.

[0004] The existing solution is usually to manually process the seams of the 3D model using image processing software after the texture mapping is completed, which takes a lot of time and energy, and the processing effect is usually not good. Summary of the invention

[0005] The purpose of the present invention is to provide a seam alignment method and device, which can solve the problem of how to automatically align the textures at the seams.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] In a first aspect, a seam alignment method is provided, comprising:

[0008] Pick up the seam line segment on the three-dimensional model, and obtain the projection coordinates of the vertices of the triangle on the seam line segment in the mapping image;

[0009] Calculate a first range and a second range according to the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment;

[0010] The triangles that are in both the first range and the second range are stored in the triangle set;

[0011] Calculate an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles in the second picture;

[0012] Performing bilinear interpolation on the first image according to the affine transformation matrix to generate a target image;

[0013] Pick up the same-name points of the first image and the target image to obtain a first same-name point pair, convert the same-name points in the first same-name point pair into the second image through an affine transformation matrix to obtain a second same-name point pair of the second image and the target image, calculate the deviation of the first same-name point pair according to the coordinates of the first same-name point pair, and calculate the deviation of the second same-name point pair according to the coordinates of the second same-name point pair;

[0014] Input the deviation of the first image and the first pair of points with the same name into a multi-level B-spline algorithm, and perform a non-rigid transformation to obtain a third image, input the deviation of the second image and the second pair of points with the same name into a multi-level B-spline algorithm, and perform a non-rigid transformation to obtain a fourth image;

[0015] The third picture is aligned with the fourth picture.

[0016] The above-mentioned seam alignment method obtains the coordinate range composed of triangles in the pictures on both sides of the seam segment by obtaining the vertex projection coordinates of the triangles on the seam segment. In addition to the projection coordinates of the triangles on the seam segment, the coordinate range also includes the projection coordinates of the triangles on the non-seam segment. The triangles in the coordinate range on both sides of the seam segment are stored in a set, and the affine transformation matrix is ​​calculated according to the projection coordinates of these triangles to obtain the coordinate conversion relationship of the pictures on both sides of the seam segment. According to the affine transformation matrix, the target picture is generated using the picture on one side of the seam segment, and the deviation between the same-name point pairs of the picture on one side of the seam segment and the target picture is calculated. According to the deviation between the same-name point pairs and the picture on one side of the seam segment, the picture on one side of the seam segment is corrected by a multi-level B-spline algorithm and non-rigid transformation. The same operation is performed on the picture on the other side of the seam segment, and the two corrected pictures are aligned to align the texture at the seam. Only simple operations such as manually picking up the seam segment can automatically realize seam alignment, and no manual work is required to spend a lot of time and energy to process. The processing efficiency is high, and the texture alignment degree at the seam is high.

[0017] A possible implementation of the first aspect, before storing the triangles on the seam segment that are simultaneously in the first range and the second range into the triangle set, further includes: expanding the first range and the second range by preset pixels.

[0018] A possible implementation of the first aspect, picking up the same-name points of the first image and the target image to obtain the first same-name point pair, includes: uniformly selecting sampling points on the seam line segment, calculating through a template matching method the sampling points on the seam line segment in the first image corresponding to the points in the intermediate image, to form the first same-name point pair.

[0019] A possible implementation of the first aspect is to input the deviation of the first image and the first pair of like-named points into a multilevel B-spline algorithm and perform a non-rigid transformation to obtain a third image, and input the deviation of the second image and the second pair of like-named points into a multilevel B-spline algorithm and perform a non-rigid transformation to obtain a fourth image, including: constructing a first weight matrix, inputting the deviation of the first image and the first pair of like-named points into a multilevel B-spline algorithm, and performing a non-rigid transformation in combination with the first weight matrix to obtain a third image; constructing a second weight matrix, inputting the deviation of the second image and the second pair of like-named points into a multilevel B-spline algorithm, and performing a non-rigid transformation in combination with the second weight matrix to obtain a fourth image.

[0020] In a possible implementation manner of the first aspect, the displacement value of the non-rigid transformation is only half.

[0021] In a second aspect, a seam alignment device is provided, comprising:

[0022] A first picking module is used to pick up a seam line segment on the three-dimensional model and obtain the projection coordinates of the vertices of the triangle on the seam line segment in the mapping image;

[0023] A first calculation module is used to calculate a first range and a second range according to the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment;

[0024] A storage module, used for storing triangles that are simultaneously in the first range and the second range into a triangle set;

[0025] A second calculation module, used for calculating an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles on the second picture;

[0026] A generating module, configured to perform bilinear interpolation on the first image according to the affine transformation matrix to generate a target image;

[0027] A second picking module is used for the same-name points of the first image and the target image to obtain a first pair of same-name points, converting the same-name points in the first pair of same-name points into the second image through an affine transformation matrix to obtain a second pair of same-name points in the second image and the target image, calculating the deviation of the first pair of same-name points according to the coordinates of the first pair of same-name points, and calculating the deviation of the second pair of same-name points according to the coordinates of the second pair of same-name points;

[0028] An input module, used for inputting the deviation of the first image and the first pair of points with the same name into a multi-level B-spline algorithm, and performing a non-rigid transformation to obtain a third image, and inputting the deviation of the second image and the second pair of points with the same name into the multi-level B-spline algorithm, and performing a non-rigid transformation to obtain a fourth image;

[0029] An alignment module is used to align the third picture with the fourth picture.

[0030] The above-mentioned seam alignment device obtains the coordinate range composed of triangles in the pictures on both sides of the seam line segment by obtaining the vertex projection coordinates of the triangles on the seam line segment. In addition to the projection coordinates of the triangles on the seam line segment, the coordinate range also includes the projection coordinates of the triangles on the non-seam line segment. The triangles in the coordinate range on both sides of the seam line segment are stored in a set, and the affine transformation matrix is ​​calculated according to the projection coordinates of these triangles to obtain the coordinate conversion relationship of the pictures on both sides of the seam line segment. According to the affine transformation matrix, the target picture is generated using the picture on one side of the seam line segment, and the deviation between the same-name point pairs of the picture on one side of the seam line segment and the target picture is calculated. According to the deviation between the same-name point pairs and the picture on one side of the seam line segment, the picture on one side of the seam line segment is corrected by a multi-level B-spline algorithm and non-rigid transformation. The same operation is performed on the picture on the other side of the seam line segment, and the two corrected pictures are aligned to align the texture at the seam. Only simple operations such as manually picking up the seam line segment are required to automatically realize seam alignment, and no manual work is required to spend a lot of time and energy to process. The processing efficiency is high, and the texture alignment degree at the seam is high.

[0031] In a possible implementation manner of the second aspect, the device further includes: an expansion module, configured to expand the first range and the second range by preset pixels.

[0032] A possible implementation of the second aspect is that the second picking module is further used to: uniformly select sampling points on the seam line segment, calculate the sampling points on the seam line segment in the first image corresponding to the points in the middle image through a template matching method, and form a first pair of points with the same name.

[0033] In a possible implementation of the second aspect, the input module is also used to: construct a first weight matrix, input the deviation of the first image and the first pair of like-named points into a multi-level B-spline algorithm, and perform a non-rigid transformation in combination with the first weight matrix to obtain a third image; construct a second weight matrix, input the deviation of the second image and the second pair of like-named points into a multi-level B-spline algorithm, and perform a non-rigid transformation in combination with the second weight matrix to obtain a fourth image.

[0034] In a possible implementation manner of the second aspect, the displacement value of the non-rigid transformation is only half. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] in:

[0037] Figure 1 This is a schematic diagram of misaligned seams;

[0038] Figure 2 is a flow chart of a seam alignment method in one embodiment;

[0039] Figure 3 A schematic diagram of a triangle coordinate range in an embodiment;

[0040] Figure 4 is a system block diagram in one embodiment;

[0041] Figure 5 is a system block diagram in one embodiment;

[0042] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] It should be noted that the terms "include", "comprises" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, terminal, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In the claims, specification and drawings of the present application, relational terms such as "first" and "second" are merely used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects.

[0046] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0047] In the process of texture mapping, you need to use software to map multiple texture photos onto the 3D model, such as Figure 1 As shown, after mapping the texture photo onto the three-dimensional model, the texture at the seam is often not aligned, for example, a continuous stripe breaks at the seam and becomes two stripes one above and one below. The existing solution is usually to manually process the seams of the three-dimensional model using image processing software after the texture mapping is completed, which takes a lot of time and energy, and the processing effect is usually not good. In this application, a seam alignment method is proposed, and the alignment idea is: first select the unaligned area, adjust and correct the unaligned area, and move the texture pixels mapped by the triangles in the unaligned area so that the adjusted and corrected area is aligned with the area on the other side of the seam. Only simple operations such as manually picking up the seam line segment can automatically achieve seam alignment, without the need for manual processing. A lot of time and energy are not required, and the texture alignment degree at the seam is high.

[0048] like Figure 2 As shown, a seam alignment method is proposed, comprising the following steps:

[0049] Step 101: The terminal picks up a seam line segment on a three-dimensional model, and obtains the projection coordinates of the vertices of the triangle on the seam line segment in a mapping image.

[0050] In the human-computer interaction interface, the seams of the texture are displayed on the three-dimensional model, and the terminal picks up the seam segments on the three-dimensional model by manually operating the mouse or other tools. The seam segments are segments on the seam line that include the start point and the end point. The smallest unit of the set in the three-dimensional model is a triangle, and the system stores the projection coordinates of the triangles on the seam segments in the three-dimensional model in the mapping image. There are two different mapping images on both sides of the seam segment, and usually there is an overlap between the two mapping images. The terminal includes a computer or a tablet computer, etc.

[0051] Step 102, the terminal calculates a first range and a second range based on the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment.

[0052] For ease of understanding, Figure 3 As shown, the oblique line is the seam segment, the coordinate range of the triangle on the left side of the seam segment, that is, the rectangular area surrounded by the dotted line, is the first range, and the coordinate range of the triangle on the right side of the seam segment, that is, the rectangular area surrounded by the realization line, is the second range. Since the projection coordinates of the triangles on the seam segment on the left and right pictures are known, the coordinate ranges of the first range and the second range can be determined according to the projection coordinates. In addition to the triangles on the seam segment, the first range also includes triangles on non-joining segments. Similarly, the second range also includes triangles on non-joining segments. The first range is in the mapping picture on one side of the seam segment, which is the left picture, that is, the first picture in this example; the second range is in the mapping picture on the other side of the seam segment, which is the right picture, that is, the second picture in this example. The first range and the second range can be understood as unaligned areas. If the first range is aligned with the second range, the alignment of the textures on both sides of the seam segment can be achieved.

[0053] Step 103: The terminal stores triangles that are simultaneously in the first range and the second range into a triangle set.

[0054] Among them, Figure 3 As shown, there are triangles on the seam segment that are both within the first range and the second range. In addition, there are triangles in the mapped images on both sides of the seam segment that are both within the first range and the second range. These triangles are stored in a triangle set. The triangles in the triangle set have projection coordinates in both the first image and the second image.

[0055] Step 104: The terminal calculates an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles in the second picture.

[0056] Among them, since the triangles in the above triangle set have projection coordinates both in the first picture and in the second picture, the mutual conversion relationship between the projection coordinates in the first picture and the second picture can be obtained according to the projection coordinates of the above triangles in the first picture and the second picture, that is, the affine transformation matrix.

[0057] Step 105: The terminal performs bilinear interpolation on the first image according to the affine transformation matrix to generate a target image.

[0058] Here, each triangle in the first image is transformed by the projection coordinate affine transformation matrix, and a new image, i.e., the target image, is generated by interpolation from the first image to the second image through the affine transformation matrix. When the projection coordinates of the triangles in the first image are consistent with the projection coordinates of the target image, seam alignment can be achieved.

[0059] Step 106, the terminal picks up the same-name points of the first image and the target image to obtain a first same-name point pair, converts the same-name points in the first same-name point pair to the second image through an affine transformation matrix to obtain a second same-name point pair of the second image and the target image, calculates the deviation of the first same-name point pair according to the coordinates of the first same-name point pair, and calculates the deviation of the second same-name point pair according to the coordinates of the second same-name point pair.

[0060] Among them, sampling points are uniformly taken on the seam line segment, and the number of sampling points is determined according to the actual situation, for example, 5 points are taken. The sampling points in the first image are calculated to correspond to the points in the target image through the template matching method to form the first pair of points of the same name. The points of the same name in the first pair of points of the same name are converted to the second image through the affine transformation matrix to obtain the second pair of points of the same name in the second image and the target image. There is a deviation between the points of the same name in the first image and the points of the same name in the target image in the first pair of points of the same name, and this deviation is the point of the same name in the first image that needs to be corrected. Similarly, there is a deviation between the points of the same name in the second image and the points of the same name in the target image in the second pair of points of the same name. The deviation can be calculated based on the projection coordinates of the points of the same name in the pair of points of the same name. When the triangle in the first image overcomes these deviations, it can be consistent with the target image to achieve seam alignment.

[0061] In step 107, the terminal inputs the deviation of the first image and the first pair of points with the same name into a multi-level B-spline algorithm, and performs a non-rigid transformation to obtain a third image, and inputs the deviation of the second image and the second pair of points with the same name into a multi-level B-spline algorithm, and performs a non-rigid transformation to obtain a fourth image.

[0062] Among them, non-rigid transformation means that the changes in direction and distance of different points in the first image can be different. In this application, it means to perform multi-level B-spline approximation to obtain a uniform cubic B-spline surface, and then use the cubic B-spline surface for bilinear interpolation. The deviation of the first image and the first pair of same-name points is input into the multi-level B-spline algorithm to obtain the first B-spline surface. According to the first B-spline surface, the deviation of other points in the first image except the same-name points of the first pair of same-name points and the corresponding points in the target image can be obtained, and the deviation of other points in the second image except the same-name points of the second pair of same-name points can also be obtained. The deviation of the same-name points of the first image and the second image is obtained in the above step 106, combining the deviation of the same-name points and non-same-name points with the corresponding points of the target image. According to the projection coordinates of the first image and the second image and the deviation from the target image, the third image and the fourth image can be obtained respectively. The third image and the fourth image are obtained after the first image and the second image are corrected, and the non-rigid transformation displacement of the third image and the fourth image is only half, which can avoid the image distortion caused by correcting the image on one side of the seam segment. Aligning the third image with the fourth image will align the seams.

[0063] In addition, there is another implementation method for this step: directly input the deviation of the first image and the first pair of points with the same name into the multi-level B-spline algorithm, and perform a non-rigid transformation, and take all the non-rigid transformation displacements to obtain the third image, that is, only correct the first image or the second image.

[0064] Step 108: The terminal aligns the third picture with the fourth picture.

[0065] Among them, in step 107, the corrected third image and the fourth image have been obtained, and the seam alignment can be achieved by aligning the third image with the fourth image.

[0066] The above-mentioned seam alignment method obtains the coordinate range composed of triangles in the pictures on both sides of the seam segment by obtaining the vertex projection coordinates of the triangles on the seam segment. In addition to the projection coordinates of the triangles on the seam segment, the coordinate range also includes the projection coordinates of the triangles on the non-seam segment. The triangles in the coordinate range on both sides of the seam segment are stored in a set, and the affine transformation matrix is ​​calculated according to the projection coordinates of these triangles to obtain the coordinate conversion relationship of the pictures on both sides of the seam segment. According to the affine transformation matrix, the target picture is generated using the picture on one side of the seam segment, and the deviation between the same-name point pairs of the picture on one side of the seam segment and the target picture is calculated. According to the deviation between the same-name point pairs and the picture on one side of the seam segment, the picture on one side of the seam segment is corrected by a multi-level B-spline algorithm and non-rigid transformation. The same operation is performed on the picture on the other side of the seam segment, and the two corrected pictures are aligned to align the texture at the seam. Only simple operations such as manually picking up the seam segment can automatically realize seam alignment, and no manual work is required to spend a lot of time and energy to process. The processing efficiency is high, and the texture alignment degree at the seam is high.

[0067] In one embodiment, before storing the triangles on the seam line segment that are simultaneously in the first range and the second range into the triangle set, the method further includes: expanding the first range and the second range by preset pixels.

[0068] The first range and the second range are enlarged by preset pixels. If they exceed the picture boundary, they are only enlarged to the picture boundary. The preset pixels are set according to actual needs, for example, 300. Enlarging the first range and the second range, i.e., the ranges to be corrected, can correct the deviation of a larger area, so that the corrected picture is smoother and will not be distorted.

[0069] In one embodiment, picking up the same-name points of the first image and the target image to obtain the first same-name point pair includes: uniformly selecting sampling points on the seam line segment, calculating the sampling points on the seam line segment in the first image corresponding to the points in the intermediate image through a template matching method, to form the first same-name point pair.

[0070] Among them, sampling points are uniformly taken on the seam line segment, and the number of sampling points is determined according to the actual situation, for example, 5 points are taken. The sampling points in the first image are calculated to correspond to the points in the target image through the template matching method to form the first pair of points of the same name. The points of the same name in the first pair of points of the same name are converted to the second image through the affine transformation matrix to obtain the second pair of points of the same name in the second image and the target image. There is a deviation between the points of the same name in the first image and the points of the same name in the target image in the first pair of points of the same name, and this deviation is the point of the same name in the first image that needs to be corrected. Similarly, there is a deviation between the points of the same name in the second image and the points of the same name in the target image in the second pair of points of the same name. The deviation can be calculated based on the projection coordinates of the points of the same name in the pair of points of the same name. When the triangle in the first image overcomes these deviations, it can be consistent with the target image to achieve seam alignment.

[0071] In one embodiment, the method of inputting the deviation of the first image and the first pair of points of the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a third image, and inputting the deviation of the second image and the second pair of points of the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a fourth image includes: constructing a first weight matrix, inputting the deviation of the first image and the first pair of points of the same name into a multilevel B-spline algorithm, and performing a non-rigid transformation in combination with the first weight matrix to obtain a third image; constructing a second weight matrix, inputting the deviation of the second image and the second pair of points of the same name into a multilevel B-spline algorithm, and performing a non-rigid transformation in combination with the second weight matrix to obtain a fourth image.

[0072] Among them, since the points in different areas of the first image need to move, i.e., correct the deviation by different distances, the points close to the seam segment need to move farther, and the points far from the seam segment need to move closer. Therefore, by constructing a weight matrix, the moving distances of different points in the first image and the second image are different, so that the corrected image is smoother. That is, the deviation of the first image and the first pair of points with the same name is input into the multi-level B-spline algorithm, and the non-rigid transformation is performed in combination with the first weight matrix to obtain the third image, and the fourth image is obtained by analogy. The third image and the fourth image in this embodiment are smoother.

[0073] In a specific embodiment, the displacement value of the non-rigid transformation is only half.

[0074] If only the image on one side of the seam segment is skewed, the moving distance of the points in the image will be too large, which is prone to distortion. The non-rigid transformation displacement of the third and fourth images is only half, that is, the images on both sides of the seam segment are skewed, and the skew correction distance is only half, which can also achieve seam alignment and avoid image distortion caused by skew correction of only one side of the seam segment.

[0075] like Figure 4 As shown, a seam alignment device is proposed, the device comprising:

[0076] The first picking module 201 is used to pick up the seam line segment on the three-dimensional model and obtain the projection coordinates of the vertices of the triangle on the seam line segment in the mapping image;

[0077] A first calculation module 202 is used to calculate a first range and a second range according to the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment;

[0078] A storage module 203, used for storing triangles that are in both the first range and the second range into a triangle set;

[0079] A second calculation module 204, configured to calculate an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles in the second picture;

[0080] A generating module 205, configured to perform bilinear interpolation on the first image according to the affine transformation matrix to generate a target image;

[0081] The second picking module 206 is used for obtaining the same-name points of the first image and the target image, obtaining a first same-name point pair, converting the same-name points in the first same-name point pair to the second image through an affine transformation matrix, obtaining a second same-name point pair of the second image and the target image, calculating the deviation of the first same-name point pair according to the coordinates of the first same-name point pair, and calculating the deviation of the second same-name point pair according to the coordinates of the second same-name point pair;

[0082] An input module 207 is used to input the deviation of the first image and the first pair of points with the same name into a multi-level B-spline algorithm, and perform a non-rigid transformation to obtain a third image, and input the deviation of the second image and the second pair of points with the same name into a multi-level B-spline algorithm, and perform a non-rigid transformation to obtain a fourth image;

[0083] The alignment module 208 is configured to align the third image with the fourth image.

[0084] The above-mentioned seam alignment device obtains the coordinate range composed of triangles in the pictures on both sides of the seam line segment by obtaining the vertex projection coordinates of the triangles on the seam line segment. In addition to the projection coordinates of the triangles on the seam line segment, the coordinate range also includes the projection coordinates of the triangles on the non-seam line segment. The triangles in the coordinate range on both sides of the seam line segment are stored in a set, and the affine transformation matrix is ​​calculated according to the projection coordinates of these triangles to obtain the coordinate conversion relationship of the pictures on both sides of the seam line segment. According to the affine transformation matrix, the target picture is generated using the picture on one side of the seam line segment, and the deviation between the same-name point pairs of the picture on one side of the seam line segment and the target picture is calculated. According to the deviation between the same-name point pairs and the picture on one side of the seam line segment, the picture on one side of the seam line segment is corrected by a multi-level B-spline algorithm and non-rigid transformation. The same operation is performed on the picture on the other side of the seam line segment, and the two corrected pictures are aligned to align the texture at the seam. Only simple operations such as manually picking up the seam line segment are required to automatically realize seam alignment, and no manual work is required to spend a lot of time and energy to process. The processing efficiency is high, and the texture alignment degree at the seam is high.

[0085] like Figure 5 As shown, in one embodiment, the device further includes: an expansion module 209, configured to expand the first range and the second range by preset pixels.

[0086] In one embodiment, the second picking module 206 is further used to: uniformly select sampling points on the seam line segment, calculate the sampling points on the seam line segment in the first image corresponding to the points in the middle image through template matching method, and form a first pair of points with the same name.

[0087] In one embodiment, the input module 207 is also used to: construct a first weight matrix, input the deviation of the first image and the first pair of like-named points into a multi-level B-spline algorithm, and perform a non-rigid transformation in combination with the first weight matrix to obtain a third image; construct a second weight matrix, input the deviation of the second image and the second pair of like-named points into a multi-level B-spline algorithm, and perform a non-rigid transformation in combination with the second weight matrix to obtain a fourth image.

[0088] In one embodiment, the displacement value of the non-rigid transformation is only half.

[0089] like Figure 6As shown, in one embodiment, a computer device is proposed, which includes a processor, a memory and a network interface connected via a terminal bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device has a storage operation terminal and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above-mentioned seam alignment method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the above-mentioned seam alignment method. Those skilled in the art can understand that Figure 6 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme of the present application, and does not constitute a limitation on the device to which the scheme of the present application is applied. The specific device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned seam alignment method.

[0091] It is understandable that the above-mentioned seam alignment method, device, equipment and storage medium belong to a general inventive concept, and the embodiments are applicable to each other.

[0092] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0093] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A seam alignment method, characterized in that: The method comprises: Pick up the seam line segment on the three-dimensional model, and obtain the projection coordinates of the vertices of the triangle on the seam line segment in the mapping image; Calculate a first range and a second range according to the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment; The triangles that are in both the first range and the second range are stored in the triangle set; Calculate an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles in the second picture; Performing bilinear interpolation on the first image according to the affine transformation matrix to generate a target image; Picking up the same-name points of the first image and the target image to obtain a first same-name point pair, converting the same-name points in the first same-name point pair to the second image through an affine transformation matrix to obtain a second same-name point pair of the second image and the target image, calculating the deviation of the first same-name point pair according to the coordinates of the first same-name point pair, and calculating the deviation of the second same-name point pair according to the coordinates of the second same-name point pair; wherein picking up the same-name points of the first image and the target image to obtain the first same-name point pair includes: uniformly selecting sampling points on the seam line segment, calculating the sampling points on the seam line segment in the first image corresponding to the points in the middle image through a template matching method to form the first same-name point pair; Inputting the deviation of the first image and the first pair of points with the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a third image, and inputting the deviation of the second image and the second pair of points with the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a fourth image; inputting the deviation of the first image and the first pair of points with the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a third image, and inputting the deviation of the second image and the second pair of points with the same name into a multilevel B-spline algorithm and performing a non-rigid transformation to obtain a fourth image, including: Constructing a first weight matrix, inputting the deviations of the first image and the first pair of points with the same name into a multi-level B-spline algorithm, and performing a non-rigid transformation in combination with the first weight matrix to obtain a third image; Constructing a second weight matrix, inputting the deviation of the second image and the second pair of points with the same name into a multi-level B-spline algorithm, and performing a non-rigid transformation in combination with the second weight matrix to obtain a fourth image; the displacement value of the non-rigid transformation is only half; The third picture is aligned with the fourth picture.

2. The seam alignment method according to claim 1, characterized in that: Before storing the triangles on the seam line segment that are simultaneously in the first range and the second range into the triangle set, the method further includes: The first range and the second range are enlarged by preset pixels.

3. A seam alignment device, characterized in that: The device comprises: A first picking module is used to pick up a seam line segment on the three-dimensional model and obtain the projection coordinates of the vertices of the triangle on the seam line segment in the mapping image; A first calculation module is used to calculate a first range and a second range according to the projection coordinates of the triangle, wherein the first range is the range of the projection coordinates of the triangle on the seam segment in the first picture, and the second range is the range of the projection coordinates of the triangle on the seam segment in the second picture, the first picture is the picture corresponding to the mapped picture on one side of the seam segment, and the second picture is the picture corresponding to the mapped picture on the other side of the seam segment; A storage module, used for storing triangles that are simultaneously in the first range and the second range into a triangle set; A second calculation module, used for calculating an affine transformation matrix according to the projection coordinates of the triangles in the triangle set on the first picture and the projection coordinates of the triangles on the second picture; A generating module, configured to perform bilinear interpolation on the first image according to the affine transformation matrix to generate a target image; The second picking module is used for the same-name points of the first image and the target image to obtain a first same-name point pair, converting the same-name points in the first same-name point pair to the second image through an affine transformation matrix to obtain a second same-name point pair of the second image and the target image, calculating the deviation of the first same-name point pair according to the coordinates of the first same-name point pair, and calculating the deviation of the second same-name point pair according to the coordinates of the second same-name point pair; the second picking module is also used to: uniformly select sampling points on the seam line segment, calculate the sampling points on the seam line segment in the first image corresponding to the points in the middle image through a template matching method, and form a first same-name point pair; An input module is used to input the deviation of the first image and the first pair of points with the same name into a multilevel B-spline algorithm, and perform a non-rigid transformation to obtain a third image, and input the deviation of the second image and the second pair of points with the same name into the multilevel B-spline algorithm, and perform a non-rigid transformation to obtain a fourth image; the input module is also used to: construct a first weight matrix, input the deviation of the first image and the first pair of points with the same name into the multilevel B-spline algorithm, and perform a non-rigid transformation in combination with the first weight matrix to obtain a third image; construct a second weight matrix, input the deviation of the second image and the second pair of points with the same name into the multilevel B-spline algorithm, and perform a non-rigid transformation in combination with the second weight matrix to obtain a fourth image; the displacement value of the non-rigid transformation is only half; An alignment module is used to align the third picture with the fourth picture.

4. The seam alignment device according to claim 3, characterized in that: The device also includes: The expansion module is used to expand the first range and the second range by preset pixels.

Images