A feature alignment method and device

Abstract

The present application provides a feature alignment method and device, which determines the correction area by picking up the same-name points, calculates the offset of the same-name points, and uses the multi-level B-spline algorithm and the offset of the same-name points to make a multi-level B-spline approximation on the correction area, obtains the original coordinates of the first correction area in the original photo and the corresponding target coordinates in the target photo, and assigns the pixel values ​​of the original coordinates to the corresponding target coordinates, thereby realizing the alignment of texture features and geometric features, and correcting the deviation between texture features and geometric features during texture mapping. And feature alignment is realized automatically, without the need to manually pick up a large number of coordinate points, and the deviation between texture features and geometric features during texture mapping is corrected quickly and efficiently, reducing the errors caused by manual operation.

Description

Technical Field

[0001] The present application relates to the technical field of texture mapping, and in particular to a feature alignment method and device. Background Art

[0002] A 3D model mainly consists of three parts: geometry, texture, and material. Texture usually refers to the color details on the surface of an object, which is visible to the human eye. Visible light is irradiated onto the surface of an object to form diffuse reflection, which is transmitted to the human eye to form a color effect. It not only expresses the uneven grooves on the surface of an object, but also includes the color patterns on the smooth surface of the object, also known as texture mapping. Textures are generally stored in computer formats in image formats, such as JPG, PNG, TGA, etc., which is a two-dimensional data format. In order to correctly display a two-dimensional texture on a three-dimensional model, it is also necessary to define the correspondence between two-dimensional and three-dimensional, that is, the UV coordinates. The texture mapping process is the process of obtaining this correspondence.

[0003] During texture mapping, since the distortion of the photo cannot be completely eliminated, some key features on the object may not be aligned after texture mapping, resulting in misalignment. That is, there are deviations between the texture features and the geometric features and they cannot be completely overlapped, such as buttons on the sofa and nails on the furniture. These deviations have a great impact on the overall mapping effect. Summary of the invention

[0004] The purpose of the present invention is to provide a feature alignment method and device, which can solve the problem of how to automatically correct the deviation between texture features and geometric features when texture mapping occurs, so as to achieve feature alignment.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0006] In a first aspect, a feature alignment method is provided, comprising:

[0007] Picking up the feature points on the original photo that need to be corrected and the feature points corresponding to the three-dimensional model as the same-name points, obtaining the original coordinates and the target coordinates of the same-name points, and calculating the deviations of the same-name points according to the original coordinates and the target coordinates of the same-name points;

[0008] Picking a first correction area on the original photo according to the same-name point;

[0009] Performing multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo;

[0010] The pixel values ​​of the original coordinates are assigned to the corresponding target coordinates through an interpolation algorithm.

[0011] The above feature alignment method picks the correction area according to the same-name points, calculates the deviation of the same-name points, and uses the multi-level B-spline algorithm and the deviation of the same-name points to make a multi-level B-spline approximation on the correction area, so as to obtain the original coordinates of the first correction area in the original photo and the corresponding target coordinates in the target photo, and assigns the pixel values ​​of the original coordinates to the corresponding target coordinates, thereby realizing the alignment of texture features and geometric features and correcting the deviation between texture features and geometric features during texture mapping. And feature alignment is realized automatically, without the need to manually pick up a large number of coordinate points, and the deviation between texture features and geometric features during texture mapping is corrected quickly and efficiently, reducing the errors caused by manual operation.

[0012] A possible implementation of the first aspect, wherein picking up a first deskewing area on the original photo according to the same-name point comprises: obtaining a coordinate range of the original photo; determining whether a target coordinate of the same-name point is within the coordinate range of the original photo; and picking up a first deskewing area on the original photo according to the same-name point when the target coordinate of the same-name point is within the coordinate range of the original photo.

[0013] A possible implementation of the first aspect is that picking up a first correction area on the original photo based on the same-name points also includes: constructing a convex hull using the original coordinates of the same-name points according to a convex hull algorithm; expanding the convex hull; and picking up points in the expanded convex hull to construct a second correction area.

[0014] A possible implementation of the first aspect is to perform multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo, including: using a multi-level B-spline algorithm to obtain a B-spline surface according to the original coordinates; obtaining the deviation between the original coordinates and the target coordinates according to the B-spline surface; and determining the corresponding target coordinates according to the deviation and the original coordinates.

[0015] A possible implementation method of the first aspect is to assign the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm, including: constructing a dual-channel matrix based on the deviation; obtaining the pixel values ​​of the original coordinates; and using the dual-channel matrix to assign the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm.

[0016] In a second aspect, a feature alignment device is provided, comprising:

[0017] The first picking module is used to pick up the feature points on the original photo that need to be corrected and the feature points corresponding to the three-dimensional model as points of the same name;

[0018] A second picking module, used for picking a first correction area on the original photo according to the same-name point;

[0019] An input module, used for performing multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo;

[0020] The assignment module is used to assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm.

[0021] The feature alignment device picks up the correction area according to the same-name points, calculates the deviation of the same-name points, and uses the multi-level B-spline algorithm and the deviation of the same-name points to make a multi-level B-spline approximation on the correction area, so as to obtain the original coordinates of the first correction area in the original photo and the corresponding target coordinates in the target photo, and assigns the pixel values ​​of the original coordinates to the corresponding target coordinates, thereby realizing the alignment of texture features and geometric features and correcting the deviation between texture features and geometric features during texture mapping. And feature alignment is realized automatically, without the need to manually pick up a large number of coordinate points, and the deviation between texture features and geometric features during texture mapping is corrected quickly and efficiently, reducing the errors caused by manual operation.

[0022] A possible implementation method of the second aspect is that the second picking module is also used to: obtain the coordinate range of the original photo; determine whether the target coordinates of the same name point are within the coordinate range of the original photo; when the target coordinates of the same name point are within the coordinate range of the original photo, pick up the first correction area on the original photo according to the same name point.

[0023] In a possible implementation of the second aspect, the second picking module is further used to: construct a convex hull using the original coordinates of the points with the same name according to a convex hull algorithm; expand the convex hull; and pick up points in the expanded convex hull to construct a second correction area.

[0024] In a possible implementation of the second aspect, the input module is further used to: obtain a B-spline surface based on the original coordinates using a multi-level B-spline algorithm; obtain a deviation between the original coordinates and the target coordinates based on the B-spline surface; and determine the corresponding target coordinates based on the deviation and the original coordinates.

[0025] A possible implementation of the second aspect is that the assignment module is also used to: construct a dual-channel matrix based on the deviation; obtain the pixel value of the original coordinate; and use the dual-channel matrix to assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] in:

[0028] Figure 1 Schematic diagram of texture features and geometric features;

[0029] Figure 2 A schematic diagram of alignment of texture features and geometric features in one embodiment;

[0030] Figure 3 is a flow chart of a feature alignment method in one embodiment;

[0031] Figure 4 A schematic diagram of convex hull expansion in one embodiment;

[0032] Figure 5 is a system block diagram in one embodiment;

[0033] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] It should be noted that the terms "include", "comprises" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, terminal, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. In the claims, specification and drawings of the present application, relational terms such as "first" and "second" are merely used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] The three-dimensional model mainly includes three parts: geometry, texture, and material. Texture usually refers to the color details on the surface of an object, which is visible to the human eye. Visible light is irradiated on the surface of an object to form diffuse reflection, which is transmitted to the human eye to form a color effect. It not only expresses the uneven grooves on the surface of the object, but also includes the color pattern on the smooth surface of the object, which is also called texture mapping. The format in which the texture is stored in the computer is generally a picture format, such as JPG, PNG, TGA, etc., which is a two-dimensional data format. In order to correctly display the two-dimensional texture on the three-dimensional model, it is also necessary to define the correspondence between the two-dimensional and three-dimensional, that is, the UV coordinates. The texture mapping process is the process of obtaining this correspondence. The general means of obtaining the real texture of an object is to take a photo with a camera. The process of camera imaging is actually the process of converting the points in the world coordinate system to the camera coordinate system, projecting to obtain the image coordinate system, and then converting it to the pixel coordinate system. However, due to the distortion introduced by the perspective process and processing accuracy, that is, the straight line in the world coordinate system is no longer a straight line when converted to other coordinate systems, resulting in deviations between the texture features and the geometric features during texture mapping, and no alignment, such as the buttons on the sofa, such as Figure 1 As shown in the figure, the left image shows that the texture features of the sofa buttons are not aligned with the geometric features, and the right image shows that the texture features of the sofa buttons are aligned with the geometric features. It can be seen that when the texture features of the buttons are not aligned with the geometric features, these deviations have a great impact on the overall mapping effect. When the texture features and geometric features deviate, the best solution is to first select the part where the deviation occurs, such as Figure 2 As shown, the solid line is the texture feature, and the dotted line is the geometric feature. The coordinates of the part on the image are offset to the projection coordinates of the part on the three-dimensional model, that is, the area surrounded by the solid line is offset to the area surrounded by the dotted line, so that the offset coordinates are consistent with or close to the projection coordinates of the three-dimensional model, and the offset image is obtained. Then the image is mapped to the three-dimensional model, and the deviation of the texture feature and the geometric feature can be corrected.

[0039] like Figure 3 As shown, a feature alignment method is proposed, including:

[0040] Step 101, the terminal picks up the feature points on the original photo that need to be corrected and the feature points corresponding to the three-dimensional model as the same-name points, obtains the original coordinates and target coordinates of the same-name points, and calculates the deviation of the same-name points based on the original coordinates and target coordinates of the same-name points.

[0041] Among them, a point in the object that corresponds to the same position in the photo and the 3D model is called a homonymous point. The system displays a list of all photos involved in the mapping, a 2D picture view, and a 3D scene view. The 3D scene view displays the 3D model with the texture mapped. In the 3D scene, the model texture mapping effect is browsed through human-computer interaction. When it is found that the texture features are not aligned with the model features, the photo corresponding to the texture is selected in the photo list. The feature points with deviations in the photo and the corresponding points in the 3D model are picked up to form a pair of homonymous points. The feature points that need to be aligned, i.e., corrected, are picked up as homonymous points. The original photo is a photo containing texture features. Since the original coordinates and the projection coordinates, i.e., the target coordinates, of the homonymous points are known, the distance that the original coordinates need to move, i.e., the deviation of the homonymous points, can be calculated. The terminals include: computers, tablet computers, etc.

[0042] Step 102: The terminal picks up a first correction area on the original photo according to the same-name points.

[0043] After selecting the same-name points, the same-name points on the periphery of the original photo are connected to form a first correction area. In addition, a convex hull can be constructed according to a convex hull algorithm, and the convex hull can be expanded so that the expanded convex hull includes the first correction area, so as to reduce the error when aligning the texture feature with the geometric feature.

[0044] Step 103: The terminal performs multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo.

[0045] Among them, in order to correct the deviation between texture features and geometric features, it is necessary to offset the coordinates of the correction area on the original photo to the projection coordinates of the area on the three-dimensional model, so that the offset coordinates are consistent with or close to the projection coordinates of the three-dimensional model. The target coordinates reflect the geometric features of the three-dimensional model, and the original coordinates reflect the texture features of the photo. When the original coordinates correspond to the target coordinates, the texture features and geometric features are also aligned. Due to the distortion of the photo, the original coordinates and the target coordinates do not correspond, resulting in deviations and misalignment between the texture features and the geometric features. The target photo is a constructed blank photo. The correction area in the original photo is corrected and copied to the target photo, and then the target photo is texture mapped with the three-dimensional model to correct the deviation between the texture features and the geometric features, so as to achieve alignment between the texture features and the geometric features. Since the coordinates of the points on the photo are known, the original coordinates of the picked points of the same name are known. According to the original coordinates of the correction area and the deviation of the same-name points, the multi-level B-spline algorithm is used to perform multi-level B-spline approximation on the correction area to obtain a multi-level B-spline surface, and the deviation between the original coordinates and the target coordinates is obtained according to the multi-level B-spline surface. After the deviation is obtained, the target coordinates can be obtained according to the deviation and the original coordinates, and finally the alignment of the texture features and the geometric features is achieved. Since the original coordinates and the projection coordinates of the same-name points, that is, the target coordinates, are known, the distance that the original coordinates need to move can be calculated. The original coordinates of the other points in the point set except the same-name points are known. By performing the multi-level B-spline approximation on the point set as described above, a uniform cubic B-spline surface is obtained, and the smooth moving distance of the other points except the same-name points can be obtained.

[0046] Step 104: The terminal assigns the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm.

[0047] After obtaining the deviation between the original photo and the target coordinates, the target coordinates can be obtained based on the original coordinates. The pixels of the original coordinates are known, so the target coordinates are found based on the original coordinates and the deviation, and the pixel values ​​of the original coordinates are assigned to the corresponding target coordinates through the interpolation algorithm. When the pixel values ​​of all pixels in the correction area are assigned to the target coordinates, the alignment of texture features and geometric features is achieved.

[0048] The above feature alignment method picks the correction area according to the same-name points, calculates the deviation of the same-name points, and uses the multi-level B-spline algorithm and the deviation of the same-name points to make a multi-level B-spline approximation on the correction area, so as to obtain the original coordinates of the first correction area in the original photo and the corresponding target coordinates in the target photo, and assigns the pixel values ​​of the original coordinates to the corresponding target coordinates, thereby realizing the alignment of texture features and geometric features and correcting the deviation between texture features and geometric features during texture mapping. And feature alignment is realized automatically, without the need to manually pick up a large number of coordinate points, and the deviation between texture features and geometric features during texture mapping is corrected quickly and efficiently, reducing the errors caused by manual operation.

[0049] In one embodiment, picking up the first deskewing area on the original photo according to the same-name point includes: obtaining the coordinate range of the original photo; determining whether the target coordinates of the same-name point are within the coordinate range of the original photo; when the target coordinates of the same-name point are within the coordinate range of the original photo, picking up the first deskewing area on the original photo according to the same-name point.

[0050] Among them, when the target coordinates are not within the coordinate range of the photo, even if the subsequent feature alignment steps are performed, feature alignment cannot be achieved. The coordinates of the pixel points on the original photo are all known, so the coordinates of the four corner points of the original photo are also known. According to the coordinates of the four corner points of the original photo, the coordinate range of the original photo can be obtained. At the same time, the original coordinates of the same-name points are also known. The three-dimensional coordinates of the same-name points on the three-dimensional model are calculated using the rotation and translation matrix of the camera to obtain the two-dimensional coordinates, which are the target coordinates of the same-name points. This embodiment determines whether the target coordinates of the same-name points are within the coordinate range of the photo. Only when the target coordinates of the same-name points are within the coordinate range of the photo, the subsequent processing steps are performed, which eliminates this situation, saves calculation time, and improves efficiency.

[0051] In one embodiment, picking up the first correction area on the original photo according to the same-name points also includes: constructing a convex hull using the original coordinates of the same-name points according to a convex hull algorithm; expanding the convex hull; and picking up points in the expanded convex hull to construct a second correction area.

[0052] Among them, the convex hull is a concept in computational geometry (graphics). In a real vector space V, for a given set X, the intersection S of all convex sets containing X is called the convex hull of X. The convex hull of X can be constructed using the convex combination of all points in X (X1,...Xn). Simply put, given a point set on a two-dimensional plane, the convex hull is a convex polygon formed by connecting the outermost points, which can contain all the points in the point set. The convex hull is constructed by the convex hull algorithm, and the convex hull is expanded. After the convex hull is expanded, the points in the expanded convex hull need to be collected as samples. First calculate the step length between points, and then sample points based on the step length. The step length is calculated as follows: divide the number of points in the point set by 200.0, add 0.5, compare the result with 1, and take the larger value as the step length. The point sampling method is: start the point set from the first element, and take sampling points at intervals of the step length calculated in the previous step. For example Figure 4 As shown, the convex hull algorithm is used to construct the convex hull, i.e., the first correction area, i.e., the area surrounded by the dotted line, and the convex hull is expanded to obtain the second correction area, i.e., the area surrounded by the dotted line. By expanding the convex hull, the collected points are sufficient (including the first correction area), the calculation result is more accurate, and finally the feature alignment accuracy is higher and the effect is better.

[0053] In one embodiment, the original coordinates of the first correction area and the deviations of the same-name points are subjected to multi-level B-spline surface fitting to obtain the corresponding target coordinates in the target photo, including: using a multi-level B-spline algorithm to obtain a B-spline surface according to the original coordinates; obtaining the deviation between the original coordinates and the target coordinates according to the B-spline surface; and determining the corresponding target coordinates according to the deviation and the original coordinates.

[0054] Among them, a multi-level B-spline algorithm is used to perform multi-level B-spline approximation on the correction area to obtain a multi-level B-spline surface. The deviation between the original coordinates and the target coordinates is obtained based on the multi-level B-spline surface. After the deviation is obtained, the target coordinates can be obtained based on the deviation and the original coordinates, and finally the alignment of texture features and geometric features is achieved.

[0055] In one embodiment, assigning the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm includes: constructing a dual-channel matrix based on the deviation; obtaining the pixel values ​​of the original coordinates; and using the dual-channel matrix to assign the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm.

[0056] Among them, a two-channel matrix is ​​first constructed, the deviation is stored in the two-channel matrix, and the two-channel matrix is ​​used for pixel interpolation, so that the pixel value of the original coordinate can be assigned to the corresponding target coordinate, so that the pixel value of each pixel point in the target photo is consistent with the pixel value of the corresponding pixel point in the original photo. The target photo is obtained by correcting the original photo, so the texture image after feature alignment can be obtained.

[0057] like Figure 5 As shown, a feature alignment device is proposed, comprising:

[0058] The first picking module 201 is used to pick the feature points on the original photo that need to be corrected and the feature points corresponding to the 3D model as homonymous points, convert the coordinates of the 3D model points from the world coordinate system to the pixel coordinate system, and calculate the deviation of the homonymous points according to the original coordinates and target coordinates of the homonymous points;

[0059] A second picking module 202, used for picking a first correction area on the original photo according to the same-name point;

[0060] An input module 203 is used to perform multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo;

[0061] The assignment module 204 is used to assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm.

[0062] The feature alignment device picks up the correction area according to the same-name points, calculates the deviation of the same-name points, and uses the multi-level B-spline algorithm and the deviation of the same-name points to make a multi-level B-spline approximation on the correction area, so as to obtain the original coordinates of the first correction area in the original photo and the corresponding target coordinates in the target photo, and assigns the pixel values ​​of the original coordinates to the corresponding target coordinates, thereby realizing the alignment of texture features and geometric features and correcting the deviation between texture features and geometric features during texture mapping. And feature alignment is realized automatically, without the need to manually pick up a large number of coordinate points, and the deviation between texture features and geometric features during texture mapping is corrected quickly and efficiently, reducing the errors caused by manual operation.

[0063] In one embodiment, the second picking module 202 is also used to: obtain the coordinate range of the original photo; determine whether the target coordinates of the same name point are within the coordinate range of the original photo; when the target coordinates of the same name point are within the coordinate range of the original photo, pick up the first correction area on the original photo according to the same name point. When the target coordinates are not within the coordinate range of the photo, feature alignment cannot be achieved even if the subsequent feature alignment steps are performed. This embodiment determines whether the target coordinates of the same name point are within the coordinate range of the photo, and only performs the subsequent processing steps when the target coordinates of the same name point are within the coordinate range of the photo, thereby eliminating this situation, saving calculation time, and improving efficiency.

[0064] In one embodiment, the second picking module 202 is further used to: construct a convex hull using the original coordinates of the same-name points according to a convex hull algorithm; expand the convex hull; pick up points in the expanded convex hull to construct a second correction area. The convex hull is constructed by the convex hull algorithm and expanded, so that enough points are collected, the calculation result is more accurate, and finally the feature alignment accuracy is higher and the effect is better.

[0065] In a specific embodiment, the input module 203 is also used to: obtain a B-spline surface according to the original coordinates using a multi-level B-spline algorithm; obtain the deviation between the original coordinates and the target coordinates according to the B-spline surface; and determine the corresponding target coordinates according to the deviation and the original coordinates. The multi-level B-spline algorithm is used to perform multi-level B-spline approximation on the correction area to obtain a B-spline surface, and the deviation between the original coordinates and the target coordinates is obtained according to the B-spline surface. After the deviation is obtained, the target coordinates can be obtained according to the deviation and the original coordinates, and finally the alignment of the texture feature and the geometric feature is achieved.

[0066] In one embodiment, the assignment module 204 is further used to: construct a dual-channel matrix according to the deviation; obtain the pixel value of the original coordinate; and use the dual-channel matrix to assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm. The deviation is stored in the dual-channel matrix, and the dual-channel matrix is ​​used to perform pixel interpolation to obtain a texture image after feature alignment.

[0067] like Figure 6As shown, in one embodiment, a computer device is proposed, which includes a processor, a memory and a network interface connected via a terminal bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device has a storage operation terminal and can also store a computer program. When the computer program is executed by the processor, the processor can implement the above-mentioned feature alignment method. The internal memory can also store a computer program. When the computer program is executed by the processor, the processor can execute the above-mentioned feature alignment method. Those skilled in the art can understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the device to which the scheme of the present application is applied. The specific device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0068] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned feature alignment method.

[0069] It can be understood that the above-mentioned feature alignment method, device, equipment and storage medium belong to a general inventive concept, and the embodiments are applicable to each other.

[0070] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0071] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A feature alignment method, characterized in that: The method comprises: Picking up the feature points on the original photo that need to be corrected and the feature points corresponding to the three-dimensional model as the same-name points, obtaining the original coordinates and the target coordinates of the same-name points, and calculating the deviations of the same-name points according to the original coordinates and the target coordinates of the same-name points; Picking a first correction area on the original photo according to the same-name point; Performing multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo; Assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm; The step of performing multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo includes: Obtaining a multilevel B-spline surface according to the original coordinates using a multilevel B-spline algorithm; Obtaining a deviation between the original coordinates and the target coordinates according to the multi-level B-spline surface; Determine the corresponding target coordinates according to the deviation and the original coordinates; The step of assigning the pixel value of the original coordinate to the corresponding target coordinate by an interpolation algorithm includes: constructing a dual-channel matrix based on the deviations; Obtaining the pixel value of the original coordinate; The dual-channel matrix is ​​used to assign the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm.

2. The feature alignment method according to claim 1, characterized in that: The step of picking up a first correction area on the original photo according to the same-name point includes: Obtaining the coordinate range of the original photo; Determine whether the target coordinates of the same-name point are within the coordinate range of the original photo; When the target coordinates of the same-name point are within the coordinate range of the original photo, a first deflection correction area is picked up on the original photo according to the same-name point.

3. The feature alignment method according to claim 1, characterized in that: The step of picking up a first correction area on the original photo according to the same-name point further includes: Constructing a convex hull using the original coordinates of the points of the same name according to a convex hull algorithm; expanding the convex hull; Picking up points in the expanded convex hull to construct a second deviation correction area; By expanding the convex hull, the calculated results are more accurate when enough points are collected.

4. A feature alignment device, characterized in that: Used to apply the feature alignment method according to any one of claims 1 to 3, the feature alignment device comprises: The first picking module is used to pick up the feature points on the original photo that need to be corrected and the feature points corresponding to the three-dimensional model as points of the same name; A second picking module, used for picking a first correction area on the original photo according to the same-name point; An input module, used for performing multi-level B-spline surface fitting on the original coordinates of the first correction area and the deviations of the same-name points to obtain the corresponding target coordinates in the target photo; The assignment module is used to assign the pixel value of the original coordinate to the corresponding target coordinate through an interpolation algorithm.

5. The feature alignment device according to claim 4, characterized in that: The second pickup module is also used for: Obtaining the coordinate range of the original photo; Determine whether the target coordinates of the same-name point are within the coordinate range of the original photo; When the target coordinates of the same-name point are within the coordinate range of the original photo, a first deflection correction area is picked up on the original photo according to the same-name point.

6. The feature alignment device according to claim 4, characterized in that: The second pickup module is also used for: Constructing a convex hull using the original coordinates of the points of the same name according to a convex hull algorithm; expanding the convex hull; Picking up points in the expanded convex hull to construct a second deviation correction area; By expanding the convex hull, the calculated results are more accurate when enough points are collected.

7. The feature alignment device according to claim 6, characterized in that: The input module is also used for: Obtaining a B-spline surface according to the original coordinates using a multi-level B-spline algorithm; Obtaining a deviation between the original coordinates and the target coordinates according to the B-spline surface; The corresponding target coordinates are determined according to the deviation and the original coordinates.

8. The feature alignment device according to claim 7, characterized in that: The assignment module is also used for: constructing a dual-channel matrix based on the deviations; Obtaining the pixel value of the original coordinate; The dual-channel matrix is ​​used to assign the pixel values ​​of the original coordinates to the corresponding target coordinates through an interpolation algorithm.

Images