Face image registration method and system
Through face recognition and feature point registration methods, combined with area registration, the problem of image registration accuracy and speed of RGB cameras and spectral cameras is solved, and fast and accurate image alignment and fusion are achieved.
Patent Information
- Application Number
- CN202411149901.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2025-01-07
AI Technical Summary
The prior art cannot achieve hard synchronization of the photography moment when the image registration of RGB cameras and spectral cameras, and the field of view is different, resulting in insufficient registration accuracy or slow registration speed.
The face image registration method is adopted, and the face recognition algorithm is used to perform coarse alignment, and the image feature points and area registration method is combined to perform fine alignment, calculate the geometric transformation relationship, realize non-rigid alignment, and fuse the spectral image with high-definition RGB image. .
It achieves fast and accurate image registration, improves registration accuracy and speed, and meets the image fusion needs of hyperspectral cameras and RGB cameras.
Smart Images
Figure CN119273723A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image registration, and in particular relates to a face image registration method and system. Background Art
[0002] Since the spatial resolution of multispectral cameras or hyperspectral cameras is not as high as that of RGB image sensors, a dual-camera solution is generally used to provide clarity of the final effect image. Therefore, it is necessary to align and fuse the image of the spectral camera with the high-definition RGB image. Image registration is the process of aligning multiple images to achieve spatial correspondence between them. The following are several common image registration methods:
[0003] Feature-based registration: It is a more traditional image registration method. Its basic idea is to detect some feature points with significant areas in the two images, such as corner points, edge points or texture points and descriptors around the feature points. These descriptors contain local structural information around the feature points and can more accurately describe the features of the image. By extracting and matching the descriptors of these features, a more accurate image transformation relationship can be calculated to achieve image registration. Commonly used feature extraction and matching algorithms include Harris corner detection, SIFT, SURF (accelerated robust features) and ORB (orientation corrected FAST features). Feature-based registration methods have good robustness and accuracy in dealing with rotation, scale, brightness changes and perspective changes in images.
[0004] Region-based registration: This method divides the image into multiple regions and aligns each region. You can use a segmentation algorithm (such as watershed algorithm, threshold segmentation, etc.) to divide the image into regions, and then align each region independently. Non-rigid registration cannot be achieved.
[0005] Phase correlation method: This method uses the frequency information in the image for registration. It calculates the Fourier transform of the image and performs correlation calculations in the frequency domain to find the location of the maximum correlation peak, thereby determining the spatial transformation between the images. Non-rigid registration cannot be achieved.
[0006] Grayscale mutual information: This method performs registration by calculating the grayscale histogram of the image and its mutual information. Mutual information measures the statistical correlation between two images and determines the optimal image transformation by maximizing the mutual information. Non-rigid registration cannot be achieved.
[0007] Morphological registration: This method uses morphological operations (such as corrosion, dilation, opening and closing operations, etc.) to register images. By performing morphological transformation on the images, the structures of the two images are aligned, thereby achieving registration. Non-rigid registration cannot be achieved.
[0008] Therefore, the registration accuracy of simple rigid registration is not enough or the registration speed is slow. In view of this, it is very meaningful to propose a face image registration method and system. Summary of the invention
[0009] In order to solve the problem that when the images of the RGB camera and the spectral camera are currently registered, hard synchronization at the time of taking pictures cannot be achieved, and the two cameras have different fields of view and distortion, so the registration accuracy of simple rigid registration is insufficient or the registration speed is slow, the present invention provides a face image registration method and system, which provides a fast and accurate registration method according to the characteristics of the face to solve the above-mentioned technical defects.
[0010] In a first aspect, the present invention provides a facial image registration method, the method comprising the following steps:
[0011] Preprocess the high-definition RGB image to obtain a high-definition RGB image containing a human face that is consistent with the pixel and size of the spectral image;
[0012] Selecting a first image to be registered from the spectral image, using a face recognition algorithm to identify and mark key points of the face in the first image to be registered and the high-definition RGB image, completing a rough alignment of the face using the acquired key points of the face, and outputting a first alignment matrix;
[0013] Further identifying matching points includes identifying the matching points by an image feature point registration method and identifying the matching points by a region registration method, finely aligning the acquired matching points, and outputting a second alignment matrix.
[0014] Preferably, the identifying the matching points by the image feature point registration method specifically includes:
[0015] Performing spatial transformation on the first image to be registered according to the first alignment matrix, and outputting a second image to be registered;
[0016] Using a feature algorithm to identify feature points of the output second image to be registered and the high-definition RGB image, and forming their own feature point sets;
[0017] A feature point pair set between the two feature point sets is further constructed, and the feature point pair set is screened according to the slope and the Euclidean distance between the two feature points in each feature point pair to obtain a first candidate matching point pair set.
[0018] Further preferably, the identifying the matching points by using a regional registration method specifically includes:
[0019] Separating the first image to be registered and the high-definition RGB image into a plurality of regions, and aligning each region to form a third alignment matrix;
[0020] The facial key points in the first image to be registered are spatially transformed according to the third alignment matrix to construct a set of key point pairs between the first image to be registered and the high-definition RGB image;
[0021] The key point pair set is screened according to the Euclidean distance between two key points in each key point pair to obtain a second candidate matching point pair set.
[0022] Further preferably, it also includes:
[0023] The face key points are used to divide the face into different areas including facial features, forehead, cheeks, and chin;
[0024] Among them, the region division and region size are optimized and adjusted according to the specific alignment effect; the region division is performed on the same horizontal plane, and the area of each region is controlled within a preset range; the alignment methods include phase alignment and image similarity alignment.
[0025] Preferably, the fine alignment of the obtained matching points and outputting a second alignment matrix includes:
[0026] The obtained first candidate matching point pair set and the second candidate matching point pair set are merged into a final matching point pair set, the geometric transformation relationship is calculated by a non-rigid alignment method, and a second alignment matrix is output.
[0027] Further preferably, the method for calculating the geometric transformation relationship by non-rigid alignment includes TPS transformation, PDE and demons algorithm.
[0028] Preferably, it also includes: performing spatial transformation on the first spectral effect graph obtained from analyzing the spectral image through the output first alignment matrix and second alignment matrix to obtain a second spectral effect graph, and then fusing the transformed second spectral effect graph with the high-definition RGB image to form a result graph.
[0029] Preferably, it also includes: extracting texture features of the high-definition RGB image to form a texture feature map; enlarging the transformed second spectral effect map to the image size of the high-definition RGB image; and superimposing the texture feature map onto the image of the second spectral effect map to form a result map.
[0030] Preferably, the first spectral effect graph includes a red graph, a brown graph, a black graph and a green graph of skin detection analysis.
[0031] Preferably, the selecting the first image to be registered from the spectral image includes: selecting the spectral image of a certain band as the first image to be registered or dividing the spectral image into three bands of R, G, and B, and then superimposing them to form a three-color image, and calculating the first image to be registered through the three-color image.
[0032] Further preferably, the method further includes: performing rigid registration with the facial key points in the first image to be registered and the high-definition RGB image to form a geometric transformation relationship, namely, a second alignment matrix.
[0033] In a second aspect, an embodiment of the present invention provides a face image registration system, comprising:
[0034] An image preprocessing module is configured to preprocess the high-definition RGB image to obtain a high-definition RGB image containing a human face that is consistent with the pixels and size of the spectral image;
[0035] A first alignment matrix module is configured to select a first image to be registered from the spectral image, use a face recognition algorithm to identify and mark facial key points in the first image to be registered and the high-definition RGB image, use the acquired facial key points to complete a rough alignment of the face, and output a first alignment matrix;
[0036] The second alignment matrix module is configured to further identify matching points, including identifying and obtaining a first set of candidate matching point pairs through an image feature point registration method and identifying and obtaining a second set of candidate matching point pairs through a region registration method, performing fine alignment on the obtained matching points, and outputting a second alignment matrix.
[0037] Preferably, it also includes: a fusion module, configured to obtain a second spectral effect graph after spatially transforming the first spectral effect graph obtained from the analysis of the spectral image through the output first alignment matrix and second alignment matrix, and then fuse the transformed second spectral effect graph with the high-definition RGB image to form a result graph.
[0038] Further preferably, in the second alignment matrix module, the identifying and obtaining the first candidate matching point pair set by the image feature point registration method specifically includes:
[0039] The first image to be registered is spatially transformed according to the first alignment matrix to output a second image to be registered; the output second image to be registered and the high-definition RGB image are respectively identified with feature points through a feature algorithm, and are respectively formed into respective feature point sets; a feature point pair set between the two feature point sets is further constructed, and the feature point pair set is screened according to the slope and the Euclidean distance between the two feature points in each feature point pair to obtain a first candidate matching point pair set.
[0040] Further preferably, in the second alignment matrix module, the identifying and obtaining the second candidate matching point pair set by the regional registration method specifically includes:
[0041] The first image to be registered and the high-definition RGB image are divided into several areas, and each area is aligned to form a third alignment matrix; the facial key points in the first image to be registered are spatially changed according to the third alignment matrix, and a set of key point pairs between the first image to be registered and the high-definition RGB image are constructed; the set of key point pairs is screened according to the Euclidean distance between the two key points in each key point pair to obtain a second set of candidate matching point pairs.
[0042] Further preferably, in the second alignment matrix module, the fine alignment of the acquired matching points and outputting a second alignment matrix includes:
[0043] The obtained first candidate matching point pair set and the second candidate matching point pair set are merged into a final matching point pair set, the geometric transformation relationship is calculated by a non-rigid alignment method, and a second alignment matrix is output.
[0044] Compared with the prior art, the beneficial results of the present invention are:
[0045] (1) Based on the characteristics of the face, the key points of the face are used for rigid coarse registration, which can quickly achieve coarse registration.
[0046] (2) Non-rigid registration is achieved by selecting matching points through feature extraction, which is combined with rigid registration to improve the accuracy of registration.
[0047] (3) Due to the uneven distribution of facial feature points, the regional registration method is used to select matching points to supplement the matching points extracted by features to achieve more accurate detail matching. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the accompanying drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.
[0049] Figure 1 A schematic diagram of a process flow of a face image registration method according to an embodiment of the present invention;
[0050] Figure 2 FIG. 4 is a schematic diagram of the architecture of a face image registration system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0053] Since the spatial resolution of multispectral cameras or hyperspectral cameras is not as high as that of RGB image sensors, a dual-camera solution is generally used to provide clarity of the final effect image. Therefore, it is necessary to align and fuse the image of the spectral camera with the high-definition RGB image.
[0054] Because when registering the images of the RGB camera and the spectral camera, it is impossible to achieve hard synchronization at the time of taking the photo, and the two cameras have different fields of view and distortion, so the registration accuracy of simple rigid registration is insufficient or the registration speed is slow. The present invention provides a fast and accurate registration method based on the characteristics of the human face.
[0055] Figure 1 The embodiment of the present invention discloses a method for facial image registration. Figure 1 As shown, the method comprises the following steps:
[0056] S1. Preprocess the high-definition RGB image to obtain a high-definition RGB image containing a human face that is consistent with the pixels and size of the spectral image;
[0057] Specifically, the high-definition RGB image is preprocessed to obtain a high-definition RGB image containing a human face that is consistent with the pixels and size of the spectral image, including:
[0058] S101, performing face detection on the high-definition RGB image and the spectrum image respectively to obtain the position, size and key points of the face on the high-definition RGB image and the spectrum image;
[0059] S102, cropping the high-definition RGB image and the spectrum image respectively, discarding non-face parts, obtaining cropped RGB image and spectrum image, and reducing the cropped RGB image to achieve a consistent pixel size with the cropped spectrum image.
[0060] S2, selecting a first image to be registered from the spectral image, using a face recognition algorithm to identify and mark facial key points in the first image to be registered and the high-definition RGB image, completing a rough alignment of the face using the acquired facial key points, and outputting a first alignment matrix;
[0061] Specifically, selecting the first image to be registered from the spectral image includes: selecting the spectral image of a certain band as the first image to be registered or dividing the spectral image into three bands of R, G, and B, and then superimposing them to form a three-color image, and calculating the first image to be registered through the three-color image.
[0062] The first image to be registered and the high-definition RGB image are subjected to a face recognition algorithm to respectively identify and mark face key points in the image; rigid registration is performed through the face key points in the image to form a geometric transformation relationship, i.e., a second alignment matrix; wherein face key point recognition can be achieved through the media pipe algorithm, the dlib algorithm or other face key point recognition algorithms.
[0063] S3. Further identifying matching points, including identifying and obtaining a first set of candidate matching point pairs through an image feature point registration method and identifying and obtaining a second set of candidate matching point pairs through a region registration method, finely aligning the obtained matching points, and outputting a second alignment matrix.
[0064] By increasing the area registration to obtain matching points, the shortcomings of the feature point registration method, such as insufficient number of matching points and uneven distribution, are improved.
[0065] Specifically, identifying and obtaining a first candidate matching point pair set by an image feature point registration method specifically includes:
[0066] S301, spatially transforming the first image to be registered according to the first alignment matrix, and outputting a second image to be registered;
[0067] S302, using a feature algorithm to identify feature points of the output second image to be registered and the high-definition RGB image, and forming their own feature point sets;
[0068] S303: further construct a feature point pair set between the two feature point sets, and screen the feature point pair set according to the slope and the Euclidean distance between two feature points in each feature point pair to obtain a first candidate matching point pair set.
[0069] The feature extraction method may be SIFT (local feature detection and extraction), SURF (speeded up robust feature detection and extraction), ORB (rapid feature point detection and extraction) or other methods.
[0070] The feature point matching method may be a Brute-force matcher, a Flann Based Matcher, or the like.
[0071] Identifying and obtaining the second candidate matching point pair set by the regional registration method specifically includes:
[0072] S311, dividing the first image to be registered and the high-definition RGB image into a plurality of regions, and aligning each region to form a third alignment matrix;
[0073] S312, spatially transforming the facial key points in the first image to be registered according to the third alignment matrix, and constructing a set of key point pairs between the first image to be registered and the high-definition RGB image;
[0074] S313: Screen the key point pair set according to the Euclidean distance between two key points in each key point pair to obtain a second candidate matching point pair set.
[0075] Furthermore, the face key points are used to divide the face, for example, into different areas such as facial features, forehead, cheeks, chin, etc. The area division and area size can be optimized and adjusted according to the specific alignment effect. It is preferred that the area division is on the same horizontal plane as much as possible, and the area of each area is controlled within a certain range to reduce the impact of depth parallax and avoid partial alignment and non-alignment in the same area, which can better improve the accuracy of registration.
[0076] The alignment method can be phase alignment, image similarity alignment, etc. Phase alignment is preferred. Due to the previous rough alignment, the images in the area are basically consistent. There are some differences in details due to distortion, shooting time, etc. Phase alignment uses high-frequency information of the image and has a fast alignment speed, which can improve the overall image registration speed.
[0077] Finely aligning the obtained matching points and outputting a second alignment matrix includes:
[0078] S321: merge the first candidate matching point pair set and the second candidate matching point pair set into a final matching point pair set, calculate the geometric transformation relationship by a non-rigid alignment method, and output a second alignment matrix.
[0079] Further preferably, the non-rigid transformation method for calculating the geometric transformation relationship by non-rigid alignment includes TPS transformation, PDE and demons algorithm
[0080] S4, performing spatial transformation on the first spectral effect graph obtained by analyzing the spectral image through the output first alignment matrix and second alignment matrix to obtain a second spectral effect graph, and then fusing the transformed second spectral effect graph with the high-definition RGB image to form a result graph.
[0081] Specifically, the transformed second spectrum effect image is fused with the high-definition RGB image to form a result image, including:
[0082] S401, performing pyramid decomposition on the transformed second spectral effect image and the high-definition RGB image to obtain a Laplacian pyramid;
[0083] S402, performing Alpha fusion on the same layer of the image on the decomposed Laplacian pyramid, wherein the Alpha value is obtained by Gaussian blurring;
[0084] S403, fusing images of different layers according to different rules, such as using a small Gaussian kernel for images with large scales on the pyramid, and using a large Gaussian kernel for images with large scales on the pyramid.
[0085] For each layer of the pyramid, fusion is performed according to the formula:
[0086]
[0087] Among them, R i Indicates the fusion factor, with a value of 0-1; Indicates the color to be merged, generally Indicates prospects, Represents the background, R i Characterizes the visibility of the foreground color in the result, R i The larger the value, the greater the proportion of the prospect in the result.
[0088] Pyramid fusion is a commonly used image fusion technology in image processing and computer vision. It is based on the idea of pyramid decomposition, decomposing the image into sub-images of different scales, then fusing them at each scale, and finally reconstructing the fused image through pyramid. Common pyramid decomposition methods include Gaussian pyramid and Laplacian pyramid.
[0089] Preferably, the texture features of the high-definition RGB image can also be extracted to form a texture feature map; the transformed spectral effect image is enlarged to the size of the RGB image; and the RGB texture feature map is superimposed on the spectral effect image to form a result image.
[0090] Among them, the spectral effect map can be a red map, a brown map, a black map, a green map, etc. for skin detection and analysis.
[0091] In a specific embodiment, the detailed method steps of the technical solution of the present invention are as follows:
[0092] 1. Change the size of the high-definition RGB image to make it consistent with the size of the spectral image:
[0093] Perform face detection on the high-definition RGB image and spectral image respectively to obtain the position, size and key points of the face on the RGB image and spectral image. Crop the RGB image and spectral image respectively, discard the non-face part, and obtain the cropped RGB image and spectral image. Reduce the cropped RGB image to achieve the same pixel size as the cropped spectral image.
[0094] 2. Perform rough alignment of the face and output alignment matrix 1:
[0095] A spectral image of a certain band is selected as the image 1 to be registered, or the spectral image is divided into three bands of R, G, and B, and superimposed to form a three-color image, and the image 1 to be registered is formed by calculating the three-color image;
[0096] The image to be registered 1 and the high-definition RGB image are subjected to a face recognition algorithm to respectively identify and mark the face key points in the image; rigid registration is performed through the face key points in the image to form a geometric transformation relationship, namely, an alignment matrix 1; wherein the face key point recognition can be implemented through the media pipe algorithm, the dlib algorithm or other face key point recognition algorithms.
[0097] 3. Identify matching points, align them precisely, and output alignment matrix 2: By adding regional registration to obtain matching points, the shortcomings of the feature point registration method, such as insufficient number of matching points and uneven distribution, are improved.
[0098] 3.1 Identify matching points through image feature point registration method:
[0099] Perform spatial transformation on the image 1 to be registered according to the alignment matrix 1, and output the image 2 to be registered;
[0100] The image 2 to be registered and the high-definition RGB image are subjected to a feature algorithm to identify feature points, a feature point pair set between the two feature point sets is constructed, and the feature point pair set is screened according to the slope and the Euclidean distance between the two feature points in each feature point pair to obtain a candidate matching point pair set 1;
[0101] The feature extraction method may be SIFT (local feature detection and extraction), SURF (speeded up robust feature detection and extraction), ORB (rapid feature point detection and extraction) or other methods.
[0102] The feature point matching method may be a Brute-force matcher, a Flann Based Matcher, or the like.
[0103] 3.2 Identify matching points through regional registration method:
[0104] The image to be registered 1 and the high-definition RGB image are divided into several regions, and each region is aligned to form an alignment matrix 3;
[0105] The face is divided into different areas such as facial features, forehead, cheeks, chin, etc. by using key points of the face. The area division and area size can be optimized and adjusted according to the specific alignment effect. It is preferred that the area division is on the same horizontal plane as much as possible, and the area of each area is controlled within a certain range to reduce the impact of depth parallax and avoid partial alignment and non-alignment in the same area, which can better improve the accuracy of registration.
[0106] The alignment method can be phase alignment, image similarity alignment, etc., and phase alignment is preferred. Due to the previous rough alignment, the images in the area are basically consistent. There are some differences in details due to distortion, shooting time, etc. Phase alignment uses high-frequency information of the image and has a fast alignment speed, which can improve the overall image registration speed.
[0107] The key points in the image 1 to be registered are spatially changed according to the alignment matrix 3 to construct a set of key point pairs between the two images; the key point pair set is screened according to the Euclidean distance between the two key points in each key point pair to obtain a set of candidate matching point pairs 2;
[0108] 3.3 The matching points obtained by the two methods are used to calculate the geometric transformation relationship through non-rigid alignment, and the alignment matrix 2 is output:
[0109] Candidate matching point pair set 1 and candidate matching point pair set 2 are merged into the final matching point pair set, and a geometric transformation relationship, namely alignment matrix 2, is formed through non-rigid transformation methods such as TPS transformation, PDE, and demons algorithm.
[0110] 4. Combine the spectrum effect image with the high-definition RGB image:
[0111] The spectral effect graph obtained from the analysis of the spectral image is realized through alignment matrix 1 and alignment matrix 2 to realize the spectral effect graph after spatial transformation. The transformed spectral effect graph is fused with the high-definition RGB image to form a result graph.
[0112] Pyramid fusion: On the Laplacian pyramid, alpha fusion is performed on the images on the same layer, where the alpha value is obtained through methods such as Gaussian blur; different layers of images are fused according to different rules, such as using a small Gaussian kernel for large-scale images on the pyramid, and a large Gaussian kernel for smaller images.
[0113] For each layer of the pyramid, fusion is performed according to the formula:
[0114]
[0115] Among them, R i Indicates the fusion factor, with a value of 0-1; Indicates the color to be merged, generally Indicates prospects, Represents the background, R i Characterizes the visibility of the foreground color in the result, R i The larger the value, the greater the proportion of the prospect in the result.
[0116] The texture features of the high-definition RGB image can also be extracted to form a texture feature map; the transformed spectral effect map is enlarged to the size of the RGB image; the RGB texture feature map is superimposed on the spectral effect image to form a result map.
[0117] The spectral effect graph can be a red graph, a brown graph, a black graph, a green graph, etc. for skin detection and analysis.
[0118] The technical solution of the present invention uses the key points of the face to perform rigid coarse registration according to the characteristics of the face, which can quickly achieve coarse registration; non-rigid registration is achieved by selecting matching points through feature extraction, which is coordinated with the rigid registration to improve the accuracy of registration; due to the uneven distribution of facial feature points, matching points are selected by regional registration to supplement the matching points extracted by feature extraction to achieve more accurate detail matching.
[0119] In a second aspect, an embodiment of the present invention further discloses a face image registration system, such as Figure 2 As shown, it includes: an image preprocessing module 21, a first alignment matrix module 22, a second alignment matrix module 23 and a fusion module 24.
[0120] In a specific embodiment, the image preprocessing module 21 is configured to preprocess the high-definition RGB image to obtain a high-definition RGB image containing a face that is consistent with the pixels and size of the spectral image; the first alignment matrix module 22 is configured to select a first image to be registered from the spectral image, and use a face recognition algorithm to identify and mark facial key points in the first image to be registered and the high-definition RGB image, respectively, use the acquired facial key points to complete the rough alignment of the face, and output the first alignment matrix; the second alignment matrix module 23 is configured to further identify matching points, including identifying the matching points by an image feature point registration method and identifying the matching points by a regional registration method, finely aligning the acquired matching points, and outputting a second alignment matrix;
[0121] It also includes: a fusion module 24, which is configured to obtain a second spectral effect graph after spatially transforming the first spectral effect graph obtained from the analysis of the spectral image through the output first alignment matrix and second alignment matrix, and then fuse the transformed second spectral effect graph with the high-definition RGB image to form a result image.
[0122] Specifically, in the second alignment matrix module 23, the identifying the matching points through the image feature point registration method specifically includes: spatially transforming the first image to be registered according to the first alignment matrix, and outputting the second image to be registered; using the output second image to be registered and the high-definition RGB image through a feature algorithm to respectively identify feature points and respectively form respective feature point sets; further constructing a feature point pair set between the two feature point sets, and screening the feature point pair set according to the slope and Euclidean distance between the two feature points in each feature point pair to obtain a first candidate matching point pair set.
[0123] In the second alignment matrix module 23, the identifying of the matching points by the image feature point registration method specifically includes: dividing the first image to be registered and the high-definition RGB image into a number of areas, aligning each area separately to form a third alignment matrix; spatially changing the facial key points in the first image to be registered according to the third alignment matrix, and constructing a set of key point pairs between the first image to be registered and the high-definition RGB image; screening the set of key point pairs according to the Euclidean distance between the two key points in each of the key point pairs to obtain a second set of candidate matching point pairs.
[0124] In the second alignment matrix module 23, the obtained matching points are finely aligned and a second alignment matrix is output, including: merging the obtained first candidate matching point pair set and the second candidate matching point pair set into a final matching point pair set, calculating the geometric transformation relationship through a non-rigid alignment method, and outputting a second alignment matrix.
[0125] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.
Claims
1. A facial image registration method, characterized in that: The method comprises the following steps: Preprocess the high-definition RGB image to obtain a high-definition RGB image containing a human face that is consistent with the pixel and size of the spectral image; Selecting a first image to be registered from the spectral image, using a face recognition algorithm to identify and mark key points of the face in the first image to be registered and the high-definition RGB image, completing a rough alignment of the face using the acquired key points of the face, and outputting a first alignment matrix; Further identifying matching points includes identifying and obtaining a first set of candidate matching point pairs through an image feature point registration method and identifying and obtaining a second set of candidate matching point pairs through a region registration method, finely aligning the obtained matching points, and outputting a second alignment matrix.
2. The facial image registration method according to claim 1, characterized in that: The method of identifying and obtaining a first candidate matching point pair set by using an image feature point registration method specifically includes: Performing spatial transformation on the first image to be registered according to the first alignment matrix, and outputting a second image to be registered; Using a feature algorithm to identify feature points of the output second image to be registered and the high-definition RGB image, and forming their own feature point sets; A feature point pair set between the two feature point sets is further constructed, and the feature point pair set is screened according to the slope and the Euclidean distance between the two feature points in each feature point pair to obtain a first candidate matching point pair set.
3. The facial image registration method according to claim 1, characterized in that: The identifying and obtaining the second candidate matching point pair set by the regional registration method specifically includes: Separating the first image to be registered and the high-definition RGB image into a plurality of regions, and aligning each region to form a third alignment matrix; The facial key points in the first image to be registered are spatially transformed according to the third alignment matrix to construct a set of key point pairs between the first image to be registered and the high-definition RGB image; The key point pair set is screened according to the Euclidean distance between two key points in each key point pair to obtain a second candidate matching point pair set.
4. The facial image registration method according to claim 3, characterized in that: Also includes: Using the facial key points to separate the first image to be registered and the high-definition RGB image into different areas including facial features, forehead, cheeks, and chin; Among them, the region division and region size are optimized and adjusted according to the specific alignment effect; the region division is performed on the same horizontal plane, and the area of each region is controlled within a preset range; the alignment methods include phase alignment and image similarity alignment.
5. The facial image registration method according to claim 1, characterized in that: The fine alignment of the obtained matching points and outputting a second alignment matrix includes: The obtained first candidate matching point pair set and the second candidate matching point pair set are merged into a final matching point pair set, the geometric transformation relationship is calculated by a non-rigid alignment method, and a second alignment matrix is output.
6. The facial image registration method according to claim 5, characterized in that: The method for calculating the geometric transformation relationship by non-rigid alignment includes TPS transformation, PDE and demons algorithm.
7. The facial image registration method according to claim 1, characterized in that: Also includes: The first spectral effect graph obtained by analyzing the spectral image is spatially transformed through the output first alignment matrix and second alignment matrix to obtain a second spectral effect graph, and then the transformed second spectral effect graph is fused with the high-definition RGB image to form a result graph.
8. The facial image registration method according to claim 7, characterized in that: Also includes: Extracting texture features of the high-definition RGB image to form a texture feature map; Enlarging the transformed second spectrum effect image to the image size of the high-definition RGB image; The texture feature map is superimposed on the image of the second spectral effect map to form a result map.
9. The facial image registration method according to claim 7, characterized in that: The first spectral effect graph includes a red graph, a brown graph, a black graph and a green graph of skin detection analysis.
10. The facial image registration method according to claim 1, characterized in that: The selecting the first image to be registered from the spectral image includes: selecting the spectral image of a certain band as the first image to be registered or dividing the spectral image into three bands of R, G, and B, and then superimposing them to form a three-color image, and calculating the first image to be registered through the three-color image.
11. A facial image registration system, characterized in that: include: An image preprocessing module is configured to preprocess the high-definition RGB image to obtain a high-definition RGB image containing a human face that is consistent with the pixels and size of the spectral image; A first alignment matrix module is configured to select a first image to be registered from the spectral image, use a face recognition algorithm to identify and mark facial key points in the first image to be registered and the high-definition RGB image, use the acquired facial key points to complete a rough alignment of the face, and output a first alignment matrix; The second alignment matrix module is configured to further identify matching points, including identifying and obtaining a first set of candidate matching point pairs through an image feature point registration method and identifying and obtaining a second set of candidate matching point pairs through a region registration method, performing fine alignment on the obtained matching points, and outputting a second alignment matrix.
12. The facial image registration system according to claim 11, characterized in that: Also includes: A fusion module is configured to obtain a second spectral effect graph after spatially transforming the first spectral effect graph obtained from the analysis of the spectral image through the output first alignment matrix and second alignment matrix, and then fuse the transformed second spectral effect graph with the high-definition RGB image to form a result graph.
13. The facial image registration system according to claim 11, characterized in that: In the second alignment matrix module, the step of identifying and obtaining a first candidate matching point pair set by using an image feature point registration method specifically includes: The first image to be registered is spatially transformed according to the first alignment matrix to output a second image to be registered; the output second image to be registered and the high-definition RGB image are respectively identified with feature points through a feature algorithm, and are respectively formed into respective feature point sets; a feature point pair set between the two feature point sets is further constructed, and the feature point pair set is screened according to the slope and the Euclidean distance between the two feature points in each feature point pair to obtain a first candidate matching point pair set.
14. The facial image registration system according to claim 11, characterized in that: In the second alignment matrix module, the step of identifying and obtaining a second candidate matching point pair set by using a regional registration method specifically includes: The first image to be registered and the high-definition RGB image are divided into several areas, and each area is aligned to form a third alignment matrix; the facial key points in the first image to be registered are spatially changed according to the third alignment matrix, and a set of key point pairs between the first image to be registered and the high-definition RGB image are constructed; the set of key point pairs is screened according to the Euclidean distance between the two key points in each key point pair to obtain a second set of candidate matching point pairs.
15. The facial image registration system according to claim 11, characterized in that: In the second alignment matrix module, fine alignment is performed on the obtained matching points, and a second alignment matrix is output, including: The obtained first candidate matching point pair set and the second candidate matching point pair set are merged into a final matching point pair set, the geometric transformation relationship is calculated by a non-rigid alignment method, and a second alignment matrix is output.