Ultra-high-definition panoramic image registration technology for the AIT process of a manned spacecraft

Abstract

The traditional SIFT registration algorithm easily identifies feature points on the same small targets at different positions as the same feature point, resulting in feature point matching errors, which affects the results of panoramic image stitching. To solve this problem, the present invention proposes an ultra-high-definition panoramic image registration technology for the AIT process of manned spacecraft. During feature registration, the perceptual hash algorithm (PHA) is used to roughly purify the matching point pairs, and then it is determined whether the matching points are on the specific target, and the matching points on the specific target are removed, which solves the problem of registration errors during feature registration. The specific steps are as follows: S1: Using the original data collected from a single site to construct a training sample set for specific target detection; S2: Training a detection model based on a Haar feature cascade classifier; S3: Determining image matching point pairs based on the SIFT method; S4: Using the perceptual hash algorithm to roughly purify the matching point pairs; S5: Removing the matching point pairs on the specific target based on the detection model; S6: Constructing an image transformation model.

Description

Technical Field

[0001] The present invention relates to an ultra-high-definition panoramic image registration technology for an AIT process of a manned spacecraft, and belongs to the technical field of digital image processing. Background Art

[0002] Image registration technology is a core component of panoramic image stitching. Panoramic image registration includes methods based on image grayscale, transform domain, and image feature. Feature-based registration methods first select feature information from the image and then search and match the feature set of overlapping image regions using the image features as a criterion. Image registration algorithms based on scale-invariant feature transform (SIFT) are not only invariant to rotation, scaling, and brightness changes, but also maintain stability to perspective changes, affine transformations, and noise. Therefore, they are widely used in panoramic image registration.

[0003] However, traditional SIFT registration algorithms suffer from feature misregistration. Images taken outside manned spacecraft cabins often contain identical targets in different locations. These targets, such as small logos and small tooling, have high contrast between the target and the background and are small in size. Traditional SIFT registration algorithms can easily misidentify feature points on the same small target in different locations as the same feature point, leading to feature point mismatching and affecting the panoramic image stitching results. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to overcome the shortcomings of the existing technology and propose an ultra-high-definition panoramic image registration technology for the AIT process of manned spacecraft. During feature registration, the perceptual hash algorithm (PHA) is used to roughly purify the matching point pairs, and then it is determined whether the matching points are on a specific target, and the matching points on the specific target are removed, thereby solving the problem of registration errors during feature registration.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] An ultra-high-definition panoramic image registration technology for manned spacecraft AIT process includes the following steps:

[0007] S1: Use the original data collected from a single site to build a training sample set for specific target detection.

[0008] S2: Detection model training based on Haar feature cascade classifier.

[0009] S3: Determine image matching point pairs based on the SIFT method.

[0010] S4: Use the perceptual hashing algorithm to roughly purify the matching point pairs.

[0011] S5: Remove matching point pairs on specific targets based on the detection model.

[0012] S6: Build an image transformation model.

[0013] S1: Constructing a training sample set for specific target detection using original images collected at a single site. Step S1 specifically includes steps S11 and S12.

[0014] S11: Determine the existence of specific targets with the same target at different locations based on the structure outside the manned spacecraft cabin.

[0015] S12: Constructing a training sample set for specific target detection

[0016] The training sample set consists of a positive sample set and a negative sample set, constructed using original images collected at a single site. Assuming the scale of the image to be stitched is M × N, the front view of each specific target is intercepted from the image sequence to be stitched and each front view is reduced by a factor of c, resulting in a reduced size of u × u.

[0017] The specific targets in this invention are absolutely rigid objects. Therefore, the positive sample set is expanded by performing three-dimensional rotation and brightness changes on a single front view of each specific target. The rotation angle range of the target during three-dimensional rotation is determined based on the state of the specific target on the manned spacecraft. T positive sample sets are constructed for each specific target by randomly varying the three-dimensional rotation angle and brightness of the front view.

[0018] T negative sample sets are formed by randomly selecting areas that do not contain specific targets from the image sequence to be stitched, and each negative sample set is reduced by c times to a size of w×w (w>u).

[0019] S2: Detection model training based on Haar feature cascade classifier

[0020] Step S2 utilizes a cascade classifier based on Haar features to train a detection model. The present invention utilizes an integral graph method to calculate Haar feature values, utilizes an Adaboost algorithm to train each weak classifier, and then combines each weak classifier according to a specific combination strategy to obtain a strong classifier. The cascade classifier is obtained by training multiple strong classifiers and combining them in a cascade manner.

[0021] The specific targets of the present invention include σ targets. Three strong classifiers are trained using the positive and negative sample sets of each specific target, resulting in a total of 3σ strong classifiers. These 3σ strong classifiers are cascaded together to form the final classifier. First, strong classifiers 1, 2, and 3 are used to detect the first specific target. Strong classifiers 4, 5, and 6 are used to detect the second specific target in the detected non-target area. This process is repeated to complete the detection of all σ specific targets.

[0022] S3: Determine image matching point pairs based on SIFT method

[0023] The step S3 includes step S31 and step S32.

[0024] S31: Feature Detection

[0025] The step S31 obtains the feature points and feature descriptors of all images in the image sequence to be spliced based on the SIFT method.

[0026] S32: Feature Registration

[0027] In step S32, the Best-Bin-First (BBF) algorithm is used to perform feature registration to obtain image matching point pairs, and all matching point pairs of the image sequence to be spliced constitute an image matching point pair dataset {X1, X2, ..., X n-1 ,X n}, where n represents the number of matching point pairs. i ={p,q,x p ,y p ,x q ,y q} represents the information of the i-th matching point pair, where p represents the serial number of the image where the first matching point in the matching point pair is located, and x p and y p Indicates the position coordinates of the first matching point on the image, q indicates the serial number of the image where the second matching point in the matching point pair is located, and x q and y q Indicates the position coordinates of the second matching point on the image.

[0028] S4: Use the perceptual hash algorithm to roughly purify the matching point pairs

[0029] Step S4 performs rough purification on each matching point pair, extracts the image of the area around the matching point pair, and uses the perceptual hash algorithm to calculate the similarity of the images of the area around the two matching points. When the similarity is less than the set threshold, the corresponding matching point pair is removed.

[0030] S5: Remove matching point pairs on specific targets based on the detection model

[0031] The step S5 includes step S51, step S52 and step S53. Figure 5 Shown is a flowchart for removing matching point pairs on specific targets based on the detection model.

[0032] S51: intercept the area near the matching point as the detection area

[0033] The present invention detects the image where the first matching point in each matching point pair is located. Since the purpose of detection is to determine whether the matching point is on a specific target, it is only necessary to detect the image in the area near the matching point. The original image in the area near the matching point is intercepted as the detection area. The position coordinates of the center point of the detection area are consistent with the position coordinates of the matching point. The position coordinates of the center point of the detection area are (x p ,y p Since the size of the small extravehicular spacecraft target is less than 1 / 10 of the size of the image to be stitched, the width of the detection area is set to M / 5 and the height is set to N / 5.

[0034] S52: Detecting specific targets using cascade classifiers

[0035] A sliding sub-window is set in the detection area, and each sliding sub-window is detected using a trained cascade classifier to determine whether there is a specific target in the sliding sub-window.

[0036] S53: Remove matching point pairs on specific targets

[0037] If a specific target is detected in the detection area, the positional relationship between the matching point and each specific target is determined. Assume that the number of specific targets in the detection area is K, and the position coordinates of the center point of the kth specific target are (x k ,y k ), width is w k , height is h k If there exists k, satisfying and If the matching point pair is on the specific target, the matching point pair is removed from the matching point pair dataset. If the specific target is not detected in the detection area or the matching point is not on the detected specific target, the corresponding matching point pair is retained. The optimized feature matching point pair dataset is represented as {X1,X2,...,X m-1 ,X m}, where m≤n.

[0038] S6: Building an image transformation model

[0039] The step S6 constructs an image transformation model using feature matching point pairs of the dataset {X1, X2, ..., X m-1 ,X mDetermine the transformation relationship of the images involved in the matching, transform them into the same coordinate system, and solve to obtain the image transformation matrix. In constructing the image transformation model, the present invention adopts the Random Sample Consensus (RANSAC) algorithm to remove mismatched point pairs.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The present invention addresses the problem of misregistration of spacecraft panoramic images. During feature matching, the perceptual hashing algorithm is used to roughly purify matching point pairs, thereby improving the accuracy of feature matching and ensuring the effect of panoramic image stitching.

[0042] (2) This invention addresses the problem of identical targets at different locations in spacecraft panoramic image registration. By using a cascade classifier to identify specific targets and remove matching point pairs on the specific targets, the present invention reduces the problem of panoramic image registration errors and improves the accuracy of ultra-high-definition panoramic image stitching for manned spacecraft. Because specific targets outside the spacecraft cabin are small in size, ignoring matching points on the specific targets will not result in an insufficient number of feature matching point pairs.

[0043] (3) In the present invention, specific target recognition only determines whether the matching point pair is on the specific target. Therefore, only the image within a certain neighborhood range of the matching point pair is detected, and there is no need to detect the entire image. This has the advantage of high target detection operation efficiency.

[0044] (4) The present invention uses a data enhancement method to increase the number of positive samples in the training set when training the specific target classifier, which has the advantage of a simple training set acquisition method.

[0045] (5) During the AIT process of manned spacecraft, data from multiple development stages must be collected. For each stage, panoramic images must be collected from multiple sites. Therefore, dozens of panoramic image stitching operations must be performed during the final assembly and testing phase at the launch site. The present invention utilizes raw data collected from a single site to construct a training sample set for specific target detection, trains a specific target detection model, and applies the detection model to each panoramic image stitching process. This eliminates the need to retrain the detection model for each stitching operation, resulting in an easy-to-use training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Flowchart of the panoramic image registration technology of the present invention;

[0047] Figure 2 Schematic diagram of the small sign outside the manned spacecraft of the present invention;

[0048] Figure 3 Schematic diagram of the small extravehicular tooling for a manned spacecraft according to the present invention;

[0049] Figure 4 The cascade classifier detection model of the present invention;

[0050] Figure 5 The present invention is a flow chart for removing matching point pairs on a specific target based on a detection model. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. In the images outside the cabin of a manned spacecraft, there are situations where targets at different positions are consistent. The traditional SIFT feature matching algorithm easily identifies feature points on the same small targets at different positions as the same feature point, resulting in errors in panoramic image registration. To solve the above problems, the present invention uses a perceptual hashing algorithm to roughly purify the matching point pairs during feature registration, and then determines whether the matching points are on a specific target, and removes the matching points on the specific target. During the AIT process of a manned spacecraft, data from multiple development stages need to be collected. The panoramic images of each stage collect panoramic data from multiple sites. Therefore, dozens of panoramic image splicings need to be performed during the final assembly and testing phase of the launch site. The present invention uses the original data collected from a single site to construct a model for specific target detection. The target detection model is applicable to each panoramic image splicing process.

[0052] A manned spacecraft AIT process ultra-high-definition panoramic image registration technology includes the following steps: S1: Use the original data collected by a single site to build a training sample set for specific target detection. S2: Train the detection model based on the Haar feature cascade classifier. S3: Determine the image matching point pairs based on the SIFT method. S4: Use the perceptual hashing algorithm to roughly purify the matching point pairs. S5: Remove the matching point pairs on the specific target based on the detection model. S6: Build an image transformation model. Figure 1 Shown is a flow chart of the panoramic image registration technology of the present invention.

[0053] S1: Construct a training sample set for specific target detection using original images collected from a single site

[0054] The step S1 specifically includes step S11 and step S12.

[0055] S11: Determine the existence of specific targets with the same target at different locations based on the structure outside the manned spacecraft cabin.

[0056] The step S11 determines that there are specific targets with the same target at different locations based on the structure outside the manned spacecraft cabin. Figure 2 and Figure 3 Shown are specific targets outside the manned spacecraft, including Figure 2 This is a schematic diagram of the small logo outside the manned spacecraft. Figure 3 This is a schematic diagram of the small extravehicular tooling for a manned spacecraft.

[0057] S12: Constructing a training sample set for specific target detection

[0058] The training sample set consists of positive and negative samples, constructed using original images collected at a single site. Assuming the image to be stitched is 8688×5792, the front view of a small extravehicular sign and a small extravehicular tooling device are captured from the image sequence to be stitched. The front view is then scaled down by a factor of 10, to a size of 80×80.

[0059] The specific target of the present invention is an absolutely rigid object, so the positive sample set is expanded by performing three-dimensional rotation and changing the brightness of a front view of the specific target. The rotation angle range of the target during three-dimensional rotation is determined based on the state of the specific target on the manned spacecraft. In this embodiment, the angle of the small extravehicular sign in the Z-axis direction is 0 degrees, 90 degrees, 180 degrees, and 270 degrees, the rotation angle in the X-axis direction is 0-90 degrees, 270 degrees to 360 degrees, and the rotation angle in the Y-axis direction is 0-90 degrees, 270 degrees to 360 degrees. The angle of the small tooling in the Z-axis direction is 0 degrees and 180 degrees, the rotation angle in the X-axis direction is 0-90 degrees, 270 degrees to 360 degrees, and the rotation angle in the Y-axis direction is 0-90 degrees, 270 degrees to 360 degrees. By randomly changing the three-dimensional rotation angle and brightness of the front view, a positive sample set of 500 small extravehicular signs and 500 small extravehicular tooling positive sample sets are constructed.

[0060] 500 negative sample sets are formed by randomly selecting areas that do not contain small extravehicular equipment and small extravehicular signs from the image sequence to be stitched, and the image of each negative sample set is reduced by 10 times to a size of 200×200.

[0061] S2: Detection model training based on Haar feature cascade classifier

[0062] Step S2 utilizes a cascade classifier based on Haar features to train a detection model. The present invention utilizes an integral graph method to calculate Haar feature values, utilizes an Adaboost algorithm to train each weak classifier, and then combines each weak classifier according to a specific combination strategy to obtain a strong classifier. The cascade classifier is obtained by training multiple strong classifiers and combining them in a cascade manner.

[0063] In this embodiment, the specific target includes two targets. Strong classifiers 1, 2, and 3 are trained with the positive sample set and negative sample set of the small extravehicular markers. Strong classifiers 4, 5, and 6 are trained with the positive sample set and negative sample set of the small extravehicular tooling. Figure 4The figure shows the detection model of the cascade classifier. Six strong classifiers are cascaded together to form the final classifier. First, strong classifiers 1, 2, and 3 are used to detect small extravehicular signs. Strong classifiers 4, 5, and 6 are used to detect small extravehicular tooling in the detected non-small sign target areas.

[0064] S3: Determine image matching point pairs based on SIFT method

[0065] The step S3 includes step S31 and step S32.

[0066] S31: Feature Detection

[0067] The step S31 obtains the feature points and feature descriptors of all images in the image sequence to be spliced based on the SIFT method.

[0068] S32: Feature Registration

[0069] In step S32, the Best-Bin-First (BBF) algorithm is used to perform feature registration to obtain image matching point pairs, and all matching point pairs of the image sequence to be spliced constitute an image matching point pair dataset {X1, X2, ..., X n-1 ,X n}, where n represents the number of matching point pairs. i ={p,q,x p ,y p ,x q ,y q} represents the information of the i-th matching point pair, where p represents the serial number of the image where the first matching point in the matching point pair is located, and x p and y p Indicates the position coordinates of the first matching point on the image, q indicates the serial number of the image where the second matching point in the matching point pair is located, and x q and y q Indicates the position coordinates of the second matching point on the image.

[0070] S4: Use the perceptual hash algorithm to roughly purify the matching point pairs

[0071] Step S4 performs a rough refinement on each matching point pair, extracts the image of the area around the matching point pair, and uses the perceptual hashing algorithm to calculate the similarity of the images of the area around the two matching points. When the similarity is less than a set threshold, the corresponding matching point pair is removed. Preferably, the similarity threshold is set to 0.8.

[0072] S5: Remove matching point pairs on specific targets based on the detection model

[0073] The step S5 includes step S51, step S52 and step S53. Figure 5 Shown is a flowchart for removing matching point pairs on specific targets based on the detection model.

[0074] S51: intercept the area near the matching point as the detection area

[0075] The present invention detects the image where the first matching point in each matching point pair is located. Since the purpose of detection is to determine whether the matching point is on a specific target, it is only necessary to detect the image in the area near the matching point. The original image in the area near the matching point is intercepted as the detection area. The position coordinates of the center point of the detection area are consistent with the position coordinates of the matching point. The position coordinates of the center point of the detection area are (x p ,y p Since the size of the small extravehicular spacecraft target is less than 1 / 10 of the size of the image to be stitched, the width of the detection area is set to 1738 and the height is set to 1158.

[0076] S52: Detecting specific targets using cascade classifiers

[0077] A sliding sub-window is set in the detection area, and each sliding sub-window is detected using a trained cascade classifier to determine whether there is a specific target in the sliding sub-window.

[0078] S53: Remove matching point pairs on specific targets

[0079] If a specific target is detected in the detection area, the positional relationship between the matching point and each specific target is determined. Assume that the number of specific targets in the detection area is K, and the position coordinates of the center point of the kth specific target are (x k ,y k ), width is w k , height is h k If there exists k, satisfying and If the matching point pair is on the specific target, the matching point pair is removed from the matching point pair dataset. If the specific target is not detected in the detection area or the matching point is not on the detected specific target, the corresponding matching point pair is retained.

[0080] S6: Building an image transformation model

[0081] The step S6 constructs an image transformation model using feature matching point pairs of the dataset {X1, X2, ..., X m-1 ,X m Determine the transformation relationship of the images involved in the matching, transform them into the same coordinate system, and solve to obtain the image transformation matrix. In constructing the image transformation model, the present invention adopts the Random Sample Consensus (RANSAC) algorithm to remove mismatched point pairs.

[0082] Although the present invention has been disclosed above in terms of preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solutions of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the scope of protection of the technical solutions of the present invention.

Claims

1. A technology for ultra-high-definition panoramic image registration during the AIT process of a manned spacecraft, characterized by: The steps include: S1: Use the original data collected at a single site to build a training sample set for specific target detection; Said S1: specifically comprises step S11 and step S12; S11: Determine the existence of specific targets with the same target at different locations based on the structure outside the manned spacecraft cabin; S12: Construct a training sample set for specific target detection; The training sample set consists of a positive sample set and a negative sample set. The training sample set is constructed using original images collected from a single site. Assuming that the scale of the image to be stitched is M×N, the front view of each specific target is intercepted from the image sequence to be stitched, and each front view is reduced by c times, and the reduced size is u×u; A positive sample set is expanded by performing three-dimensional rotation and brightness changes on one front view of each specific target. The rotation angle range of the target during three-dimensional rotation is determined according to the state of the specific target on the manned spacecraft. T positive sample sets are constructed for each specific target by randomly changing the three-dimensional rotation angle and brightness of the front view. T negative sample sets are constructed by randomly selecting areas that do not contain the specific target from the image sequence to be stitched, and each negative sample set is reduced by a factor of c to a size of w×w after reduction, where w>u; S2: Detection model training based on Haar feature cascade classifier; Said S2: specifically, a detection model is trained using a cascade classifier based on Haar features, Haar feature values are calculated using the integral graph method, each weak classifier is trained using the Adaboost algorithm, and then each weak classifier is combined to obtain a strong classifier according to a combination strategy. A cascade classifier is obtained by training multiple strong classifiers and combining them in a cascade manner. The specific target includes σ targets, and 3 strong classifiers are trained respectively with the positive sample set and the negative sample set of each specific target. A total of 3σ strong classifiers are trained, and the 3σ strong classifiers are cascaded together to form a final classifier. First, strong classifiers 1, 2 and 3 are used to detect the first specific target. The detected non-target area is detected using strong classifiers 4, 5 and 6 to detect the second specific target. And so on to complete the detection of σ specific targets; S3: Determine image matching point pairs based on SIFT method; Said S3: comprises step S31 and step S32; S31: Feature Detection The step S31 obtains the feature points and feature descriptors of all images in the image sequence to be spliced based on the SIFT method; S32: Feature Registration In step S32, the Best-Bin-First (BBF) algorithm is used to perform feature registration to obtain image matching point pairs, and all matching point pairs of the image sequence to be spliced constitute an image matching point pair dataset {X1, X2, ..., X n-1 ,X n }, where n represents the number of matching point pairs, X i ={p,q,x p ,y p ,x q ,y q } represents the information of the i-th matching point pair, where p represents the serial number of the image where the first matching point in the matching point pair is located, and x p and y p Indicates the position coordinates of the first matching point on the image, q indicates the serial number of the image where the second matching point in the matching point pair is located, and x q and y q Indicates the position coordinates of the second matching point on the image; S4: Use the perceptual hash algorithm (PHA) to roughly purify the matching point pairs; S4: specifically, performing rough purification on each matching point pair, extracting an image of the area around the matching point pair, and calculating the similarity of the images of the area around the two matching points using a perceptual hash algorithm. When the similarity is less than a set threshold, the corresponding matching point pair is removed; S5: Remove matching point pairs on specific targets based on the detection model; Said S5: comprises step S51, step S52 and step S53; S51: intercept the area near the matching point as the detection area The image where the first matching point of each matching point pair is located is detected. Since the purpose of detection is to determine whether the matching point is on a specific target, it is only necessary to detect the image in the area near the matching point. The original image in the area near the matching point is intercepted as the detection area. The position coordinates of the center point of the detection area are consistent with the position coordinates of the matching point. The position coordinates of the center point of the detection area are (x p ,y p ); S52: Detecting specific targets using cascade classifiers Set a sliding sub-window in the detection area, and use the trained cascade classifier to detect each sliding sub-window to determine whether there is a specific target in the sliding sub-window; S53: Remove matching point pairs on specific targets If a specific target is detected in the detection area, the position relationship between the matching point and each specific target is determined. Assuming that the number of specific targets in the detection area is K, the position coordinates of the center point of the kth specific target are (x k ,y k ), width is w k , height is h k , if there exists k that satisfies and If the matching point pair is on the specific target, the matching point pair is removed from the matching point pair dataset. If the specific target is not detected in the detection area or the matching point is not on the detected specific target, the corresponding matching point pair is retained; S6: Construct image transformation model; Said S6: specifically, determines the transformation relationship of the images involved in the matching through the feature matching point pairs, transforms them into the same coordinate system, and solves to obtain the image transformation matrix.

2. The ultra-high-definition panoramic image registration technology for the AIT process of manned spacecraft according to claim 1 is characterized in that: In constructing the image transformation model, the Random Sample Consensus (RANSAC) algorithm is used to remove mismatched point pairs.

Images