An image stitching method used in underground coal mine tunnels
Through the combination of SIFT algorithm and directed line segment model, the problem of mismatch of feature points caused by complex environment and uneven lighting in the underground tunnel of coal mine is solved, and efficient and accurate splicing of the underground tunnel of coal mine is achieved, improving the splicing effect and real-time performance .
Patent Information
- Application Number
- CN202110338716.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-03-30
AI Technical Summary
The underground tunnel environment of coal mines is complex, the lighting is uneven, the feature points are mismatched, and projection distortion is prone to occur during image stitching.
The SIFT algorithm is used to extract and match features, construct a directed line segment model and eliminate false matching points, and perform secondary culling with probability statistical models. Finally, the AANAP algorithm is used to align and fuse the image with weighted average method.
It realizes stronger mismatch and removal capabilities, improves the accuracy and real-timeness of image stitching, obtains a more accurate image transformation model, and has faster running time, which is suitable for image stitching in underground tunnels of coal mines.
Smart Images

Figure CN112862692B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an image stitching method applied to underground coal mine tunnels, and is particularly suitable for use in underground coal mines, belonging to the field of computer vision. Background Art
[0002] In recent years, with the rapid integration and development of big data, cloud computing and artificial intelligence, as well as breakthroughs in related software and hardware, intelligent coal mines have become a major strategic direction for the development of the coal industry. The environment of underground coal mines is harsh, and information is mostly obtained manually. Videos and images, as an important carrier of environmental information, have gradually been used in the daily production and life of coal mines. Image stitching of underground coal mine tunnels can obtain a panoramic view of the tunnels, and then obtain the spatial environment information of underground coal mine tunnels, which has certain guiding significance for the spatial reconstruction of coal mine tunnels, personnel prevention and control, hazard identification and early warning, and coal mining.
[0003] The core and key of image stitching method is image registration, and the quality of image registration results depends on the accuracy of image feature matching points. At the same time, the lighting in coal mine tunnels is uneven, and the existing feature extraction algorithm is prone to a certain amount of mismatching points. Therefore, a good mismatch elimination method is of great significance to the image registration results. Many researchers have done a lot of work on image stitching methods and mismatch elimination methods for coal mine tunnels.
[0004] Literature (Tao Zhihui. Research on video stitching algorithm of coal mine working face [D]. Xi'an University of Science and Technology, 2020.) Moravec corner detection and PCA principal component analysis are used to optimize the SIFT feature extraction process, and the matching feature points are screened through the feature approximate nearest neighbor matching algorithm and the improved RANSAC algorithm to obtain the image transformation model, and the fade-in and fade-out fusion method is used to realize image stitching; Literature (Cheng Lin, Cai Xiaowan, Zhang Guoyong. Algorithm research on image stitching in coal mines [J]. Coal Mine Electromechanical, 2011) CSS corner detection algorithm is used to extract image feature points, feature point matching points are extracted through the normalized mutual correlation coefficient (NCC), and RANSAC is used to delete mismatches. Points are matched to complete image stitching; the literature (Yang Xuanxuan, Liu Jingyong, Yanfen, et al. Large parallax image stitching algorithm for underground tunnels [J]. Industrial and Mining Automation, 2020.) uses the SIFT algorithm to detect and match feature points, groups feature matching points based on multiple planes in the image and calculates the corresponding homography matrix, and finally uses the stitching line to align the images to synthesize the stitched image to complete the image stitching; the literature (Guan Zenglun, Gu Jun, Zhao Guangyuan. Underground video stitching algorithm based on improved accelerated robust features [J]. Industrial and Mining Automation, 2018.) uses the improved accelerated robust feature algorithm to extract feature points of video images, dynamically track the number of feature points, select the homography matrix according to the number of feature points, and then use the gradual entry and exit The weighted average method is used to fuse each frame of the image to complete the video stitching; the literature (Wang Yan, Xiong Feixue. Downhole image stitching method combined with Retinex enhancement [J]. Journal of Liaoning Technical University (Natural Science Edition), 2015.) uses local bilateral filtering to improve the Retinex algorithm to enhance the image, then uses the SURF algorithm for feature extraction, and uses the RANSAC algorithm to perform feature point precise matching, and then obtains the transformation matrix, and finally uses linear gradient fusion to complete the image stitching; the literature (Wang Yan, Zhao Jingyu, Zhao Yanguang. Downhole image stitching based on image enhancement [J]. Computer Engineering and Applications, 2016.) uses the local bilateral filtering algorithm to enhance the image Line enhancement, then use SURF algorithm for feature extraction, use the nearest distance to the next nearest distance to accurately match the feature points, and then obtain the transformation matrix, and finally use linear gradient to fuse the images and complete the image stitching; Literature (Zhao Yanguang. Research on the application of image stitching technology in digital mines [D]. Liaoning University of Engineering and Technology, 2014.) First use SURF algorithm to extract feature points, then use cosine similarity and clarity to remove feature points with small similarity, and use Euclidean distance for feature matching, and finally use bilinear interpolation and weighted average method to process the image to get the final result; Literature (Zhang Baolong. Research on coal mine image stitching technology based on feature points [J]. Coal Mine Machinery, 2012.)Use the curvature scale space to improve the adaptive threshold CSS corner detection algorithm to extract feature points, use the similarity measure NCC to extract feature point matching points, use RANSAC to delete the wrong matching point pairs, use the fade-in and fade-out to merge the images, and realize image stitching; Literature (Liang Yu, Li Dan, Niu Cuixi, Zhang Guoyong. Research on SIFT Algorithm Based on Underground Environment [J]. Industrial and Mining Automation, 2011.) Use SIFT to construct and match feature vectors for images preprocessed by Curvelt denoising, and then use the RANSAC method to remove the wrong matching points, obtain the image transformation model, and complete the image stitching. . Summary of the invention
[0005] In view of the shortcomings of the above-mentioned technologies, an image stitching method applied to underground mine tunnels is provided to solve the problems of complex environment in underground mine tunnels, uneven lighting, a large number of mismatched feature points, and projection distortion that is easy to occur during image stitching.
[0006] In order to solve the above technical problems, the present invention provides an image stitching method applied to underground coal mine tunnels, and the specific steps are as follows:
[0007] Step 1) Using the SIFT (Scale Invariant Feature Transform) algorithm, feature extraction and matching are performed between the reference image and the image to be registered, where the two images are adjacent and partially overlapped, to obtain a set of coarse matching point pairs between the reference image and the image to be registered;
[0008] Step 2) constructing a directed line segment model using feature points corresponding to the coarse matching point pairs between the reference image and the image to be registered, and constructing a slope threshold interval and a length constraint model through the directed line segment model;
[0009] Step 3) integrating the slope threshold interval and the length constraint model, and removing the mismatched point pairs in the rough matching point pairs, thereby obtaining a new set of matching point pairs;
[0010] Step 4) establishing directed line segment models and direction labels of feature points in the reference image and the image to be registered in step 1) based on the new matching point pairs;
[0011] Step 5) Direction matching is then performed on the corresponding directed line segments in the reference image and the image to be registered, and the mismatched point pairs are eliminated twice through a probability statistical model to obtain the final precise matching point pairs;
[0012] Step 6) Use the AANAP (Adaptive As-Natural-As-Possible) algorithm to align the reference image and the image to be registered, and use the weighted average method to fuse the images to complete the splicing of the reference image and the image to be registered.
[0013] The specific contents of step 1) include:
[0014] Step 11) Perform SIFT feature extraction on two reference images I1 and the image to be registered I2 of the same size and overlapping areas of the coal mine tunnel, respectively, to obtain the descriptor ds1 of the feature points of the reference image and the descriptor ds2 of the image to be registered, as well as the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered;
[0015] Step 12) Calculate the Euclidean distance between the descriptor of the i-th point in the descriptor ds1 of the image to be registered and the descriptor of each point in the descriptor ds2 of the image to be registered, and obtain the minimum value as the Euclidean distance MIN between the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered ij , if the Euclidean distance MIN ij If it is still smaller than all other Euclidean distances after multiplying by 1.5, it is judged that the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered are a pair of matching points, otherwise they are not. Using the corresponding values of the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered, we can finally get n rough matching point pairs, and the coordinate values of the n rough matching points are and
[0016] where i∈(1,n) (1)
[0017] where i∈(1,n) (2)
[0018] The specific steps of constructing the directed line segment model of coarse matching point pairs between adjacent images in step 2) are as follows:
[0019] Step 21) By placing the reference image I1 and the image to be registered I2 in the same window in left-right order, in this window, for the obtained n rough matching point pairs, a directed line segment model is established for all the rough matching point pairs:
[0020] where i∈(1,n) (3)
[0021] Where: I w is the width of the image. The reference image I1 and the image to be registered I2 have the same size, and the height is I h , the corresponding coordinate y value, width is I w , the x value of the corresponding coordinate.
[0022] The steps of constructing the slope threshold interval model through the directed line segment model in step 2) are:
[0023] Step 22) Calculate the slope of the directed line segment model using the following formula:
[0024] where i∈(1,n) (4)
[0025] Step 23) To accurately find the slope threshold interval, and based on image matching experience, it is known that the slope of the image matching directed line segment is roughly in the interval [-1, 1], so the slope interval from negative infinity to positive infinity is divided into 22 intervals, namely [-∞, -1.0], [-1.0, -0.9], [-0.9, -0.8] ... [0.8, 0.9], [0.9, 1.0], [1.0, +∞];
[0026] Step 24) calculates the slopes of all directed line segments, and divides each directed line segment into the 22 intervals in step 23) according to its size, so as to count the number of directed line segments divided in each interval, and obtains the interval [slp1, slp2] with the largest number of directed line segments. Considering that the image to be registered may have a slight scale change and rotation transformation relative to the reference image, the slope threshold interval is expanded outward by 0.1 on both sides, that is, the slope threshold interval is [slp1-0.1, slp2+0.1].
[0027] In step 2), the specific steps of constructing the length constraint model through the directed line segment model are as follows:
[0028] Step 25) Use the following formula to obtain the length of the j-th directed line segment model between images:
[0029] where j∈(1,p) (5)
[0030] Step 26) Use the length calculation formula of the line segment model to establish a length constraint model:
[0031] where j∈(1,p) (6)
[0032] Where: is the average of the squared Euclidean distances of all matching points, T d is the length constraint control value obtained through experiments, T d It is related to the size of the images involved in image stitching. If the image pixels are in the millions, T d =4, if the image pixel value is less than one million pixels, T d =1.5.
[0033] The specific steps of step 3) are:
[0034] Step 31) Eliminate the matching points that do not meet the constraints in the rough matching point pairs between the reference image and the image to be registered through the obtained slope threshold space, and obtain a set of p matching point index values that meet the slope threshold space among the n matching points.
[0035] where i∈(1,n) (7)
[0036] Step 32) Perform the length constraint of step 26) on the obtained p matching points to obtain a set of k correct matching point index values that meet the length constraint among the p matching points.
[0037] where j∈(1,p) (8)
[0038] The specific steps of step 4) are:
[0039] Step 41) After obtaining the new matching point pair, a directed line segment is established in the reference image and the image to be registered, and a directed line segment model of the feature points in the image is constructed based on the new matching point as follows:
[0040]
[0041] Where: i∈(1,k-1), j∈(i+1,k), and (XianD 内1 (i,j), XianD 内2 (i,j)) are two pairs of matching points.
[0042] Step 42) Calculation can be obtained to obtain the directed line segment models in the reference image and the image to be registered. elements, and the directed line segment model of the reference image and the directed line segment model of the image to be registered correspond to each other according to the index value, then it is expressed according to the index value as follows:
[0043] in
[0044] Step 43) establishes a direction coordinate system with the feature point as the origin, so as to determine the direction label of the directed line segment, and then determines and The quadrant in the direction coordinate system is used to label the directed line segment:
[0045]
[0046] Step 44) Using the label flag as the matching condition for the directed line segment, the directed line segment model is changed to:
[0047] in
[0048] The specific steps of step 5) are:
[0049] Step 51) If It means and is a pair of correctly matched line segments, that is and is a pair of correctly matched line segments, the endpoints of the two corresponding line segments and All are correct matching points, otherwise they are all wrong matching points;
[0050] Step 52) By counting the number of times each pair of matching points is classified as a mismatch point, it is determined whether the matching point is a mismatch point, and a quantity statistics matrix TongJ∈R is established. 2×k :
[0051] where i∈(1,k) (13)
[0052] Step 53) The number of times each pair of matching points is judged as a mismatch point is placed in the quantity statistics matrix TongJ, thereby obtaining the probability statistics model corresponding to each pair of matching points:
[0053] where i∈(1,k) (14)
[0054] Step 54) Set a threshold T for the probability statistical model p , if the probability of a pair of matching points is greater than the threshold, it means that it is a wrong matching point, then we can get l The correct matching point index value in:
[0055] where i∈(1,k) (15)
[0056] Step 55) By using equations (8) and (15), we can obtain l correct matching point index values in the original coordinate set:
[0057]
[0058] Combining equations (1), (2) and (16), we can get the coordinate value of the correct matching point after the mismatching point is proposed:
[0059]
[0060] The specific steps of step 6) are:
[0061] Step 61) For the obtained correct matching points D′1 and D′2, the AANAP algorithm is used to align the reference image and the image to be registered, that is, image registration.
[0062] Step 62) After the reference image and the image to be registered are registered, the weighted average method is used to fuse the images. The calculation formula of the weighted average method is as follows:
[0063]
[0064] Where: I1 and I2 are adjacent images with overlapping areas of the input, I(x, y) is the final output fused image, I1∩I2 represents the overlapping area of I1 and I2, w1 and w2 are weighted values, and w1+w2=1, w1,w1∈(0,1). The weights are determined as follows:
[0065]
[0066] Where: x l and x r are the left and right boundaries of the overlapping area, and x corresponds to the column number of the point in the image.
[0067] Step 63) completes image stitching and obtains a panoramic stitching image.
[0068] Beneficial effects:
[0069] This method has a stronger ability to eliminate false matches, faster real-time performance, and better accuracy, and realizes panoramic stitching of coal mine tunnel images. Compared with traditional false match elimination methods, it requires fewer feature points and has a better stitching effect.
[0070] The false matching elimination algorithm based on the feature point directed line segment model proposed in the present invention can effectively eliminate false matching and obtain a more accurate image transformation model. At the same time, it has a faster running time and higher matching accuracy and real-time performance. Then, the AANAP algorithm is used to align and stitch the images, so that the image stitching effect and natural appearance of the present invention are better when there are fewer feature points but higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 is a schematic diagram of directed line segments of matching points between images of the present invention,
[0072] Figure 2 is a schematic diagram of directed line segments of feature points in an image of the present invention,
[0073] Figure 3 is a schematic diagram of the directional coordinate system of feature points in an image of the present invention,
[0074] Figure 4 The present invention is a flow chart of the image stitching method applied to underground coal mine tunnels. DETAILED DESCRIPTION
[0075] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the protection scope of the present invention.
[0076] Example 1: An image stitching method for underground coal mine tunnels
[0077] The present invention discloses an image stitching method applied to underground coal mine tunnels. First, the SIFT algorithm is used to extract and match the features of coal mine tunnel images to obtain rough matching point pairs; then a directed line segment model of rough matching point pairs of adjacent images is constructed, and the direction and length attributes of the line segments are used to eliminate the mismatched point pairs; then, the directed line segment models of feature points within each image and their direction labels are established, and then the corresponding directed line segments of adjacent images are matched in direction, and the mismatched point pairs are eliminated twice through a probability statistics model to obtain the final precise matching point pairs; finally, the AANAP algorithm is used to align the stitched images, and the weighted average method is used to fuse the images to complete the image stitching. The advantages of this method over other algorithms are: the mismatched point elimination algorithm has high accuracy and good real-time performance, and the final panoramic stitching image has better quality, and is more suitable for underground coal mine tunnel images.
[0078] like Figure 4 As shown, the present invention adopts an image stitching method applied to underground coal mine tunnels as follows:
[0079] Step 1) Using the SIFT (Scale Invariant Feature Transform) algorithm, feature extraction and matching are performed between the reference image and the image to be registered, where the two images are adjacent and partially overlapped, to obtain a set of coarse matching point pairs between the reference image and the image to be registered;
[0080] The specific process is:
[0081] Step 11) Perform SIFT feature extraction on two reference images I1 and the image to be registered I2 of the same size and overlapping areas of the coal mine tunnel, respectively, to obtain the descriptor ds1 of the feature points of the reference image and the descriptor ds2 of the image to be registered, as well as the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered;
[0082] Step 12) Calculate the Euclidean distance between the descriptor of the i-th point in the descriptor ds1 of the image to be registered and the descriptor of each point in the descriptor ds2 of the image to be registered, and obtain the minimum value as the Euclidean distance MIN between the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered ij , if the Euclidean distance MINij If it is still smaller than all other Euclidean distances after multiplying by 1.5, it is judged that the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered are a pair of matching points, otherwise they are not. Using the corresponding values of the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered, we can finally get n rough matching point pairs, and the coordinate values of the n rough matching points are and
[0083] where i∈(1,n) (1)
[0084] where i∈(1,n) (2)
[0085] Figure 1 As shown, step 2) uses the feature points corresponding to the coarse matching point pairs between the reference image and the image to be registered to construct a directed line segment model, AF and af in the reference image and the image to be registered correspond one to one, and constructs a slope threshold interval and a length constraint model through the directed line segment model;
[0086] The specific process is:
[0087] Step 21) By placing the reference image I1 and the image to be registered I2 in the same window in left and right order, the obtained n rough matching point pairs are obtained in this window, such as Figure 1 As shown, <aa> 、 <bb> 、 <cc> 、 <mm> 、 <ee> 、 <ff>This is an example of a directed line segment formed by matching point pairs between images. A directed line segment model is established for all coarse matching point pairs:
[0088] where i∈(1,n) (3)
[0089] Where: I w is the width of the image. The reference image I1 and the image to be registered I2 have the same size, and the height is I h (corresponding to the y value of the coordinate), the width is I w (corresponding to the x value of the coordinate).
[0090] Step 22) Calculate the slope of the directed line segment model using the following formula:
[0091] where i∈(1,n) (4)
[0092] Step 23) To accurately find the slope threshold interval, and based on image matching experience, it is known that the slope of the image matching directed line segment is roughly in the interval [-1, 1], so the slope interval from negative infinity to positive infinity is divided into 22 intervals, namely [-∞, -1.0], [-1.0, -0.9], [-0.9, -0.8] ... [0.8, 0.9], [0.9, 1.0], [1.0, +∞];
[0093] Step 24) calculates the slopes of all directed line segments, and divides each directed line segment into 22 intervals in step 32) according to its size, so as to count the number of directed line segments divided in each interval, and obtains the interval [slp1, slp2] with the largest number of directed line segments. Considering that the image to be registered may have a slight scale change and rotation transformation relative to the reference image, the slope threshold interval is expanded outward by 0.1 on both sides, that is, the slope threshold interval is [slp1-0.1, slp2+0.1].
[0094] Step 25) Use the following formula to obtain the length of the j-th directed line segment model between images:
[0095] where j∈(1,p) (5)
[0096] Step 26) Use the length calculation formula of the line segment model to establish a length constraint model:
[0097] where j∈(1,p) (6)
[0098] Where: is the average of the squared Euclidean distances of all matching points, T d is the length constraint control value obtained through experiments, T d It is related to the size of the images involved in image stitching. Experiments show that when the image pixels are in the millions, T d =4 is better, when the image pixel value is less than one million pixels T d =1.5 is better.
[0099] Step 3) integrating the slope threshold interval and the length constraint model, and removing the mismatched point pairs in the rough matching point pairs, thereby obtaining a new set of matching point pairs;
[0100] like Figure 3 As shown, the specific process is:
[0101] Step 31) Eliminate the matching points that do not meet the constraints in the rough matching point pairs between the reference image and the image to be registered by using the slope threshold space obtained, and obtain a set of p matching point index values that meet the slope threshold space among the n matching points.
[0102] where i∈(1,n) (7)
[0103] Step 32) Perform the length constraint of step 26) on the obtained p matching points to obtain a set of k correct matching point index values that meet the length constraint among the p matching points.
[0104] where j∈(1,p) (8)
[0105] like Figure 2 is a schematic diagram of directed line segments of feature points in the image. Figure 3 is a schematic diagram of the directional coordinate system of the feature points in the image. Step 4) establishes the directed line segment models of the feature points in the reference image and the image to be registered in step 1) and their directional labels based on the new matching point pairs;
[0106] The specific process is:
[0107] Step 41) After obtaining the new matching point pair, a directed line segment is established in the reference image and the image to be registered, such as Figure 2 As shown, (B,b), (C,c), (H,h), (Q,q), (V,v) are the matching points obtained after the slope threshold interval and length constraint. <bc> 、 <bh> 、 <bq> 、 <bv>and <bc> 、 <bh> 、 <bq> 、 <bv>are examples of directed line segments of the reference image and the image to be registered, and they correspond one to one according to the matching points, that is,<BC,bc> ,<BH,bh> ,<BQ,bq> and<BV,bv> There are four matching line segments composed of five pairs of matching points. Then the directed line segment model of the feature points in the image is constructed based on the new matching points as follows:
[0108]
[0109] Where: i∈(1,k-1), j∈(i+1,k), and (XianD 内1 (i,j), XianD 内2 (i,j)) are two pairs of matching points.
[0110] Step 42) Calculation can be obtained to obtain the directed line segment models in the reference image and the image to be registered. elements, and the directed line segment model of the reference image and the directed line segment model of the image to be registered correspond to each other according to the index value, then it can be expressed as follows according to the index value:
[0111] in
[0112] Step 43) Establish a directional coordinate system with the feature point as the origin (such as Figure 3 shown), where<P,P1> ,<P,P2> ,<P,P3> ,<P,P4> Represents a direction and corresponds to the four quadrants of the coordinate system one by one, so as to determine the direction label of the directed line segment, and then determine and The quadrant in the direction coordinate system is used to label the directed line segment:
[0113]
[0114] Step 44) Using the label flag as the matching condition for the directed line segment, the directed line segment model is changed to:
[0115] in
[0116] Step 5) Direction matching is then performed on the corresponding directed line segments in the reference image and the image to be registered, and the mismatched point pairs are eliminated twice through a probability statistical model to obtain the final precise matching point pairs;
[0117] The specific process is:
[0118] Step 51) If It means and is a pair of correctly matched line segments, that is and is a pair of correctly matched line segments, the endpoints of the two corresponding line segments and All are correct matching points, otherwise they are all wrong matching points;
[0119] Step 52) By counting the number of times each pair of matching points is classified as a mismatch point, it is determined whether the matching point is a mismatch point, and a quantity statistics matrix TongJ∈R is established. 2×k :
[0120] where i∈(1,k) (13)
[0121] Step 53) The number of times each pair of matching points is judged as a mismatch point is placed in the quantity statistics matrix TongJ, thereby obtaining the probability statistics model corresponding to each pair of matching points:
[0122] where i∈(1,k) (14)
[0123] Step 54) Set a threshold T for the probability statistical model p , if the probability of a pair of matching points is greater than the threshold, it means that it is a wrong matching point, then we can get l The correct matching point index value in:
[0124] where i∈(1,k) (15)
[0125] Step 55) By using equations (8) and (15), we can obtain l correct matching point index values in the original coordinate set:
[0126]
[0127] Combining equations (1), (2) and (16), we can get the coordinate value of the correct matching point after the mismatching point is proposed:
[0128]
[0129] Step 6) Use the AANAP (Adaptive As-Natural-As-Possible) algorithm to align the reference image and the image to be registered, and use the weighted average method to fuse the images to complete the splicing of the reference image and the image to be registered;
[0130] The specific process is:
[0131] Step 61) For the obtained correct matching points D′1 and D′2, the AANAP algorithm is used to align the reference image and the image to be registered, that is, image registration.
[0132] Step 62) After the reference image and the image to be registered are registered, the weighted average method is used to fuse the images. The calculation formula of the weighted average method is as follows:
[0133]
[0134] Where: I1 and I2 are adjacent images with overlapping areas of the input, I(x, y) is the final output fused image, I1∩I2 represents the overlapping area of I1 and I2, w1 and w2 are weighted values, and w1+w2=1, w1,w1∈(0,1). The weights are determined as follows:
[0135]
[0136] Where: x l and x r are the left and right boundaries of the overlapping area, and x corresponds to the column number of the point in the image.
[0137] Step 63) completes image stitching and obtains a panoramic stitching image.< / bv> < / bq> < / bh> < / bc> < / bv> < / bq> < / bh> < / bc> < / ff> < / ee> < / mm> < / cc> < / bb> < / aa>
Claims
1. An image stitching method for underground coal mine tunnels, characterized in that This method comprises the following steps: Step 1) Using the SIFT (Scale Invariant Feature Transform) algorithm, feature extraction and matching are performed between the reference image and the image to be registered, where the two images are adjacent and partially overlapped, to obtain a set of coarse matching point pairs between the reference image and the image to be registered; The specific contents of step 1) include: Step 11) Perform SIFT feature extraction on two reference images I1 and the image to be registered I2 of the same size and overlapping areas of the coal mine tunnel, respectively, to obtain the descriptor ds1 of the feature points of the reference image and the descriptor ds2 of the image to be registered, as well as the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered; Step 12) Calculate the Euclidean distance between the descriptor of the i-th point in the descriptor ds1 of the image to be registered and the descriptor of each point in the descriptor ds2 of the image to be registered, and obtain the minimum value as the Euclidean distance MIN between the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered ij , if the Euclidean distance MIN ij If it is still smaller than all other Euclidean distances after multiplying by 1.5, it is judged that the i-th point in the descriptor ds1 of the image to be registered and the j-th point in the descriptor ds2 of the image to be registered are a pair of matching points, otherwise they are not. Using the corresponding values of the pixel coordinate system position kp1 in the reference image and the pixel coordinate system position kp2 in the image to be registered, we can finally get n rough matching point pairs, and the coordinate values of the n rough matching points are and where i∈(1,n)(1) where i∈(1,n)(2) Step 2) constructing a directed line segment model using feature points corresponding to the coarse matching point pairs between the reference image and the image to be registered, and constructing a slope threshold interval and a length constraint model through the directed line segment model; Step 21) By placing the reference image I1 and the image to be registered I2 in the same window in left-right order, in this window, for the obtained n rough matching point pairs, a directed line segment model is established for all the rough matching point pairs: where i∈(1,n)(3) Where: I w is the width of the image. The reference image I1 and the image to be registered I2 have the same size, and the height is I h , the corresponding coordinate y value, width is I w , the x value of the corresponding coordinate; Step 22) Calculate the slope of the directed line segment model using the following formula: where i∈(1,n)(4) Step 23) To accurately find the slope threshold interval, and based on the experience of image matching, the slope of the image matching directed line segment in the underground engineering field is in the interval [-1,1], so the slope interval from negative infinity to positive infinity is divided into 22 intervals, namely [-∞, -1.0], [-1.0, -0.9], [-0.9, -0.8] ... [0.8, 0.9], [0.9, 1.0], [1.0, +∞]; Step 24) calculates the slopes of all directed line segments, and divides each directed line segment into the 22 intervals in step 23) according to its size, so as to count the number of directed line segments divided in each interval, and obtains the interval [slp1, slp2] with the largest number of directed line segments. Considering that the image to be registered may have a slight scale change and rotation transformation relative to the reference image, the slope threshold interval is expanded outward by 0.1 on both sides, that is, the slope threshold interval is [slp1-0.1, slp2+0.1]; Step 25) Use the following formula to obtain the length of the j-th directed line segment model between images: where j∈(1,p)(5) Step 26) Use the length calculation formula of the line segment model to establish a length constraint model: where j∈(1,p)(6) Where: is the average of the squared Euclidean distances of all matching points, T d is the length constraint control value obtained through experiments, T d It is related to the size of the images involved in image stitching. If the image pixels are in the millions, T d =4, if the image pixel value is less than one million pixels, T d =1.5; Step 3) integrating the slope threshold interval and the length constraint model, and removing the mismatched point pairs in the rough matching point pairs, thereby obtaining a new set of matching point pairs; Step 31) Eliminate the matching points that do not meet the constraints in the rough matching point pairs between the reference image and the image to be registered through the obtained slope threshold space, and obtain a set of p matching point index values that meet the slope threshold space among the n matching points. where i∈(1,n)(7) Step 32) Perform the length constraint of step 26) on the obtained p matching points to obtain a set of k correct matching point index values that meet the length constraint among the p matching points. where j∈(1,p)(8) Step 4) establishing directed line segment models and direction labels of feature points in the reference image and the image to be registered in step 1) based on the new matching point pairs; Step 5) Direction matching is then performed on the corresponding directed line segments in the reference image and the image to be registered, and the mismatched point pairs are eliminated twice through a probability statistical model to obtain the final precise matching point pairs; Step 6) Use the AANAP (Adaptive As-Natural-As-Possible) algorithm to align the reference image and the image to be registered, and use the weighted average method to fuse the images to complete the splicing of the reference image and the image to be registered.
2. According to claim 1, the image stitching method applied to underground coal mine tunnels is characterized by: The specific steps of step 4) are: Step 41) After obtaining the new matching point pair, a directed line segment is established in the reference image and the image to be registered, and a directed line segment model of the feature points in the image is constructed based on the new matching point as follows: Where: i∈(1,k-1), j∈(i+1,k), and (XianD 内1 (i,j), XianD 内2 (i,j)) are two pairs of matching points; Step 42) Calculation can be obtained to obtain the directed line segment models in the reference image and the image to be registered. elements, and the directed line segment model of the reference image and the directed line segment model of the image to be registered correspond to each other according to the index value, then it is expressed according to the index value as follows: in Step 43) establishes a direction coordinate system with the feature point as the origin, so as to determine the direction label of the directed line segment, and then determines and The quadrant in the direction coordinate system is used to label the directed line segment: Step 44) Using the label flag as the matching condition for the directed line segment, the directed line segment model is changed to: in 3. The image stitching method applied to underground coal mine tunnels according to claim 1, characterized in that: The specific steps of step 5) are: Step 51) If It means and is a pair of correctly matched line segments, that is and is a pair of correctly matched line segments, the endpoints of the two corresponding line segments and All are correct matching points, otherwise they are all wrong matching points; Step 52) By counting the number of times each pair of matching points is classified as a mismatching point, it is determined whether the matching point is a mismatching point, and a quantity statistics matrix TongJ∈R is established. 2×k : where i∈(1,k)(13) Step 53) The number of times each pair of matching points is judged as a mismatch point is placed in the quantity statistics matrix TongJ, thereby obtaining the probability statistics model corresponding to each pair of matching points: where i∈(1,k)(14) Step 54) Set a threshold T for the probability statistical model p , if the probability of a pair of matching points is greater than the threshold, it means that it is a wrong matching point, then we can get l The correct matching point index value in: where i∈(1,k)(15) Step 55) By using equations (8) and (15), we can obtain l correct matching point index values in the original coordinate set: Combining equations (1), (2) and (16), we can get the coordinate value of the correct matching point after the mismatching point is proposed:
4. The image stitching method for underground coal mine tunnels according to claim 3 is characterized by: The specific steps of step 6) are: Step 61) For the obtained correct matching points D1′ and D2′, use the AANAP algorithm to align the reference image and the image to be registered, i.e., image registration; Step 62) After the reference image and the image to be registered are registered, the weighted average method is used to fuse the images. The calculation formula of the weighted average method is as follows: Where: I1 and I2 are adjacent images with overlapping areas of the input, I(x, y) is the final output fused image, I1∩I2 represents the overlapping area of I1 and I2, w1 and w2 are weighted values, and w1+w2=1, w1,w1∈(0,1); the weights are determined as follows: Where: x l and x r are the left and right boundaries of the overlapping area, and x corresponds to the column number of the pixel in the image; Step 63) completes image stitching and obtains a panoramic stitching image.
