A relative positioning method based on visual image fusion

Abstract

The application discloses a relative positioning method based on visual image fusion and belongs to the field of image processing. The method can realize scene image splicing of a region and relative position determination between nodes by matching and fusing images taken by nodes at different positions in the same region at different angles. The method comprises the following steps: acquiring visual images of the nodes; extracting and matching feature information in the images; determining relative projection relationship between the images; realizing image fusion and splicing through a unified coordinate system; determining rotation and translation relationship between the images; and realizing relative position relationship determination between corresponding nodes of the images. The application can not only realize scene image splicing and fusion, but also quickly and accurately position the nodes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, in particular to a relative positioning method based on visual image fusion. BACKGROUND

[0002] In recent years, with the rapid popularization and development of intelligent terminals, computer vision, pattern recognition and digital image processing technologies have been applied to various fields of modern society and have played an irreplaceable role. Among them, the positioning technology based on visual image has become a global research hotspot and has attracted widespread attention from various fields of society.

[0003] Compared with other positioning methods, the visual positioning method has the advantages of low cost, simple operation, strong real-time performance and high reliability. The current smart phones are generally equipped with high-pixel visual sensors, which can replace the traditional industrial cameras to simply and quickly capture the surrounding scene images. Compared with other positioning methods, the visual positioning method undoubtedly saves a lot of labor cost and equipment cost, and the operation is very simple. In addition, under sufficient light conditions, the visual image contains rich scene information, and through the information mining of the visual image, people can better perceive the surrounding environment, which is very similar to the way people use their eyes to perceive the surrounding things.

[0004] In addition, the multi-view image fusion technology can avoid the problem that the single view cannot accurately perceive the surrounding environment due to occlusion or interference, and through the interaction of multiple views, not only the environment perception can be more complete and comprehensive, but also the relative position information between different views can be calculated, which has broad development prospects in the future. SUMMARY

[0005] In order to solve the problem that different nodes in the same scene cannot establish relative position correlation, the present application proposes a relative positioning method based on visual image fusion. This method not only can realize the splicing and fusion of scene images, but also can quickly and accurately determine the relative position relationship between nodes in the scene.

[0006] The technical scheme adopted by the present application is:

[0007] A relative positioning method based on visual image fusion, comprising the following steps:

[0008] Step 1, each node at different positions in the same area uses a camera to shoot the surrounding environment at different angles to obtain scene images shot by each node;

[0009] Step 2, extract the feature information contained in each image, and use the feature information to establish the correlation between images;

[0010] Step 3, feature point matching between images is carried out, and the error matched feature point pairs are removed to obtain the homography matrix between images;

[0011] Step 4, the projection relationship between images is solved, the images are projected into the same coordinate system, and the splicing and fusion between images are realized;

[0012] Step 5, the rotation and translation relationship between images is solved, and the relative position relationship between nodes is calculated;

[0013] The splicing and fusion of scene images and the confirmation of the relative position relationship between nodes are completed.

[0014] Further, in step 2, the feature information contained in each image is extracted, and the specific mode is as follows: the image pyramid corresponding to each image is constructed according to the Hessian matrix, the position and main direction of the key point are determined, and the construction of the SURF local feature descriptor in the image is completed.

[0015] Further, in step 3, the feature point matching between images is carried out, and the specific mode is as follows: the Euclidean distance between each corresponding feature point between images is solved, these distances are arranged from small to large, and the nearest several feature point pairs are output and displayed;

[0016] The error matched feature point pairs are removed to obtain the homography matrix between images, and the specific mode is as follows: the RANSAC algorithm is used to iteratively update the matched point pairs to obtain the optimal model, so that the homography matrix is solved.

[0017] Further, in step 4, the specific mode of realizing the splicing and fusion between images is as follows: the superimposition mapping between image feature points is realized by using the homography matrix, the APAP algorithm is used to align the superimposed image feature points, the stitched image after alignment is removed, and the redundant part is removed.

[0018] Further, the specific mode of step 5 is as follows: the rotation matrix and the translation vector between the coordinate systems corresponding to two images are calculated through the feature point matching between images, the nodes are unified in the same coordinate system through the rotation and translation relationship, and the coordinates of each node corresponding to the coordinate system are respectively solved, and finally the relative position relationship between nodes is obtained.

[0019] The beneficial effects of the present application are as follows:

[0020] 1, the method of the present application can better perceive the spatial information in the scene by matching and fusing multi-view images to realize the splicing of panoramic images, which can solve the problem that a single view may not accurately perceive the surrounding environment due to occlusion or interference.

[0021] 2、The method of the present application realizes the confirmation of the relative position relationship between each node in the scene through the matching and fusion of visual images by using visual positioning technology, which not only has high real-time positioning, but also can guarantee good positioning accuracy, and has broad development prospects in the future. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 Extraction of SURF features for scene images.

[0023] Figure 2 Rough matching of SURF feature points between images.

[0024] Figure 3 Fine matching of SURF feature points between images.

[0025] Figure 4 Image splicing and fusion.

[0026] Figure 5 Positioning error curve. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0028] A relative positioning method based on visual image fusion, comprising the following steps:

[0029] Step 1: Each node at different positions in the same area uses a camera to shoot the surrounding environment at different angles, and obtains scene images shot by each node;

[0030] Step 2: Feature information contained in each image is extracted, and the correlation between images is established by using the feature information;

[0031] Step 3: Feature point matching between images is realized, and the error matched feature point pairs are removed, to obtain the homography matrix between images;

[0032] Step 4: The projection relationship between images is solved, the images are projected into the same coordinate system, and the splicing and fusion between images are realized;

[0033] Step 5: The rotation and translation relationship between images is solved, and the relative position relationship between nodes is calculated.

[0034] The splicing and fusion of scene images and the confirmation of the relative position relationship between nodes are completed.

[0035] Further, in step 2, the specific way of extracting the feature information contained in each image is: constructing the image pyramid corresponding to each image according to the Hessian matrix, determining the position and main direction of the key point, and completing the construction of the SURF local feature descriptor in the image.

[0036] Further, in step 3, the specific way of realizing the feature point matching between images is: solving the Euclidean distance between each corresponding feature point between images, arranging these distances from small to large, and outputting and displaying the nearest feature point pairs. The specific way of eliminating the error matched feature point pairs is: using the RANSAC algorithm to iteratively update the matched point pairs to obtain the optimal model, so as to solve the homography matrix.

[0037] Further, in step 4, the specific way of realizing the splicing and fusion between images is: using the homography matrix to realize the superimposition mapping between image feature points, using the APAP algorithm to align the superimposed image feature points, stitching the aligned image, and removing the overlapping redundant part.

[0038] Further, in step 5, the specific way of calculating the relative position relationship between nodes is: through the feature point matching between images, the rotation matrix and translation vector between the coordinate systems corresponding to two images are calculated, the nodes are unified in the same coordinate system through the rotation and translation relationship, and the coordinates of each node in the coordinate system are calculated respectively, and finally the relative position relationship between nodes is obtained.

[0039] The following is a more specific example:

[0040] A relative positioning method based on visual image fusion is applied to the mutual position confirmation task between nodes in a specified scene, and the method comprises the following steps:

[0041] Step 1: Each node shoots a scene image at a different angle;

[0042] In order to facilitate the subsequent principle explanation, this embodiment takes two nodes as an example to deduce the corresponding formula. This embodiment establishes the position relationship between nodes by processing the visual images shot by the nodes, so it is necessary to ensure that the angles of view of the two nodes overlap, and then the relative positioning between nodes can be performed according to the same object observed by them as a reference. It is assumed that two nodes in the scene can see a three-dimensional point P, and the two nodes can see the three-dimensional point from different angles. The projection points of the three-dimensional point P in the images shot by the two nodes are respectively set as p1 and p2, and the two images satisfy the epipolar geometry constraint relationship. The constraint relationship between the two projection points p1 and p2 is:

[0043]

[0044] Wherein, F is the fundamental matrix between two images.

[0045] Step 2: extracting feature points of node images;

[0046] In order to realize the node image feature extraction more quickly and accurately, the SURF algorithm is used for image feature extraction. In order to speed up the extraction of image feature points, the SURF algorithm uses Hessian matrix to realize the extraction of image feature points, and the Hessian matrix can be expressed as follows:

[0047]

[0048] Wherein, f(x,y) is the gray value corresponding to the pixel point (x,y). The schematic diagram of SURF feature extraction of scene image is shown in Figure 1

[0049] Step 3: realizing feature matching between node images;

[0050] When the feature matching between images is carried out, the feature vectors corresponding to each feature point in the image are first calculated, then the Euclidean distance between each corresponding feature point between images is solved, and the nearest several pairs of feature points are output and displayed as matching point pairs, Figure 2 The result of SURF feature point coarse matching between images can be seen from the figure that most of the feature point pairs satisfy the true corresponding relationship, but there are still a small amount of feature point pairs that are matching errors, at this time, the RANSAC algorithm is used to realize the precise matching of SURF feature points. The RANSAC algorithm calculates the projection error of the coarse matching point pair by iteration, and adds the matching point pair meeting the threshold requirement to the optimal inlier set, until the iteration times reaches k, the iteration is stopped, and the expression of k is:

[0051]

[0052] Wherein, p is the confidence, generally taking 0.995; w is the inlier proportion; m is the sample number, generally taking 4. At the same time, the RANSAC algorithm can also be used to solve the homography matrix between images, which reflects the mapping relationship of pixel points between images. The SURF feature point precise matching result obtained by RANSAC algorithm is shown in Figure 3 It can be seen that there is no error matching point pair at this time.

[0053] Step 4: realizing the splicing and fusion of node images;

[0054] ​After obtaining the matching point pairs by using the RANSAC algorithm, the APAP algorithm is used to register the matching point pairs, and then the image stitching is realized through translation, rotation and affine transformation between images and other operations. Figure 4 As shown in the image fusion result of image stitching

[0055] Step 5: relative position relationship between nodes is calculated;

[0056] By using the SURF feature point matching between the node shooting images, the essential matrix E between the images can be solved, the matrix reflects the relative position relationship between the cameras of the node collected images, and the expression is as follows:

[0057] E = [t r ] X R r (4)

[0058] Wherein, R r represents the rotation matrix between the cameras, t r represents the translation vector between the cameras, and [x] X represents the skew-symmetric matrix.

[0059] Suppose that the positions of two cameras in the same coordinate system are P1 and P2, then two three-dimensional coordinate points satisfy the following relationship:

[0060]

[0061] At this time, one of the cameras is taken as the origin of the coordinate system to establish a three-dimensional coordinate system, and the three-dimensional coordinates of the other camera in the coordinate system are solved by using the above formula, so that the relative position relationship between the two nodes is confirmed.

[0062] In order to verify the relative positioning accuracy, 50 positioning experiments are carried out, and the finally drawn positioning error curve is as shown in Figure 5 The error curve graph can be seen that the positioning error of the application can be controlled within 5m, and the positioning accuracy is high.

[0063] In summary, the application provides a relative positioning method based on visual image fusion. The method can realize the splicing and fusion of scene images and the confirmation of the relative position relationship between each node in the scene. Through corresponding experimental verification, the experimental results show that the application can not only realize the splicing and fusion of the scene images taken by each node, but also quickly and accurately realize the relative positioning between each node, and has a broad development prospect.

[0064] The above describes in detail the relative positioning method based on visual image fusion. The above description of the embodiments is only used to help understand the method and the core idea of the application. Meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the application.

Claims

1. A relative positioning method based on visual image fusion, characterized in that, The method comprises the following steps: Step 1, each node at different positions in the same area uses a camera to shoot the surrounding environment at different angles to obtain scene images shot by each node; Step 2, feature information contained in each image is extracted, and the feature information is used to establish the association between images; Step 3, feature point matching between images is performed, and the wrong matched feature point pairs are removed to obtain the homography matrix between images; Step 4, the projection relationship between images is solved, the images are projected into the same coordinate system, and the image splicing and fusion are realized; Step 5, the rotation and translation relationship between images is solved, and the relative position relationship between nodes is calculated; the specific mode is as follows: By matching feature points between images, a rotation matrix between coordinate systems corresponding to two images is calculated and a translation vector ; The nodes are unified in the same coordinate system through the rotation and translation relationship, and the coordinates of each node corresponding to the coordinate system are calculated, and finally the relative position relationship between nodes is obtained: ； wherein, and are the positions of the two cameras in the same coordinate system, respectively; The scene image splicing and fusion and the relative position relationship confirmation between nodes are completed.

2. The relative positioning method based on visual image fusion according to claim 1, characterized in that, In step 2, the feature information contained in each image is extracted, and the specific mode is as follows: the image pyramid corresponding to each image is constructed according to the Hessian matrix, the position and main direction of the key point are determined, and the construction of the SURF local feature descriptor in the image is completed.

3. The relative positioning method based on visual image fusion according to claim 1, characterized in that, In step 3, the feature point matching between images is performed, and the specific mode is as follows: the Euclidean distances between each corresponding feature point between images are solved, the distances are arranged from small to large, and the nearest feature point pairs are output and displayed; The wrong matched feature point pairs are removed to obtain the homography matrix between images, and the specific mode is as follows: the RANSAC algorithm is used to iteratively update the matched point pairs to obtain the optimal model, so that the homography matrix is solved.

4. The relative positioning method based on visual image fusion according to claim 1, characterized in that, In step 4, the specific mode of realizing the splicing and fusion between images is as follows: the superimposed mapping between image feature points is realized by using the homography matrix, the APAP algorithm is used to align the superimposed image feature points, the stitched image after alignment is removed, and the redundant part is removed.

Images