Real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm

Abstract

The present invention belongs to the field of visual synthesis technology and moving target tracking technology of optoelectronic systems, and specifically relates to a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithms. The method divides the dynamic registration problem into two parallel tasks, matching and tracking, and realizes the dynamic registration of optoelectronic images and maps through the collaborative constraints of the two. Compared with the prior art, the present invention proposes a real-time heterogeneous dynamic map registration method, which can realize real-time registration of images or videos with map data, has low time complexity, and a processing speed of more than 60fps on a computer. The algorithm is simple and easy to transplant on an embedded platform. The present invention adopts template matching and feature point tracking to dynamically adjust the search area through matching accuracy, which can reduce the search difficulty and time and improve the robustness of the algorithm.

Description

Technical Field

[0001] The invention belongs to the field of vision synthesis technology and moving target tracking technology of optoelectronic systems, and in particular relates to a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithms. Background Art

[0002] Stereoscopic vision reconstruction and digital map fusion for airborne optoelectronic systems are important research topics for drones and helicopters. Generally, the television images obtained by airborne optoelectronic pods are two-dimensional color images or grayscale images with limited fields of view. Too small a field of view is not conducive to the pilot's overall grasp of the aircraft's environment and the selection of a safe flight path. Digital maps are three-dimensional vector maps drawn based on real scenes, which can provide pilots with more diverse information and a wider field of view. However, due to the great differences in the features of the two maps, how to align and fuse optoelectronic images and map data in real time and efficiently is the biggest challenge currently faced.

[0003] Commonly used methods for image registration include template matching, feature point matching, and image matching based on deep learning. The template matching method calculates the similarity between the image to be matched and the template in an ergodic manner, and returns the position with the highest similarity and the confidence probability. The matching speed is fast and real-time matching can be achieved, but its accuracy is highly correlated with the richness and uniqueness of the image feature information. For dynamically moving images, the registration points are not continuous and there is a large matching error. Feature point matching methods include SIFT, SURF, etc., which generate feature descriptors through image feature information and further achieve matching through feature similarity. The algorithm has high accuracy but high algorithm complexity. Moreover, when heterogeneous images are directly matched, due to the large difference in feature similarity, it is easy to cause the effect of zero feature point registration. Image matching based on deep learning uses training to achieve feature point registration, which is more accurate than SIFT and SURF algorithms, but still faces the problem of difficulty in matching heterogeneous images and solving real-time requirements.

[0004] In addition, the above three methods are difficult to solve the continuity problem of matching results of continuously changing images. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is: in view of the fact that the above methods are difficult to solve the problem of real-time and continuous registration of dynamic optoelectronic images and maps, how to provide a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithms. The method divides the dynamic registration problem into two parallel tasks, matching and tracking, and realizes the dynamic registration of optoelectronic images and maps through the collaborative constraints of the two.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention provides a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm, and the method comprises the following steps:

[0009] Step S1: image preprocessing is performed on the input photoelectric image and the two-dimensional pixel-level map generated using the map data; since heterogeneous images have large differences in target features including size, contour, and color, it is difficult to directly match them, so image preprocessing is required first;

[0010] Step S2: Initial matching tracking position: using the photoelectric image as a template, traversing the entire map through the template matching algorithm, obtaining an accurate initial matching position L0, generating a feature descriptor of the initial frame image, and calculating the initial homography matrix H0 of the image and the map;

[0011] Step S3: input the next frame image and the map; use the tracking algorithm to obtain a rough registration position L1; determine the dynamic search area on the map, and use the template matching algorithm to obtain a rough registration position L2;

[0012] Step S4: Position correction; since template matching does not have spatial continuity in principle, it often causes a large position displacement between the matching positions of the two frames of images; it is necessary to use the displacement vector transformation of the tracking algorithm to constrain; and although the displacement vector transformation has spatial continuity, it essentially does not effectively align the subsequent photoelectric image and map, and the error has a cumulative effect, so it needs to be corrected with the help of the template matching algorithm;

[0013] Step S5: Determine whether reinitialization is required; if the matching position needs to be reinitialized, re-perform global template matching to obtain an accurate matching position; when position initialization is required, the output registration position at the current moment is the corrected tracking position L1;

[0014] If position initialization is not required, the output registration position at the current moment is the corrected matching position L2;

[0015] If the number of feature descriptors does not meet the requirement, regenerate the feature descriptors in the current frame;

[0016] Step S6: Output the final registration result, and repeat steps S3 to S6 to process the next frame of image.

[0017] The image preprocessing operations in step S1 include basic image operations such as scale transformation, contour feature extraction, principal component analysis, and grayscale conversion.

[0018] In step S1, image preprocessing is performed to ensure that the same target in the image and the map remains consistent at the pixel level.

[0019] Wherein, the step S3 comprises:

[0020] Step 3.1: Input the next frame image and the map; calculate the displacement vector transformation of the photoelectric image before and after the frame respectively, combine the homography matrix to obtain the rotation displacement transformation of the photoelectric image relative to the map, and calculate the matching position L1 of the next frame image;

[0021] Step 3.2: According to the initial template matching and confidence probability as well as the displacement transformation of the previous and next frame maps, determine and narrow the next search area on the map, and use the template matching method within the search area to obtain the matching position L2 of the next frame image.

[0022] Among them, in the step S3.1, the next frame image f1 is obtained, and the optical flow method is used to calculate the movement vectors of the correct matching feature descriptors of the photoelectric image and the map respectively, and the feature descriptors ranked in the top 20 in matching accuracy are obtained. The homography matrix H1 and the rough translation vector v1 of the photoelectric image before and after the frame movement, as well as the homography matrix H2 and the rough translation vector v2 of the map before and after the frame movement are further calculated; and the position relationship and the average vector of the next frame image A' relative to the map B' are calculated:

[0023] A′=H1×H0×H2×B′

[0024] v=v2-v1

[0025] Where A′ is the mapping of a new frame of image on the map B′, that is, position L1.

[0026] In step 3.2, it is assumed that the size of the scale-transformed photoelectric image is w×h, and the search area S for template matching of the next frame image is determined according to the matching confidence probability p returned by the initial frame f0. To reduce the difficulty of calculation, S is a piecewise function:

[0027]

[0028] The center position of S is the center position at the previous moment plus the translation vector v;

[0029] Then, a new matching position L2 is obtained in the search area by using template matching.

[0030] Wherein, the step S4 comprises:

[0031] Step 4.1: Constrain the displacement of the previous and next frames; use the template matching algorithm to obtain the matching positions of the previous and next frames, and calculate the displacement, which is compared with the displacement vector obtained by the tracking algorithm. If the offset exceeds the threshold d0, the coefficient correction method is used to limit the displacement;

[0032] Step 4.2: Since template matching is not directional, but displacement is directional, in order to ensure the accuracy of template matching, the template matching algorithm is constrained by the displacement vector; if the matching direction is inconsistent with the movement direction, the displacement obtained by the tracking algorithm is used as the benchmark;

[0033] Step 4.3: When the confidence probability of the template matching algorithm is higher than a certain threshold, it is assumed that the positioning information obtained by the template matching algorithm for this frame image is more accurate, and the points with higher confidence probability of template matching are used to update the homography matrix H0 of the tracking algorithm.

[0034] Wherein, in the step 4.1, the displacement v′ between the position L2 and the position L0 at the previous moment is calculated, and the angle θ between v′ and the translation vector v and the displacement difference d are calculated;

[0035] Then, the matching position is corrected, the direction of the angle θ is determined, and the displacement vector v′ is constrained according to the direction and displacement difference; the specific real-time scheme is as follows:

[0036]

[0037] In step 4.2, if the angle θ is greater than 90°, it is determined that the return value of the template matching algorithm at this moment is an incorrect position, and the motion vector at the current moment is mainly based on the tracking algorithm;

[0038] In step 4.3, the position of the tracking algorithm is reversely optimized using the matching confidence probability. When the confidence probability of the template matching exceeds the set threshold, it is assumed that the template matching at this moment obtains an accurate registration position, and this position is used to reversely correct the starting position of the tracking algorithm and the homography matrix H0.

[0039] Wherein, the step S5 comprises:

[0040] Step 5.1: Reinitialization judgment of matching position: set a counter to count the number of times the template matching result and the tracking result have a large error. If the number of consecutive occurrences exceeds the preset threshold, the matching position is rematched globally and the tracking position is reinitialized. In this process, the current matching position is based on the result obtained by the tracking algorithm.

[0041] Step 5.2: Initialization judgment of tracking feature descriptors: Count the number of feature descriptors in each tracking. When the number is lower than the minimum feature descriptor threshold d1, regenerate the feature descriptors and then perform a new round of tracking. In this frame image, the registration position is based on the result obtained by the matching algorithm.

[0042] Among them, in the step 5.1, the matching position is re-initialized: a counter is set to count the number of times the situation in step 4.3 occurs. If it occurs multiple times in succession, the preset threshold number is 10, then it is necessary to re-match the matching position globally and re-initialize the tracking position; in this process, the current matching position is based on the result obtained by the tracking algorithm;

[0043] In step 5.2, the tracking feature descriptor initialization judgment is performed: the number of tracking feature descriptors is counted each time, and the minimum feature descriptors are set to 20. When the number of descriptors is less than 20, the descriptors are reinitialized, and then a new round of tracking is performed; in this frame image, the registration position is based on the result obtained by the matching algorithm.

[0044] (III) Beneficial effects

[0045] In view of the fact that the existing technical methods are difficult to solve the problem of real-time and continuous real-time registration of dynamic optoelectronic images and maps, the present invention provides a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithms. The method divides the dynamic registration problem into two parallel tasks, matching and tracking, and realizes dynamic registration of optoelectronic images and maps through the collaborative constraints of the two.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] (1) The present invention proposes a real-time heterogeneous dynamic map registration method, which can realize real-time registration of images or videos with map data, has low time complexity, and the processing speed on the computer is greater than 60fps. The algorithm is simple and easy to be transplanted to embedded platforms.

[0048] (2) The present invention adopts the method of template matching and feature point tracking, and dynamically adjusts the search area by matching accuracy, which can reduce the search difficulty and time and improve the robustness of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of the steps of the present invention.

[0050] Figure 2 Schematic diagram of input samples of the present invention.

[0051] Figure 3 It is a result schematic diagram of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0053] In order to solve the above technical problems, the present invention provides a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm, such as Figure 1 As shown, the method comprises the following steps:

[0054] Step S1: image preprocessing is performed on the input photoelectric image and the two-dimensional pixel-level map generated using the map data; since heterogeneous images have large differences in target features including size, contour, and color, it is difficult to directly match them, so image preprocessing is required first;

[0055] Step S2: Initial matching tracking position: using the photoelectric image as a template, traversing the entire map through the template matching algorithm, obtaining an accurate initial matching position L0, generating a feature descriptor of the initial frame image, and calculating the initial homography matrix H0 of the image and the map;

[0056] Step S3: input the next frame image and the map; use the tracking algorithm to obtain a rough registration position L1; determine the dynamic search area on the map, and use the template matching algorithm to obtain a rough registration position L2;

[0057] Step S4: Position correction; since template matching does not have spatial continuity in principle, it often causes a large position displacement between the matching positions of the two frames of images; it is necessary to use the displacement vector transformation of the tracking algorithm to constrain; and although the displacement vector transformation has spatial continuity, it essentially does not effectively align the subsequent photoelectric image and map, and the error has a cumulative effect, so it needs to be corrected with the help of the template matching algorithm;

[0058] Step S5: Determine whether reinitialization is required; if the matching position needs to be reinitialized, re-perform global template matching to obtain an accurate matching position; when position initialization is required, the output registration position at the current moment is the corrected tracking position L1;

[0059] If position initialization is not required, the output registration position at the current moment is the corrected matching position L2;

[0060] If the number of feature descriptors does not meet the requirement, regenerate the feature descriptors in the current frame;

[0061] Step S6: Output the final registration result, and repeat steps S3 to S6 to process the next frame of image.

[0062] The image preprocessing operations in step S1 include basic image operations such as scale transformation, contour feature extraction, principal component analysis, and grayscale conversion.

[0063] In step S1, image preprocessing is performed to ensure that the same target in the image and the map remains consistent at the pixel level.

[0064] Wherein, the step S3 comprises:

[0065] Step 3.1: Input the next frame image and the map; calculate the displacement vector transformation of the photoelectric image before and after the frame respectively, combine the homography matrix to obtain the rotation displacement transformation of the photoelectric image relative to the map, and calculate the matching position L1 of the next frame image;

[0066] Step 3.2: According to the initial template matching and confidence probability as well as the displacement transformation of the previous and next frame maps, determine and narrow the next search area on the map, and use the template matching method within the search area to obtain the matching position L2 of the next frame image.

[0067] Among them, in the step S3.1, the next frame image f1 is obtained, and the optical flow method is used to calculate the movement vectors of the correct matching feature descriptors of the photoelectric image and the map respectively, and the feature descriptors ranked in the top 20 in matching accuracy are obtained. The homography matrix H1 and the rough translation vector v1 of the photoelectric image before and after the frame movement, as well as the homography matrix H2 and the rough translation vector v2 of the map before and after the frame movement are further calculated; and the position relationship and the average vector of the next frame image A' relative to the map B' are calculated:

[0068] A′=H1×H0×H2×B′

[0069] v=v2-v1

[0070] Where A′ is the mapping of a new frame of image on the map B′, that is, position L1.

[0071] In step 3.2, it is assumed that the size of the scale-transformed photoelectric image is w×h, and the search area S for template matching of the next frame image is determined according to the matching confidence probability p returned by the initial frame f0. To reduce the difficulty of calculation, S is a piecewise function:

[0072]

[0073] The center position of S is the center position at the previous moment plus the translation vector v;

[0074] Then, a new matching position L2 is obtained in the search area by using template matching.

[0075] Wherein, the step S4 comprises:

[0076] Step 4.1: Constrain the displacement of the previous and next frames; use the template matching algorithm to obtain the matching positions of the previous and next frames, and calculate the displacement, which is compared with the displacement vector obtained by the tracking algorithm. If the offset exceeds the threshold d0, the coefficient correction method is used to limit the displacement;

[0077] Step 4.2: Since template matching is not directional, but displacement is directional, in order to ensure the accuracy of template matching, the template matching algorithm is constrained by the displacement vector; if the matching direction is inconsistent with the movement direction, the displacement obtained by the tracking algorithm is used as the benchmark;

[0078] Step 4.3: When the confidence probability of the template matching algorithm is higher than a certain threshold, it is assumed that the positioning information obtained by the template matching algorithm for this frame image is more accurate, and the points with higher confidence probability of template matching are used to update the homography matrix H0 of the tracking algorithm.

[0079] Wherein, in the step 4.1, the displacement v′ between the position L2 and the position L0 at the previous moment is calculated, and the angle θ between v′ and the translation vector v and the displacement difference d are calculated;

[0080] Then, the matching position is corrected, the direction of the angle θ is determined, and the displacement vector v′ is constrained according to the direction and displacement difference; the specific real-time scheme is as follows:

[0081]

[0082] In step 4.2, if the angle θ is greater than 90°, it is determined that the return value of the template matching algorithm at this moment is an incorrect position, and the motion vector at the current moment is mainly based on the tracking algorithm;

[0083] In step 4.3, the position of the tracking algorithm is reversely optimized using the matching confidence probability. When the confidence probability of the template matching exceeds the set threshold, it is assumed that the template matching at this moment obtains an accurate registration position, and this position is used to reversely correct the starting position of the tracking algorithm and the homography matrix H0.

[0084] Wherein, the step S5 comprises:

[0085] Step 5.1: Reinitialization judgment of matching position: set a counter to count the number of times the template matching result and the tracking result have a large error. If the number of consecutive occurrences exceeds the preset threshold, the matching position is rematched globally and the tracking position is reinitialized. In this process, the current matching position is based on the result obtained by the tracking algorithm.

[0086] Step 5.2: Initialization judgment of tracking feature descriptors: Count the number of feature descriptors in each tracking. When the number is lower than the minimum feature descriptor threshold d1, regenerate the feature descriptors and then perform a new round of tracking. In this frame image, the registration position is based on the result obtained by the matching algorithm.

[0087] Among them, in the step 5.1, the matching position is re-initialized: a counter is set to count the number of times the situation in step 4.3 occurs. If it occurs multiple times in succession, the preset threshold number is 10, then it is necessary to re-match the matching position globally and re-initialize the tracking position; in this process, the current matching position is based on the result obtained by the tracking algorithm;

[0088] In step 5.2, the tracking feature descriptor initialization judgment is performed: the number of tracking feature descriptors is counted each time, and the minimum feature descriptors are set to 20. When the number of descriptors is less than 20, the descriptors are reinitialized, and then a new round of tracking is performed; in this frame image, the registration position is based on the result obtained by the matching algorithm.

[0089] Example 1

[0090] This embodiment sees Figure 1 Step flow chart, a real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm is proposed. The specific operation steps are as follows:

[0091] 1. Preprocess the photoelectric image and map data to make the feature description of the same target in the image and map closer. Figure 2 The specific process includes:

[0092] ① Principal component analysis of the image; ② Contour extraction and processing; ③ Non-linear filtering; ④ Scale transformation to ensure that the same target in the image and map remains consistent at the pixel level, etc.

[0093] 2. Initialize the registration position. The specific process includes:

[0094] 2.1 For the initial frame f0, the template matching method is used to obtain the accurate matching position L0.

[0095] 2.2 Generate the feature descriptor subset of the initial frame. Calculate the homography matrix H0 between the photoelectric image and the map.

[0096] 3. Design a registration method, determine the dynamic search area, and obtain a rough matching position.

[0097] 3.1 Get the next frame image f1, use the optical flow method to calculate the movement vectors of the correct matching feature descriptors of the photoelectric image and the map, and get the feature descriptors with the top 20 matching accuracy. Further calculate the homography matrix H1 and rough translation vector v1 of the photoelectric image before and after the frame movement, and the homography matrix H2 and rough translation vector v2 of the map before and after the frame movement. And calculate the position relationship and average vector of the next frame image A' relative to the map B':

[0098] A′=H1×H0×H2×B′

[0099] v=v2-v1

[0100] Where A′ is the mapping of a new frame of image on the map B′, that is, position L1.

[0101] 3.2 Assume that the size of the scaled photoelectric image is w×h. According to the matching confidence probability p returned by the initial frame f0, determine the search area S for the template matching of the next frame image. To reduce the computational difficulty, S is a piecewise function:

[0102]

[0103] The center position of S is the center position at the previous moment plus the translation vector v.

[0104] 3.3 Use template matching to obtain a new matching position L2 in the search area.

[0105] 4. Position judgment and correction.

[0106] 4.1 Calculate the displacement v′ between position L2 and position L0 at the previous moment, and calculate the angle θ between v′ and the translation vector v and the displacement difference d;

[0107] 4.2 Matching position correction. Determine the direction of the angle θ and constrain the displacement vector v′ based on the direction and displacement difference. The specific real-time solution is as follows:

[0108]

[0109] 4.3 If the angle θ is greater than 90°, the template matching algorithm return value at that moment is determined to be an incorrect position, and the motion vector at the current moment is mainly based on the tracking algorithm.

[0110] 4.4 Use the matching confidence probability to reversely optimize the position of the tracking algorithm. When the accuracy of template matching exceeds the set threshold (0.35 in the experiment), it can be assumed that the template matching at this moment obtains an accurate registration position, and this position is used to reversely correct the starting position and homography matrix H0 of the tracking algorithm.

[0111] 5. Reinitialize the judgment.

[0112] 5.1 Matching position reinitialization judgment: Design a counter to count the number of times 4.3 occurs. If it occurs multiple times in a row (preset to 10), it is necessary to rematch the matching position globally and reinitialize the tracking position. In this process, the current matching position is based on the results obtained by the tracking algorithm.

[0113] 5.2 Tracking feature descriptor initialization judgment: Count the number of tracking feature descriptors each time, set the minimum feature descriptor to 20, and when the number of descriptors is less than 20, reinitialize the descriptor and then perform a new round of tracking. In this frame image, the registration position is based on the result of the matching algorithm.

[0114] 6. Output the final registration result and repeat steps S3-S6 to process the next frame of image.

[0115] The effect of heterogeneous image registration is shown in the attached Figure 3 As shown. Among the two boxes, the outer box is the search area, and the inner box is the registration position. The black binary image in the inner box is the image obtained after the photoelectric image is preprocessed in step 1.

[0116] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm, characterized in that: The method comprises the following steps: Step S1: image preprocessing is performed on the input photoelectric image and the two-dimensional pixel-level map generated using the map data; since heterogeneous images have large differences in target features including size, contour, and color, it is difficult to directly match them, so image preprocessing is required first; Step S2: Initial matching tracking position: using the photoelectric image as a template, traversing the entire map through the template matching algorithm, obtaining an accurate initial matching position L0, generating a feature descriptor of the initial frame image, and calculating the initial homography matrix H0 of the image and the map; Step S3: input the next frame image and map; A rough registration position L1 is obtained by using a tracking algorithm; a dynamic search area is determined on a map, and a rough registration position L2 is obtained by using a template matching algorithm; Step S4: position correction; Since template matching does not have spatial continuity in principle, it often causes a large positional displacement between the matching positions of the two frames of images. It is necessary to use the displacement vector transformation of the tracking algorithm to constrain it. Although the displacement vector transformation has spatial continuity, it essentially does not effectively align the subsequent photoelectric images and maps, and the errors have a cumulative effect. Therefore, it is necessary to use the template matching algorithm to correct it. Step S5: Determine whether reinitialization is required; if the matching position needs to be reinitialized, re-perform global template matching to obtain an accurate matching position; when position initialization is required, the output registration position at the current moment is the corrected tracking position L1; If position initialization is not required, the output registration position at the current moment is the corrected matching position L2; If the number of feature descriptors does not meet the requirement, regenerate the feature descriptors in the current frame; Step S6: Output the final registration result, and repeat steps S3 to S6 to process the next frame of image.

2. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 1, characterized in that: The image preprocessing operations in step S1 include basic image operations such as scale transformation, contour feature extraction, principal component analysis, and grayscale conversion.

3. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 2, characterized in that: In step S1, image preprocessing is performed to ensure that the same object in the image and the map remains consistent at the pixel level.

4. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 2, characterized in that: The step S3 comprises: Step 3.1: Input the next frame image and the map; calculate the displacement vector transformation of the photoelectric image before and after the frame respectively, combine the homography matrix to obtain the rotation displacement transformation of the photoelectric image relative to the map, and calculate the matching position L1 of the next frame image; Step 3.2: According to the initial template matching and confidence probability as well as the displacement transformation of the previous and next frame maps, determine and narrow the next search area on the map, and use the template matching method within the search area to obtain the matching position L2 of the next frame image.

5. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 4, characterized in that: In step 3.1, the next frame image f1 is obtained, and the optical flow method is used to calculate the movement vectors of the correct matching feature descriptors of the photoelectric image and the map, respectively, to obtain the feature descriptors with the top 20 matching accuracy, and further calculate the homography matrix H1 and the rough translation vector v1 of the photoelectric image before and after the frame movement, as well as the homography matrix H2 and the rough translation vector v2 of the map before and after the frame movement; and calculate the position relationship and the average vector of the next frame image A' relative to the map B': A′=H1×H0×H2×B′ v=v2-v1 Where A′ is the mapping of a new frame of image on the map B′, that is, position L1.

6. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 5, characterized in that: In step 3.2, assuming that the size of the scale-transformed photoelectric image is w×h, the search area S for template matching of the next frame image is determined according to the matching confidence probability p returned by the initial frame f0. To reduce the computational difficulty, S is a piecewise function: The center position of S is the center position at the previous moment plus the translation vector v; Then, a new matching position L2 is obtained in the search area by using template matching.

7. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 6, characterized in that: The step S4 comprises: Step 4.1: Constrain the displacement of the previous and next frames; use the template matching algorithm to obtain the matching positions of the previous and next frames, and calculate the displacement, which is compared with the displacement vector obtained by the tracking algorithm. If the offset exceeds the threshold d0, the coefficient correction method is used to limit the displacement; Step 4.2: Since template matching is not directional, but displacement is directional, in order to ensure the accuracy of template matching, the template matching algorithm is constrained by the displacement vector; if the matching direction is inconsistent with the movement direction, the displacement obtained by the tracking algorithm is used as the benchmark; Step 4.3: When the confidence probability of the template matching algorithm is higher than a certain threshold, it is assumed that the positioning information obtained by the template matching algorithm for this frame image is more accurate, and the points with higher confidence probability of template matching are used to update the homography matrix H0 of the tracking algorithm.

8. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 7, characterized in that: In step 4.1, the displacement v′ between the position L2 and the position L0 at the previous moment is calculated, and the angle θ between v′ and the translation vector v and the displacement difference d are calculated; Then, the matching position is corrected, the direction of the angle θ is determined, and the displacement vector v′ is constrained according to the direction and displacement difference; the specific real-time scheme is as follows: In step 4.2, if the angle θ is greater than 90°, it is determined that the return value of the template matching algorithm at this moment is an incorrect position, and the motion vector at the current moment is mainly based on the tracking algorithm; In step 4.3, the position of the tracking algorithm is reversely optimized using the matching confidence probability. When the confidence probability of the template matching exceeds the set threshold, it is assumed that the template matching at this moment obtains an accurate registration position, and this position is used to reversely correct the starting position of the tracking algorithm and the homography matrix H0.

9. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 8, characterized in that: The step S5 comprises: Step 5.1: Reinitialization judgment of matching position: set a counter to count the number of times the template matching result and the tracking result have a large error. If the number of consecutive occurrences exceeds the preset threshold, the matching position is rematched globally and the tracking position is reinitialized. In this process, the current matching position is based on the result obtained by the tracking algorithm. Step 5.2: Initialization judgment of tracking feature descriptors: Count the number of feature descriptors in each tracking. When the number is lower than the minimum feature descriptor threshold d1, regenerate the feature descriptors and then perform a new round of tracking. In this frame image, the registration position is based on the result obtained by the matching algorithm.

10. The real-time heterogeneous dynamic map registration method based on template matching and tracking algorithm as claimed in claim 9, characterized in that: In the step 5.1, the matching position is re-initialized: a counter is set to count the number of times the situation in step 4.3 occurs. If it occurs multiple times in succession, and the preset threshold number is 10, it is necessary to re-match the matching position globally and re-initialize the tracking position; in this process, the current matching position is based on the result obtained by the tracking algorithm; In step 5.2, the tracking feature descriptor initialization judgment is performed: the number of tracking feature descriptors is counted each time, and the minimum feature descriptors are set to 20. When the number of descriptors is less than 20, the descriptors are reinitialized, and then a new round of tracking is performed; in this frame image, the registration position is based on the result obtained by the matching algorithm.

Images