Topological map scene recognition method and device based on density filtering and landmark saliency

Abstract

The application discloses a topological map scene recognition method and device based on density filtering and landmark saliency, effectively solves the scene recognition problem under the view angle change, extracts SIFT key points from the obtained landmarks on the basis of a target detection algorithm, fully utilizes the characteristic that the SIFT key points are robust to the view angle change, adopts a density filtering algorithm to obtain the landmarks with the view angle invariability, then, cross-authenticates the depth global descriptors of the view angle invariability landmarks in a query frame and a reference frame and compares the shape scores of the landmarks, obtains the mutually matched landmarks in the two frames, considers that the extracted landmarks only represent a small part of the image, some low-identification-degree landmarks may cause confusion, thereby negatively affecting the matching result, therefore, landmark positioning network is used to obtain the landmark saliency, and then a topological map structure based on the spatial relationship between the landmarks, the appearance and the image saliency is used to calculate the matching scores of the query frame and the reference frame.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and particularly to a topological graph scene recognition method and device based on density filtering and landmark saliency. BACKGROUND

[0002] Simultaneous localization and mapping (SLAM) is the process of placing a robot in an unknown environment and making the robot move while gradually mapping the surrounding environment. The robot needs to be able to successfully match images from the same location during repeated traversals of the environment to correct drift errors accumulated over time on the map. This is called loop closure detection (also known as visual scene recognition). When a mobile robot is in an environment with a dramatic change in viewing angle, the performance of loop closure detection decreases due to severe drift and large state estimation errors. Therefore, visual scene recognition is a very challenging problem in the case of viewing angle changes.

[0003] From recent research, some methods use landmarks and topological graphs to solve such challenges in severe viewing angle change environments, and the results show that good performance improvements are obtained. However, the extraction of landmarks is obtained by target detection, and the extracted landmarks are not robust enough to dramatic viewing angle changes, so scene recognition may fail. In addition, in the process of constructing the topological graph, these methods do not fully utilize the information of the landmarks contained in the frames. If the extracted landmarks have a low degree of recognition, it may cause confusion of the scene, ultimately leading to a matching failure of the scene recognition method. SUMMARY

[0004] To solve the deficiencies of the prior art and improve the scene recognition capability under viewing angle changes, the present application adopts the following technical solutions:

[0005] The method and device for topology graph scene recognition based on density filtering and landmark saliency can overcome the adverse effects of view angle changes on scene recognition results, and ensure the accuracy and robustness of the scene recognition system. On the basis of a target detection algorithm, the SIFT key points in the landmarks are quantified and modeled by a density filtering algorithm, the number and distribution of the key points in each landmark are calculated to obtain landmarks with view angle invariance, the problem of landmark dependence on the detector is overcome, and the performance and accuracy of scene matching are improved. Then, the depth global descriptors of the landmarks in the query frame and the reference frame are extracted, preliminary matching of the landmarks is performed through cross authentication, and false matching is eliminated by using the shape similarity of the matched landmarks. The saliency of the landmarks is obtained by using a landmark localization network, and in the process of constructing a topology graph, the matching scores of the query frame and the reference frame are calculated by encoding the attributes of whether the landmarks have high recognition, the spatial relationship between the landmarks, and the appearance of the landmarks, thereby solving the problems of scene confusion caused by low-recognition landmarks, poor robustness of the scene recognition system under view angle changes, and low accuracy.

[0006] The method for topology graph scene recognition based on density filtering and landmark saliency comprises the following steps:

[0007] Step one: using a target detection algorithm to extract landmarks from an input frame;

[0008] Step two: using a density filtering algorithm to quantify and model the SIFT key points in the landmarks, calculating the number and distribution of the key points in each landmark, and obtaining landmarks with view angle invariance by using the property that the SIFT key points have good robustness to view angle changes;

[0009] Step three: using a convolutional neural network to extract depth global descriptors of the view angle invariance landmarks in the query frame and the reference frame, and completing preliminary matching of the landmarks through cross authentication;

[0010] Step four: eliminating false matching by using the shape similarity of the matched landmarks, and obtaining the landmarks in the query frame and the reference frame that match each other;

[0011] Step five: using a landmark localization network to obtain the saliency of the landmarks;

[0012] Step six: encoding the spatial relationship of the landmarks, the landmark saliency, and the landmark appearance in the query frame and the reference frame into a topology graph structure, and calculating the similarity scores of the query frame and the reference frame, and the frame with the highest score is the matching frame.

[0013] Further, in step two, the landmarks with view angle invariance are obtained by calculating the frequency and coefficient of variation of each landmark, and the specific steps include the following steps:

[0014] Step 2.1: Quantify and model the frequency of key points in the current landmark:

[0015]

[0016] where CR represents the frequency of key points in the landmark, K h represents the number of key points, H h represents the height of the landmark, W h represents the width of the landmark, and f represents the scaling factor.

[0017] Step 2.2: Divide the landmark into a set of grids and quantify and model the distribution of key points in the current landmark:

[0018]

[0019] where std_sum represents the coefficient of variation, which is a statistical measure of the degree of variation in each grid value in the landmark and reflects the dispersion of key points, std(G) represents the standard deviation of the number of key points in the m grids, avg(G) represents the average value of the number of key points in the m grids, and m represents the number of grids.

[0020] Step 2.3: When CR is greater than a first threshold T1 and std_sum is less than a second threshold T2, the corresponding landmark has viewpoint invariance.

[0021] Further, in the step four, in order to ensure the accuracy of landmark matching, the shape similarity of the matched landmark is introduced:

[0022]

[0023] where Shape ab represents the shape similarity score of the preliminary matched landmark, w a , h a , w b , h b are the width and height of the preliminary matched landmark, respectively, a represents the landmark of the query frame A, b represents the landmark of the reference frame B, and a and b are the landmarks of the A frame and the B frame matched by the preliminary feature matching.

[0024] Further, in the step six, the calculation of the similarity score of the query frame and the reference frame includes the following steps:

[0025] Step S6.1: Calculate the similarity score of the angles in the query frame and the reference frame:

[0026]

[0027]

[0028] where w θ denotes the similarity score of the angle in the query frame and the reference frame, denotes the spatial angle similarity in the landmark spatial relationship in the query frame and the reference frame, z ii′ , z jj′ , z kk′ denotes the appearance similarity score of the three pairs of matched landmarks that construct a triangle, S i , S i′ , S j , S j′ , S k , S k′ denote the normalized landmark saliency of the three pairs of landmarks in the query frame and the reference frame, respectively, avg denotes the average, and max denotes the maximum;

[0029] Step S6.2: Calculate the similarity score of the distance in the query frame and the reference frame:

[0030]

[0031] where w d denotes the similarity score of the distance in the query frame and the reference frame, d ii′,jj′ denotes the spatial distance similarity in the landmark spatial relationship in the query frame and the reference frame, z ii′ , z jj′ denotes the appearance similarity score of the two pairs of matched landmarks that construct an edge, S i , S i′ , S j , S j′ denote the normalized landmark saliency of the two pairs of landmarks in the query frame and the reference frame, respectively, avg denotes the average, and max denotes the maximum;

[0032] Step S6.3: Since the number of matched landmarks between different frames is not the same, in order to limit the range of the similarity score to [0, 1], calculate the final similarity score between frames:

[0033]

[0034] where Score denotes the similarity score between frames, and n denotes the number of matched landmarks between frames.

[0035] Further, the spatial relationship in the step six includes a distance spatial relationship, and the distance spatial relationship is constructed as follows:

[0036] d ii′,jj′ = exp(-|e i,j -e i′,j′ |) (4)

[0037] where dii′,jj′ represents the spatial distance similarity of landmarks in the query frame and the reference frame, e i,j represents the spatial distance similarity of landmarks in the query frame and the reference frame, e i′,j′ represents the edge of the topology graph constructed by landmarks in the query frame and the reference frame, respectively, i, i' and j, j' represent two pairs of landmarks in the query frame and the reference frame, i.e. the i, i' and j, j' nodes encoded into the topology graph.

[0038] Further, the spatial relationship in the step six includes an angle spatial relationship, and the angle spatial relationship is constructed as follows:

[0039]

[0040] wherein represents the spatial angle similarity of landmarks in the query frame and the reference frame, k, k' represents a pair of landmarks in the query frame and the reference frame, i.e. the k, k' nodes encoded into the topology graph, θ u represents the angle of a triangle in the topology graph constructed by landmarks in the query frame, θ v represents the angle of a triangle in the topology graph constructed by landmarks in the reference frame.

[0041] Further, in the step six, the landmark saliency is normalized and encoded into the topology graph:

[0042]

[0043] wherein S l represents the normalized landmark saliency, s l represents the landmark saliency before normalization, min(s l ) represents the minimum value of all landmark saliencies, and max(s l ) represents the maximum value of all landmark saliencies.

[0044] Further, the key point is a SIFT key point.

[0045] Further, the grid in the step 2.2 is a group of grids with the same size.

[0046] The topology graph scene recognition device based on density filtering and landmark saliency comprises a memory and one or more processors, the memory stores executable codes, and the one or more processors execute the executable codes to implement the topology graph scene recognition method based on density filtering and landmark saliency.

[0047] The present application has the following advantages and beneficial effects:

[0048] The landmark and landmark saliency map matching method based on density filtering of the application solves the scene recognition problem of insufficient robustness of extracted landmarks and insufficient utilization of landmark information in a frame under severe view angle changes. The application obtains good experimental results in improving the scene recognition matching accuracy and robustness under view angle changes. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a method flowchart of the application.

[0050] Figure 2 is a model representation effect diagram of the application using the density filtering method to obtain landmarks with view angle invariance.

[0051] Figures 3a-3c is a landmark saliency visualization effect diagram of the application.

[0052] Figure 4 is a demonstration effect diagram of the application constructing a topological graph for frame matching.

[0053] Figure 5 is a comparison result table of the AUC values of the method of the application and other methods.

[0054] Figure 6 is a comparison result table of the R values of the method of the application and other methods. P＝100

[0055] Figure 7 is a scene under different severe view angle changes designed by the application.

[0056] Figure 8 is a comparison result table of the AUC values of the method of the application and other methods under severe view angle changes.

[0057] Figure 9 is a comparison result table of the R values of the method of the application and other methods under severe view angle changes. P＝100

[0058] Figure 10 is a device structure schematic diagram of the application. DETAILED DESCRIPTION

[0059] The specific embodiments of the application are described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0060] For scene recognition technology under view angle changes, it is necessary to improve the scene matching accuracy while increasing the robustness of the technology, which is of practical significance. The application provides a method based on landmarks, landmark saliency and topological graph combination. The algorithm flow is as shown in Figure 1 .​​

[0061] On the basis of the target detection algorithm, the SIFT key points of the obtained landmarks are extracted, the characteristic that the SIFT key points are robust to the view angle change is fully utilized, and a designed density filtering algorithm is adopted to convert the obtained landmarks with the view angle invariance into the distribution and quantity problems of the SIFT key points. Then, the depth global descriptors of the view angle invariance landmarks in the query frame and the reference frame are extracted, the preliminary matching of the landmarks is completed through cross authentication, then the shape similarity of the matched landmarks is utilized to eliminate the false matching, and the mutually matched landmarks in the two frames are obtained. Considering that the low-recognizability landmarks will cause the confusion of the matching scene and the good performance of the topological graph structure in solving the scene recognition problem under the view angle change, the application proposes a method of combining the landmark saliency with the construction of the topological graph. The landmark positioning network (Landmark Localization Network) is applied to obtain the saliency of the landmarks, the attribute whether the landmarks have high recognizability, the spatial relationship between the landmarks and the appearance are fully utilized to construct the topological graph structure to calculate the matching scores of the query frame and the reference frame. The application has high accuracy and robustness in the scene recognition task under the view angle change.

[0062] A topological graph scene recognition method based on density filtering and landmark saliency, the specific implementation steps are as follows:

[0063] Step one: as shown in the figure, the existing target detection algorithm is used to extract the landmarks and the SIFT key points from the input query frame. Figure 2

[0064] Step two: the attribute that the SIFT key points are robust to the view angle change is utilized to convert the obtained landmarks with the view angle invariance into the quantity and distribution problems of the SIFT key points in the current landmarks. The key points in the landmarks are quantized and modeled through the density filtering algorithm, and the landmarks with the view angle invariance are obtained by calculating the frequency and coefficient of variation of each landmark.

[0065] Specifically, the problem of obtaining the landmarks with the view angle invariance is converted into the quantity and distribution problems of the SIFT key points in the current landmarks. The steps include the following steps:

[0066] Step 2.1: firstly, the frequency of the SIFT key points in the current landmark is quantized and modeled.

[0067]

[0068] Wherein CR is used to quantize and model the frequency of the SIFT key points in the landmark, K h is the number of SIFT key points, H h represents the height of the landmark, W h represents the width of the landmark, and f is a scaling factor. ​

[0069] Step 2.2: Divide the landmark into 9 grids of the same size, and quantize and model the distribution of SIFT key points in the current landmark.

[0070]

[0071] Where std_sum is the coefficient of variation, which is a statistical measure of the degree of variation in each grid value in the landmark, and reflects the dispersion degree of key points, std(G) is the standard deviation of the number of SIFT key points of the 9 grids, and avg(G) is the average value of the number of SIFT key points of the 9 grids.

[0072] Step 2.3: If CR is greater than threshold T1, and std_sum is less than threshold T2, the landmark has view invariance.

[0073] Step three: The application uses a convolutional neural network to extract a deep global descriptor for each frame of the landmark, and completes the preliminary matching of the landmark through cross authentication.

[0074] Specifically, for the view-invariant landmark in the query frame and the reference frame, a deep global descriptor is extracted, and the preliminary matching of the landmark is completed through cross authentication.

[0075] Step four, the shape similarity of the matched landmark is used to eliminate false matches, and the mutually matched landmarks in the two frames are obtained.

[0076] Specifically, in order to ensure the accuracy of landmark matching, the shape similarity of the matched landmark is introduced

[0077]

[0078] Where Shape ab is the shape similarity score of the preliminary matching landmark, w a , h a , w b , h b are the width and height of the preliminary matching landmark, respectively.

[0079] Step five: The application applies a landmark localization network (Landmark Localization Network) to obtain the saliency of the landmark, aiming at the problem that low recognition degree of the landmark will cause confusion of the matching scene, as shown in Figures 3a-3c

[0080] Step six, the application encodes the landmark saliency, the spatial relationship between landmarks, and the appearance of the landmark into a topological graph structure, and the effect of the construction process is shown in Figure 4 The reference frame with the highest score is the final matching frame by calculating the similarity score of the query frame and the reference frame.

[0081] ​Specifically, the spatial relations of landmarks in the query frame and the reference frame are encoded into the topological graph, which includes distance spatial relations and angle spatial relations.

[0082] Firstly, the distance spatial relations are constructed as follows:

[0083] d ii′,jj′ = exp(-|e i,j -e i′,j′ |) (4)

[0084] where d ii′,jj′ represents the spatial distance similarity of landmarks in the query frame and the reference frame, e i,j , e i′,j′ are the edges of the topological graph constructed by landmarks in the query frame and the reference frame, respectively.

[0085] Secondly, the angle spatial relations are constructed as follows:

[0086]

[0087] where represents the spatial angle similarity of landmarks in the query frame and the reference frame, u∈{i,j,k}, v∈{i',j',k'}, θ u , θ v are the angles of the triangle in the topological graph constructed by landmarks in the query frame and the reference frame, respectively.

[0088] The landmark saliency is introduced into the topological graph construction. Firstly, the landmark saliency is normalized as follows:

[0089]

[0090] where S l represents the normalized landmark saliency, s l represents the landmark saliency before normalization, min(s l ) represents the minimum value of all landmark saliencies, and max(s l ) represents the maximum value of all landmark saliencies.

[0091] In order to get the correct matching frame, we construct the topological graph structure based on the spatial relations between landmarks, the appearance similarity scores and the image saliency to calculate the matching scores of the query frame and the reference frame. The final similarity score between frames is calculated.

[0092]

[0093] where w θ represents the angle similarity score between the query frame and the reference frame, which is calculated by formula (5), z ii ', zjj′ , z kk′ represent appearance similarity scores of the three pairs of matched landmarks that form a triangle, S i , S i′ , S j , S j′ , S k , S k′ are calculated by formula (6), avg represents taking average, and max represents taking maximum.

[0094]

[0095] wherein, w d represent appearance similarity scores of the two pairs of matched landmarks that form an edge, S ii′,jj′ are calculated by formula (4), z ii′ , z jj′ represent appearance similarity scores of the two pairs of matched landmarks that form an edge, S i , S i′ , S j , S j′ are calculated by formula (6), avg represents taking average, and max represents taking maximum.

[0096] Since the number of matched landmarks between different frames is not the same, in order to limit the range of similarity scores to [0, 1], the final similarity score between frames is:

[0097]

[0098] wherein, Score represents the similarity score between frames, and n represents the number of matched landmarks between frames.

[0099] The present application aims at the problem that the scene recognition system is not robust under the change of view angle. The result of scene matching is shown in Figs. 1 and 2. Figure 5 , Figure 6 From the displayed result, the algorithm of the present application reduces the influence of the change of view angle on scene recognition. In order to verify the performance of the present application under the severe change of view angle, the present application increases the severity of the change of view angle by adding translation, rotation, and combination of the two to the scene, as shown in Fig. 3. Figure 7 The result of scene recognition under the severe change of view angle is shown in Figs. 4 and 5. Figure 8 , Figure 9 The result shows that the algorithm of the present application is better.

[0100] In summary, the topology scene recognition algorithm based on the density filtering and landmark saliency can overcome the adverse effects of the view angle change on the scene recognition result, solve the problems of the scene matching confusion caused by the dependence of the landmark on the detector and the low-identification landmark, and poor performance of the scene recognition system, and ensure the accuracy and robustness of the scene recognition.

[0101] The present application fully considers the limitations of scene recognition under the view angle change, and provides an algorithm based on the density filtering algorithm, landmark saliency and topology graph. Through the designed density filtering algorithm, the landmark problem with the view angle invariance is converted into the number and distribution of the SIFT key points in the current landmark on the basis of the existing target detection algorithm, and the problems of the dependence of the landmark extraction on the detector and the non-robustness of the landmark to the view angle change are overcome. In order to solve the problem of the scene matching confusion caused by the low-identification landmark, the landmark saliency is obtained by using the landmark localization network, and the landmark saliency, the spatial relationship between the landmarks and the appearance of the landmark are encoded into the topology graph structure to match the query frame and the reference frame. The present application ensures the accuracy and strong robustness of the scene recognition system under the view angle change by using the designed density filtering algorithm, landmark saliency and topology graph, and provides a reliable guarantee for the operation of the method.

[0102] Corresponding to the foregoing embodiment of the topology scene recognition method based on the density filtering and landmark saliency, the present application further provides an embodiment of a topology scene recognition device based on the density filtering and landmark saliency.

[0103] Referring to Figure 10 The topology scene recognition device based on the density filtering and landmark saliency provided by the embodiment of the present application includes a memory and one or more processors, the memory stores executable codes, and the one or more processors execute the executable codes to implement the topology scene recognition method based on the density filtering and landmark saliency in the foregoing embodiment.

[0104] The embodiment of the topology scene recognition device based on the density filtering and landmark saliency of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. Taking the software realization as an example, as a logically meaningful device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability. From the hardware level, for example, Figure 10As shown in the figure, it is a hardware structure diagram of any data processing capable device where the device based on density filtering and landmark saliency topology map scene recognition apparatus of the present application is located, in addition to Figure 10 In addition to the processor, the memory, the network interface, and the nonvolatile memory shown in the figure, any data processing capable device where the device in the embodiment is located can also include other hardware according to the actual functions of the data processing capable device, and no further description is given.

[0105] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, and no further description is given.

[0106] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the part of the method embodiment. The device embodiment described above is only illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0107] The embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the density filtering and landmark saliency topology map scene recognition method in the above embodiment.

[0108] The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of any data processing capable device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0109] The above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for topological map scene recognition based on density filtering and landmark saliency, characterized in that The method comprises the following steps: Step 1: extracting landmarks from input frames using a target detection algorithm; Step 2: quantifying and modeling key points in the landmarks by using a density filtering algorithm, calculating the number and distribution of key points in each landmark, and obtaining landmarks with perspective invariance; by calculating the frequency and coefficient of variation of each landmark, the landmark frequency is the frequency of key points based on the number of key points, the shape of the landmark, and the scaling ratio of the key points in the quantified and modeled landmark, and the coefficient of variation is a statistical measure of the degree of change of each grid value in the landmark, reflecting the dispersion degree of key points, when the landmark frequency is greater than the first threshold value and the coefficient of variation is less than the second threshold value, the corresponding landmark has perspective invariance; Step 3: extracting deep global descriptors of the perspective invariance landmarks in the query frame and the reference frame using a convolutional neural network, and completing the preliminary matching of the landmarks through cross authentication; Step 4: eliminating false matches by using the shape similarity of the matched landmarks to obtain the landmarks in the query frame and the reference frame that match each other; Step 5: obtaining the saliency of the landmarks using a landmark positioning network; Step 6: encoding the spatial relationship, landmark saliency, and landmark appearance of the landmarks in the query frame and the reference frame into a topological graph structure, and calculating the similarity score of the query frame and the reference frame, and the frame with the highest score is the matching frame. 2.The method according to claim 1, wherein In step 2, the landmark with perspective invariance is obtained by calculating the frequency and coefficient of variation of each landmark, and the specific steps include the following steps: Step 2.1: quantifying and modeling the key point frequency in the current landmark; where CR represents the frequency of key points in the quantized and modeled landmarks, K h represents the number of key points, H h represents the height of the landmark, W h represents the width of the landmark, and f represents a scaling factor; Step 2.2: quantifying and modeling the key point distribution in the current landmark by dividing the landmark into a set of grids; Where std_sum represents the coefficient of variation, which is a statistical measure of the degree of change of each grid value in the landmark, and reflects the dispersion degree of key points, std(G) represents the standard deviation of the number of key points in m grids, avg(G) represents the average value of the number of key points in m grids, and m represents the number of grids; Step 2.3: when CR is greater than the first threshold value T1 and std_sum is less than the second threshold value T2, the corresponding landmark has perspective invariance. 3.The method of claim 1, wherein In step 4, the shape similarity of the matched landmarks is introduced. where Shape ab represents the shape similarity score of the preliminary matched landmark, w a , h a , w b , h b are the width and height of the preliminary matched landmark, respectively, a represents the landmark of the query frame, and b represents the landmark of the reference frame. 4.The method of claim 1, wherein In step 6, the calculation of the similarity score of the query frame and the reference frame includes the following steps: Step S6.1: calculating the similarity score of the angles in the query frame and the reference frame: where w θ denotes the similarity score of the angle in the query frame and the reference frame, denotes the spatial angle similarity in the landmark spatial relationship in the query frame and the reference frame, z ii′ , z jj′ , z kk′ denotes the appearance similarity score of the three pairs of matching landmarks that construct a triangle, S i , S i′ , S j , S j′ , S k , S k′ denote the normalized landmark saliencies of the three pairs of landmarks in the query frame and the reference frame, respectively, avg denotes the average, and max denotes the maximum. Step S6.2: calculating the similarity score of the distances in the query frame and the reference frame: where w d denotes the similarity score of the distance in the query frame and the reference frame, d ii′,jj′ denotes the spatial distance similarity in the landmark spatial relationship in the query frame and the reference frame, z ii′ , z jj′ denotes the appearance similarity score of the two pairs of matched landmarks that construct the edge, S i , S i′ , S j , S j′ denote the normalized landmark saliencies of the two pairs of landmarks in the query frame and the reference frame, respectively, avg denotes the average, and max denotes the maximum. Step S6.3: calculating the final similarity score between the frames: Where Score represents the similarity score between the frames, and n represents the number of matched landmarks between the frames.

5. The method according to claim 1 or 4, wherein The spatial relationship in step 6 includes the distance spatial relationship, and the distance spatial relationship is constructed as follows: d ii′,jj′ = exp(-|e i,j -e i′,j′ |) (4) where d ii′,jj′ represents the spatial distance similarity of landmarks in the query frame and the reference frame, e i,j , e i′,j′ represent edges of the topological graph constructed from landmarks in the query frame and the reference frame, respectively, and i, i' and j, j' represent pairs of landmarks in the query frame and the reference frame, respectively, i.e., the i, i' and j, j' nodes encoded into the topological graph.

6. The method according to claim 1 or 4, wherein The spatial relationship in step 6 includes the angle spatial relationship, and the angle spatial relationship is constructed as follows: wherein denotes the spatial angular similarity of landmarks in the query and reference frames, k, k' denotes a pair of landmarks in the query and reference frames, i.e. the k, k' -th node encoded in the topological graph, θ u denotes the angle of a triangle in the topological graph constructed from landmarks in the query frame, u e {i, j, k}, θ v denotes the angle of a triangle in the topological graph constructed from landmarks in the reference frame, v e {i', j', k'}.

7. The method according to claim 1 or 4, wherein In step 6, the landmark saliency is normalized and encoded into the topological graph structure: where S l denotes the normalized landmark saliency, s l denotes the landmark saliency before normalization, min(s l ) denotes the minimum value among all landmark saliencies, and max(s l ) denotes the maximum value among all landmark saliencies.

8. The method of claim 1, wherein The key points are SIFT key points.

9. The method of claim 2, wherein the method further comprises: determining a density of the landmarks in the image; and determining a density of the landmarks in the image based on the density of the landmarks in the image. The grid in step 2.2 is a set of grids with the same size.

10. A device for topology map scene recognition based on density filtering and landmark saliency, characterized in that, The device comprises a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the method for identifying a topological map scene based on density filtering and landmark saliency according to any one of claims 1-9.

Images