A method and related device for identifying scenic spot pictures

Through the convolutional neural network based on attention mechanism, the local features and feature point index of attractions are extracted, and the sampling consistency verification algorithm is combined with the problem of low accuracy and efficiency in the recognition of scenic spots by users taking pictures of attractions by themselves, and the improvement is achieved Efficient attraction recognition effect.

CN111797254BActive Publication Date: 2024-12-06BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910276394.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-04-08
Publication Date
2024-12-06
Estimated Expiration
2039-04-08

AI Technical Summary

Technical Problem

The prior art has low accuracy and efficiency in the recognition of scenic spots photographs taken by users. This is mainly because the pictures taken by users include non-scenic spots or only some scenic spots, resulting in a large difference from the pictures of online attractions, resulting in scenic spots. Inaccurate identification.

Method used

The local feature model of scenic spots obtained by using convolutional neural network training based on attention mechanism is extracted, and the local features of the target scenic spots are obtained through feature point indexes, and the pictures of candidate scenic spots are filtered from the scenic spot image database, and combined with the sampling consistency verification algorithm to improve Identification accuracy.

Benefits of technology

It significantly improves the accuracy and efficiency of users' identification of attractions pictures taken by themselves, and improves the search speed and accuracy of attractions recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111797254B_ABST
    Figure CN111797254B_ABST
Patent Text Reader

Abstract

The present application discloses a method and related device for identifying scenic spot pictures, which includes: inputting a target scenic spot picture into a scenic spot local feature model obtained by pre-training a convolutional neural network based on an attention mechanism according to training samples, and obtaining the target scenic spot local features including N target feature points; obtaining multiple similar feature points corresponding to the N target feature points from the feature point index; obtaining candidate scenic spot pictures from a scenic spot picture database based on the multiple similar feature points of the N target feature points and the feature point index; and determining the scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the scenic spot local features can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, and significantly improve the recognition effect of the scenic spot pictures taken by the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and related device for scenic spot image recognition. Background Art

[0002] With the rapid development of image recognition technology, it is increasingly widely used in daily life and work. Especially in some scenarios, such as identifying scenes by looking at pictures, for scenic spot pictures, it is necessary to identify the scenic spots in them.

[0003] In the prior art, scenic spots in scenic spot pictures are generally identified based on the scenic spot picture to be identified, and similar or close scenic spot pictures are searched and recalled on the Internet to obtain scenic spot information corresponding to the similar or close scenic spot pictures, and the corresponding scenic spot names are determined as the scenic spot identification results of the above-mentioned scenic spot picture to be identified.

[0004] However, the above-mentioned scenic spot picture recognition method is only effective for recognizing online scenic spot pictures, and is poor for recognizing scenic spot pictures taken by users themselves. This is mainly because the scenic spot pictures taken by users themselves contain non-scenic spot areas or only contain part of the scenic spot areas, which are quite different from online scenic spot pictures, resulting in inaccurate scenic spot recognition. However, the scenic spot pictures in most scenic spot recognition tasks are taken by users themselves. Using the above-mentioned scenic spot picture recognition method, the scenic spot recognition efficiency and recognition accuracy are low. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a method and related devices for scenic spot picture recognition, which can better explore the scenic spot features in scenic spot pictures, improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves.

[0006] In a first aspect, an embodiment of the present application provides a method for identifying a scenic spot image, the method comprising:

[0007] Obtaining local features of a target scenic spot according to a target scenic spot picture and a local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local features of the target scenic spot include N target feature points;

[0008] Obtaining a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index;

[0009] Obtaining a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index;

[0010] Based on the scenic spot entries corresponding to the candidate scenic spot pictures, a scenic spot recognition result of the target scenic spot picture is determined.

[0011] Optionally, obtaining a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index includes:

[0012] Based on the similarity between each target feature point and each feature point in the feature point index, obtaining a plurality of similar feature points corresponding to each target feature point;

[0013] For the N target feature points, multiple similar feature points of each target feature point are filtered based on the coincidence relationship between multiple similar feature points of each target feature point, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained.

[0014] Optionally, obtaining a plurality of similar feature points corresponding to each target feature point based on the similarity between each target feature point and each feature point in the feature point index includes:

[0015] Calculating the similarity between each target feature point and each feature point in the feature point index;

[0016] According to the similarity and the similarity threshold, a plurality of similar feature points corresponding to each target feature point are obtained from the feature point index.

[0017] Optionally, obtaining a candidate scenic spot picture from a scenic spot picture database based on the multiple similar feature points of the N target feature points and the feature point index includes:

[0018] Based on the multiple similar feature points of the N target feature points and the feature point index, searching the scenic spot picture database to obtain multiple similar scenic spot pictures;

[0019] According to the target scenic spot picture, the multiple similar scenic spot pictures and a sampling consistency verification algorithm, the multiple similar scenic spot pictures are filtered to obtain candidate scenic spot pictures.

[0020] Optionally, searching the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index includes:

[0021] Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points;

[0022] The multiple scenic spot pictures are directly determined as the multiple similar scenic spot pictures.

[0023] Optionally, searching the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index includes:

[0024] Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points;

[0025] Determining the number of similar feature points included in each of the plurality of scenic spot pictures;

[0026] According to the number of similar feature points included in each of the scenic spot pictures, the multiple scenic spot pictures are screened to obtain the multiple similar scenic spot pictures.

[0027] Optionally, the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein a preset initial sample point is determined based on the similarity corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is a preset number of iterations.

[0028] Optionally, if the candidate scenic spot picture is a plurality of candidate scenic spot pictures, determining the scenic spot recognition result of the target scenic spot picture based on the scenic spot entries corresponding to the candidate scenic spot pictures includes:

[0029] Obtaining candidate scenic spot entries according to the scenic spot entries corresponding to the plurality of candidate scenic spot images;

[0030] The scenic spot recognition result is determined from the candidate scenic spot entries based on the correspondence between the multiple candidate scenic spot pictures and the candidate scenic spot entries and / or the number of similar feature points included in each of the multiple candidate scenic spot pictures.

[0031] Optionally, the feature point index is pre-constructed for feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures.

[0032] In a second aspect, an embodiment of the present application provides a device for identifying a scenic spot image, the device comprising:

[0033] A first obtaining unit is used to obtain the local features of the target scenic spot according to the target scenic spot picture and the local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local features of the target scenic spot include N target feature points;

[0034] A second obtaining unit, configured to obtain a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index;

[0035] A third obtaining unit, configured to obtain a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index;

[0036] The determination unit is used to determine the scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture.

[0037] Optionally, the second obtaining unit includes:

[0038] A first obtaining subunit, configured to obtain a plurality of similar feature points corresponding to each target feature point based on a similarity between each target feature point and each feature point in the feature point index;

[0039] The second obtaining subunit is used to filter the multiple similar feature points of each target feature point based on the coincidence relationship between the multiple similar feature points of each target feature point, and comprehensively obtain multiple similar feature points corresponding to the N target feature points.

[0040] Optionally, the first obtaining subunit includes:

[0041] A first obtaining module, used for calculating the similarity between each target feature point and each feature point in the feature point index;

[0042] The second obtaining module is used to obtain a plurality of similar feature points corresponding to each target feature point from the feature point index according to the similarity and the similarity threshold.

[0043] Optionally, the third obtaining unit includes:

[0044] A third obtaining subunit is used to search the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index;

[0045] The fourth obtaining subunit is used to filter the multiple similar scenic spot pictures to obtain candidate scenic spot pictures according to the target scenic spot picture, the multiple similar scenic spot pictures and a sampling consistency verification algorithm.

[0046] Optionally, the third obtaining subunit includes:

[0047] A first search module, configured to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0048] The first determining module is used to directly determine the plurality of scenic spot pictures as the plurality of similar scenic spot pictures.

[0049] Optionally, the third obtaining subunit includes:

[0050] A second search module is used to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0051] A second determining module is used to determine the number of similar feature points included in each of the plurality of scenic spot pictures;

[0052] The third obtaining module is used to screen the plurality of scenic spot pictures to obtain the plurality of similar scenic spot pictures according to the number of similar feature points included in each scenic spot picture.

[0053] Optionally, the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein a preset initial sample point is determined based on the similarity corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is a preset number of iterations.

[0054] Optionally, if the candidate scenic spot picture is a plurality of candidate scenic spot pictures, the determining unit includes:

[0055] A fifth obtaining subunit, configured to obtain a candidate scenic spot entry according to the scenic spot entries corresponding to the plurality of candidate scenic spot images;

[0056] The determination subunit is used to determine the scenic spot recognition result from the candidate scenic spot entries based on the correspondence between the multiple candidate scenic spot pictures and the candidate scenic spot entries and / or the number of similar feature points included in each candidate scenic spot picture in the multiple candidate scenic spot pictures.

[0057] Optionally, the feature point index is pre-constructed for feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures.

[0058] In a third aspect, an embodiment of the present application provides a device for scenic spot image recognition, the device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations:

[0059] Obtaining local features of a target scenic spot according to a target scenic spot picture and a local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local features of the target scenic spot include N target feature points;

[0060] Obtaining a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index;

[0061] Obtaining a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index;

[0062] Based on the scenic spot entries corresponding to the candidate scenic spot pictures, a scenic spot recognition result of the target scenic spot picture is determined.

[0063] In a fourth aspect, an embodiment of the present application provides a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, enables the device to perform a method for identifying scenic spot images as described in one or more of the first aspects above.

[0064] Compared with the prior art, this application has at least the following advantages:

[0065] Adopting the technical solution of the embodiment of the present application, first, the target scenic spot picture is input into the scenic spot local feature model obtained by pre-training the convolutional neural network based on the attention mechanism according to the training samples, and the target scenic spot local features including N target feature points are obtained; secondly, for the N target feature points, multiple similar feature points corresponding to the N target feature points are obtained from the feature point index; then, based on the multiple similar feature points and feature point index of the N target feature points, the candidate scenic spot picture is obtained from the scenic spot picture database; finally, the scenic spot recognition result of the target scenic spot picture is determined based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the local features of the scenic spot can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 A schematic diagram of a system framework involved in an application scenario in an embodiment of the present application;

[0068] Figure 2 A flowchart of a method for identifying scenic spot images provided in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of a target scenic spot image recognition interface provided in an embodiment of the present application;

[0070] Figure 4 A flowchart of another method for identifying scenic spot images provided in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of the structure of a device for identifying scenic spot pictures provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of the structure of a device for scenic spot image recognition provided in an embodiment of the present application;

[0073] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0075] The traditional method of identifying scenic spot pictures refers to searching and recalling scenic spot pictures that are similar or close to the scenic spot pictures to be identified on the Internet, and determining the scenic spot name as the scenic spot identification result of the scenic spot pictures to be identified based on the scenic spot information corresponding to the similar or close scenic spot pictures. However, after research, the inventors found that the scenic spot pictures taken by users themselves contain more non-scenic spot areas, which are quite different from the scenic spot pictures on the Internet. The above-mentioned scenic spot picture recognition method is inaccurate for scenic spot pictures taken by users themselves, and the recognition effect is poor. However, most scenic spot pictures in scenic spot recognition tasks are taken by users themselves. The use of the above-mentioned scenic spot picture recognition method results in low scenic spot recognition efficiency and recognition accuracy.

[0076] In order to solve this problem, in an embodiment of the present application, first, the target scenic spot picture is input into the scenic spot local feature model obtained by pre-training the convolutional neural network based on the attention mechanism according to the training samples, and the target scenic spot local features including N target feature points are obtained; secondly, for the N target feature points, multiple similar feature points corresponding to the N target feature points are obtained from the feature point index; then, based on the multiple similar feature points and feature point index of the N target feature points, the candidate scenic spot picture is obtained from the scenic spot picture database; finally, the scenic spot recognition result of the target scenic spot picture is determined based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the local features of the scenic spot can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves.

[0077] For example, one of the scenarios of the embodiments of the present application may be applied to Figure 1 In the scene shown, the scene includes a user terminal 101, a processor 102 and a scenic spot picture database 103, wherein the user terminal 101 may be a PC or other mobile terminals, such as a mobile phone or a platform computer, etc. The user confirms the target scenic spot picture to be identified through the user terminal 101 and sends it to the processor 102. The scenic spot picture database 103 includes scenic spot pictures and corresponding scenic spot entries, and the feature points included in the local features of the scenic spot pictures are constructed into a feature point index.

[0078] The processor 102 obtains the local features of the target scenic spot including N target feature points based on the target scenic spot picture and the local feature model of the scenic spot pre-trained by a convolutional neural network based on the attention mechanism according to the training samples; the processor 102 obtains multiple similar feature points corresponding to the N target feature points based on the N target feature points and the feature point indexes; the processor 102 obtains a candidate scenic spot picture from the scenic spot picture database 103 based on the multiple similar feature points and the feature point indexes of the N target feature points; the processor 102 determines the scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture.

[0079] It can be understood that in the above application scenario, although the action description of the implementation mode of the present application is executed by the processor 102, the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation mode of the present application are executed.

[0080] It can be understood that the above scenario is only an example scenario provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.

[0081] The specific implementation of the method and related devices for scenic spot image recognition in the embodiment of the present application will be described in detail below with reference to the accompanying drawings by way of examples.

[0082] Exemplary Methods

[0083] See also Figure 2 , shows a flow chart of a method for scenic spot image recognition in an embodiment of the present application. In this embodiment, the method may include the following steps:

[0084] Step 201: obtaining the local features of the target scenic spot according to the target scenic spot picture and the local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local features of the target scenic spot include N target feature points.

[0085] When the target scenic spot picture is a scenic spot picture taken by the user himself, the target scenic spot picture contains non-scenic spot areas, and it is impossible to accurately obtain the scenic spot features of the scenic spot area in the target scenic spot picture. In an embodiment of the present application, first, a large number of training samples including training scenic spot pictures and scenic spot logos can be obtained, and a convolutional neural network based on an attention mechanism is trained based on a large number of training samples to obtain a scenic spot local feature model for extracting local features of scenic spots in scenic spot pictures in advance. Then, after determining the target scenic spot picture, the target scenic spot picture is input into the scenic spot local feature model obtained by the above-mentioned pre-training, and the target scenic spot local features composed of N target feature points corresponding to the target scenic spot picture can be obtained.

[0086] Among them, it should be noted that the attention mechanism and convolutional neural network are used in the process of obtaining the local feature model of the scenic spot. The local feature model of the scenic spot has higher extraction accuracy than other feature extraction methods, that is, the local features of the target scenic spot obtained by using the local feature model of the scenic spot have higher accuracy.

[0087] Step 202: obtaining a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index.

[0088] It is understandable that after obtaining the local features of the target scenic spot including N target feature points in step 201, it is necessary to search for scenic spot pictures that are similar or close to the target scenic spot pictures based on the N target feature points. Since a large number of scenic spot pictures and scenic spot entries are correspondingly stored in the database in the embodiment of the present application, which is recorded as a scenic spot picture database, it is necessary to search for similar scenic spot pictures in the scenic spot picture database based on the N target feature points. In practical applications, first, it is necessary to search for multiple feature points similar to the N target feature points based on the N target feature points, and then search for scenic spot pictures including feature points similar to the N target feature points in the scenic spot picture database. Therefore, searching for multiple feature points similar to the N target feature points based on the N target feature points is actually searching for multiple similar feature points similar to the N target feature points from the feature points included in the local features of the scenic spots of each scenic spot picture in the scenic spot picture database. In order to achieve fast and convenient search, a feature point index can be constructed in advance based on the feature points included in the local features of the scenic spots of each scenic spot picture and the corresponding relationship between the feature points and the scenic spot pictures.

[0089] Among them, the relationship between the scenic spot names included in each scenic spot entry in the scenic spot picture database is relatively complex. The parent scenic spot generally includes multiple sub-scenic spots. In order to avoid the same scenic spot picture corresponding to both the scenic spot entry representing the parent scenic spot name and the scenic spot entry representing the sub-scenic spot name, which is not convenient for subsequent judgment of the corresponding situation, the scenic spot entry is stored in a tree structure. Therefore, in some implementations of the embodiments of the present application, the scenic spot picture database includes scenic spot pictures and corresponding scenic spot entries, and each of the scenic spot entries is stored in a tree structure; the feature point index is pre-constructed for the feature points included in the local features of the scenic spot of the scenic spot picture in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot picture. Specifically, multiple feature points are obtained by integrating the feature points included in the local features of the scenic spot of each scenic spot picture in the scenic spot picture database; the corresponding relationship between each feature point and the scenic spot picture to which it belongs is determined based on the scenic spot picture to which each feature point belongs; and the feature point index is pre-constructed based on multiple feature points and the corresponding relationship between each feature point and the scenic spot picture, taking the feature point as a unit.

[0090] Based on the above description, it can be understood that the multiple similar feature points of N target feature points actually need to be obtained from the feature point index and synthesized, wherein the multiple similar feature points of each target feature point are selected from the feature point index according to the similarity between each target feature point and each feature point in the feature point index, and because there is a certain overlap relationship between the multiple similar feature points of each target feature point, filtering based on the overlap relationship is required before synthesis. Therefore, in some implementations of the embodiments of the present application, the step 202 may, for example, include the following steps:

[0091] Step A: Based on the similarity between each target feature point and each feature point in the feature point index, a plurality of similar feature points corresponding to each target feature point are obtained.

[0092] For step A, since the similarity between each target feature point and each feature point is generally different, the higher the similarity, the more similar the target feature point is to the feature point; then, after calculating the similarity between each target feature point and each feature point in the feature point index, a similarity threshold is set after the calculation to distinguish the high and low similarities, and then based on the calculated similarities and the high and low similarity thresholds, multiple similar feature points corresponding to each target feature point are obtained. For each target feature point, in order to obtain relatively similar feature points, the similarity between multiple similar feature points of any target feature point and the target feature point should be higher than the similarity threshold. Therefore, in some implementations of the embodiments of the present application, step A may, for example, include the following steps:

[0093] Step A1: Calculate the similarity between each target feature point and each feature point in the feature point index;

[0094] Step A2: According to the similarity and the similarity threshold, a plurality of similar feature points corresponding to each target feature point are obtained from the feature point index.

[0095] Step B: for the N target feature points, multiple similar feature points of each target feature point are filtered based on the coincidence relationship between multiple similar feature points of each target feature point, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained.

[0096] Among them, the coincidence relationship between multiple similar feature points of each target feature point means that any two similar feature points corresponding to the same scenic spot picture among the multiple similar feature points of each target feature point have a coincidence relationship, and it is necessary to filter out the similar feature points with lower similarity to the target feature point. For example, among the multiple similar feature points of the target feature point u, similar feature point a and similar feature point b correspond to the same scenic spot picture, then similar feature point a and similar feature point b have a coincidence relationship, if the similarity between the target feature point u and similar feature point a is greater than the similarity between the target feature point u and similar feature point b, then it is necessary to filter out the similar feature point b among the multiple similar feature points of the target feature point u.

[0097] Step 203: obtaining candidate scenic spot pictures from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index.

[0098] It can be understood that since the multiple similar feature points of the N target feature points obtained in step 202 refer to the combination of multiple similar feature points with high similarity to each of the N target feature points, after step 202, based on the multiple similar feature points of the N target feature points and the feature point index, scenic spot pictures that are relatively similar to the target scenic spot pictures can be searched in the scenic spot picture database and recorded as candidate scenic spot pictures.

[0099] In the embodiment of the present application, candidate scenic spot pictures can be obtained from the scenic spot picture database based on multiple similar feature points of N target feature points and feature point indexes in at least the following two ways.

[0100] In the implementation of the first optional step 203, since the multiple similar feature points of the N target feature points have a high degree of similarity with the corresponding target feature points among the N target feature points, the scenic spot pictures corresponding to each of the above similar feature points searched from the scenic spot picture database based on the feature point index can be used as candidate scenic spot pictures.

[0101] As an example, 20 scenic spot pictures are obtained by searching from a scenic spot picture database using multiple similar feature points and feature point indexes of N target feature points, and all 20 scenic spot pictures are directly used as candidate scenic spot pictures.

[0102] In the implementation of the second optional step 203, for any similar feature point among the multiple similar feature points of the N target feature points, the scenic spot picture database is searched based on the feature point index to obtain the scenic spot picture. Although the similar feature point (or other similar feature point) of the scenic spot picture has a high similarity with the corresponding target feature point (or other target feature point) of the target scenic spot picture, the scenic spot picture is not substantially similar to the target scenic spot picture. For example, the target scenic spot picture is a picture of the Bird's Nest, and the scenic spot picture corresponding to a similar feature point among the multiple similar feature points of its N target feature points is a picture of the Eiffel Tower. The two pictures are only similar in one or some feature points but not substantially similar. Then the scenic spot picture obtained by searching from the scenic spot picture database based on the multiple similar feature points and feature point index of the N target feature points is first recorded as a similar scenic spot picture, and the sampling consistency verification algorithm is subsequently used to verify its substantial similarity with the target scenic spot picture, and the scenic spot pictures that are only similar in one or some feature points but not substantially similar can be filtered, and the filtered similar scenic spot pictures are used as the above-mentioned candidate scenic spot pictures. Therefore, in some implementations of the embodiments of the present application, the step 203 may, for example, include the following steps:

[0103] Step C: Based on the multiple similar feature points of the N target feature points and the feature point index, search the scenic spot picture database to obtain multiple similar scenic spot pictures.

[0104] In the embodiment of the present application, a plurality of similar scenic spot pictures can be obtained by searching a scenic spot picture database based on a plurality of similar feature points of N target feature points and a feature point index in at least the following two ways.

[0105] In the first implementation, each of the similar feature points of the N target feature points can be used as a similar scenic spot picture to the target scenic spot picture, and then multiple scenic spot pictures are searched based on the feature point index and the multiple similar feature points of the N target feature points and directly determined as similar scenic spot pictures. Therefore, in some implementations of the embodiments of the present application, the step C may include the following steps, for example:

[0106] Step C1: searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0107] Step C2: directly determining the plurality of scenic spot pictures as the plurality of similar scenic spot pictures.

[0108] As an example, 20 scenic spot pictures are obtained by searching from a scenic spot picture database using multiple similar feature points of N target feature points and feature point indexes, and all 20 scenic spot pictures are directly used as similar scenic spot pictures.

[0109] In a second implementation manner, the number of similar feature points included in the scenic spot pictures corresponding to each of the multiple similar feature points of the N target feature points is likely to be different. The more similar feature points the searched scenic spot pictures include, the more similar they are to the target scenic spot pictures. Based on the multiple scenic spot pictures corresponding to the multiple similar feature points of the searched N target feature points, the scenic spot pictures including the more similar feature points can be screened out. Therefore, in some implementation manners of the embodiments of the present application, the step C may include, for example, the following steps:

[0110] Step C3: searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0111] Step C4: determining the number of similar feature points included in each of the plurality of scenic spot pictures;

[0112] Step C5: screening the plurality of scenic spot pictures to obtain the plurality of similar scenic spot pictures according to the number of similar feature points included in each scenic spot picture.

[0113] As an example, 20 scenic spot pictures are searched from the scenic spot picture database using multiple similar feature points and feature point indexes of N target feature points, the number of similar feature points included in each of the 20 scenic spot pictures is determined, and the top 10 scenic spot pictures ranked from largest to smallest in terms of the number of similar feature points are screened as similar scenic spot pictures.

[0114] Step D: According to the target scenic spot picture, the multiple similar scenic spot pictures and the sampling consistency verification algorithm, the multiple similar scenic spot pictures are filtered to obtain candidate scenic spot pictures.

[0115] It should also be noted that the sampling consistency verification algorithm refers to the use of certain similar feature points in similar scenic spot pictures as initial sample points, combined with the corresponding target feature points in the target scenic spot picture for multiple iterations to verify the substantial similarity between the similar scenic spot picture and the target scenic spot picture. When certain similar feature points in similar scenic spot pictures are randomly selected as initial sample points, the initial sample points may not be selected accurately enough, and many iterations are required, resulting in a more complicated verification process and slower speed and efficiency. Based on the different similarities between the various similar feature points included in the similar scenic spot pictures and the corresponding target feature points, similar feature points with higher corresponding similarities are selected as initial sample points, with fewer iterations. Therefore, in some embodiments of the embodiments of the present application, the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein the preset initial sample points are determined based on the similarities corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is the preset number of iterations.

[0116] As an example, if 20 scenic spot pictures are directly searched from the scenic spot picture database using multiple similar feature points and feature point indexes of N target feature points, all of them are used as similar scenic spot pictures; for each of the 20 similar scenic spot pictures, whether filtering is required is determined based on the target scenic spot picture, similar scenic spot pictures and sampling consistency verification algorithm. If only 10 similar scenic spot pictures remain after filtering the 20 candidate scenic spot pictures, the 10 similar scenic spot pictures are used as candidate scenic spot pictures.

[0117] As an example, if 20 scenic spot pictures are searched from the scenic spot picture database using multiple similar feature points and feature point indexes of N target feature points, 10 similar scenic spot pictures are obtained by screening based on the number of similar feature points included in each of the 20 scenic spot pictures; for each of the 10 similar scenic spot pictures, whether filtering is needed is determined based on the target scenic spot picture, similar scenic spot pictures and the sampling consistency verification algorithm. If only 5 similar scenic spot pictures remain after filtering the 10 candidate scenic spot pictures, the 5 similar scenic spot pictures are used as candidate scenic spot pictures.

[0118] Step 204: Determine a scenic spot recognition result of the target scenic spot picture based on the scenic spot entries corresponding to the candidate scenic spot pictures.

[0119] It can be understood that if the candidate scenic spot pictures obtained in step 203 have a high similarity with the target scenic spot picture, then the scenic spot entry corresponding to the candidate scenic spot picture can represent the scenic spot recognition result of the target scenic spot picture.

[0120] It should be noted that if the candidate scenic spot picture is one candidate scenic spot picture, the scenic spot entry corresponding to the candidate scenic spot picture is directly determined as the scenic spot recognition result of the target scenic spot picture. If the candidate scenic spot picture is multiple candidate scenic spot pictures, first, the scenic spot entries corresponding to the multiple candidate scenic spot pictures are recorded as candidate scenic spot entries. Then, since the correspondence between each candidate scenic spot picture and the candidate scenic spot entry in the multiple candidate scenic spot pictures is known, the candidate scenic spot entry with the largest number of corresponding candidate scenic spot pictures can be used as the scenic spot recognition result of the target scenic spot picture; since the number of similar feature points included in each candidate scenic spot picture in the multiple candidate scenic spot pictures is known, the candidate scenic spot entry corresponding to the candidate scenic spot picture with the largest number of similar feature points can also be used as the scenic spot recognition result of the target scenic spot picture. Of course, the above two methods can also be combined to determine the scenic spot recognition result of the target scenic spot picture. Therefore, in some implementations of the embodiments of the present application, if the candidate scenic spot picture is multiple candidate scenic spot pictures, the step 204 can, for example, include the following steps:

[0121] Step E: obtaining candidate scenic spot entries according to the scenic spot entries corresponding to the plurality of candidate scenic spot images;

[0122] Step F: determining the scenic spot recognition result from the candidate scenic spot entries based on the correspondence between the plurality of candidate scenic spot images and the candidate scenic spot entries and / or the number of similar feature points included in each of the plurality of candidate scenic spot images.

[0123] As an example, step 203 obtains 5 candidate scenic spot pictures, of which 2 candidate scenic spot pictures correspond to scenic spot terms A, and the other 3 candidate scenic spot pictures correspond to scenic spot terms B, C and D, respectively, and the candidate scenic spot terms are A, B, C and D. Since candidate scenic spot term A corresponds to 2 candidate scenic spot pictures, and candidate scenic spot terms B, C and D correspond to only 1 candidate scenic spot picture respectively, candidate scenic spot term A is determined as the scenic spot recognition result of the target scenic spot picture.

[0124] As an example, step 203 obtains 6 candidate scenic spot pictures, among which 2 candidate scenic spot pictures correspond to scenic spot terms A, 2 candidate scenic spot pictures correspond to scenic spot terms B, and 2 candidate scenic spot pictures correspond to scenic spot terms C, and the candidate scenic spot terms are A, B, and C. Since the scenic spot term corresponding to the selected scenic spot picture including the largest number of similar feature points is A, the candidate scenic spot term A is determined as the scenic spot recognition result of the target scenic spot picture.

[0125] In summary, if Figure 3 The target scenic spot picture recognition interface is shown as a schematic diagram, wherein the target scenic spot picture is a scenic spot picture taken by the user himself, the target scenic spot picture is input into the scenic spot local feature model obtained by training the convolutional neural network based on the attention mechanism according to the training sample, and the target scenic spot local features including N target feature points are obtained; for the N target feature points, based on the coincidence relationship between multiple similar feature points corresponding to each target feature point in the feature point index, multiple similar feature points of each target feature point are filtered, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained; candidate scenic spot pictures are obtained from the scenic spot picture database based on multiple similar feature points and feature point index of the N target feature points; based on the scenic spot entry corresponding to the candidate scenic spot picture, Guizhou Huangguoshu Waterfall is determined as the scenic spot recognition result of the target scenic spot picture; and the scenic spot recognition result is displayed on the interface as Guizhou Huangguoshu Waterfall.

[0126] Through the various implementation methods provided in this embodiment, first, the target scenic spot picture is input into the scenic spot local feature model obtained by pre-training the convolutional neural network based on the attention mechanism according to the training samples, and the target scenic spot local features including N target feature points are obtained; secondly, for the N target feature points, multiple similar feature points corresponding to the N target feature points are obtained from the feature point index; then, based on the multiple similar feature points and feature point index of the N target feature points, the candidate scenic spot picture is obtained from the scenic spot picture database; finally, the scenic spot recognition result of the target scenic spot picture is determined based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the local features of the scenic spot can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves.

[0127] See also Figure 4 , shows a flow chart of another method for scenic spot image recognition in an embodiment of the present application. In this embodiment, the method may include the following steps:

[0128] Step 401: obtaining the local features of the target scenic spot according to the target scenic spot picture and the local feature model of the scenic spot. The local feature model of the scenic spot is obtained by training a convolutional neural network based on the attention mechanism according to the training samples. The local features of the target scenic spot include N target feature points.

[0129] Step 402: Calculate the similarity between each target feature point and each feature point in the feature point index.

[0130] Step 403: According to the similarity and the similarity threshold, a plurality of similar feature points corresponding to each target feature point are obtained from the feature point index.

[0131] Step 404: for the N target feature points, multiple similar feature points of each target feature point are filtered based on the coincidence relationship between multiple similar feature points of each target feature point, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained.

[0132] Step 405: searching the scenic spot picture database for multiple scenic spot pictures corresponding to multiple similar feature points of the N target feature points based on the feature point index.

[0133] Step 406: Determine the number of similar feature points included in each of the plurality of scenic spot pictures.

[0134] Step 407: Filter multiple scenic spot pictures to obtain multiple similar scenic spot pictures according to the number of similar feature points included in each scenic spot picture.

[0135] Step 408: According to the target scenic spot picture, the multiple similar scenic spot pictures and the sampling consistency verification algorithm, the multiple similar scenic spot pictures are filtered to obtain candidate scenic spot pictures.

[0136] Among them, the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, the preset initial sample point is determined according to the similarity corresponding to similar feature points included in similar scenic spot pictures; and the number of iterations is a preset number of iterations.

[0137] Step 409: Obtain candidate scenic spot entries according to the scenic spot entries corresponding to the plurality of candidate scenic spot images.

[0138] Step 410: determining a scenic spot recognition result from the candidate scenic spot entries based on the correspondence between the plurality of candidate scenic spot images and the candidate scenic spot entries and / or the number of similar feature points included in each of the plurality of candidate scenic spot images.

[0139] Through the various implementation methods provided in this embodiment, first, the target scenic spot picture is input into the scenic spot local feature model obtained by pre-training the convolutional neural network based on the attention mechanism according to the training samples, and the target scenic spot local features including N target feature points are obtained; secondly, for the N target feature points, multiple similar feature points corresponding to the N target feature points are obtained from the feature point index; then, based on the multiple similar feature points and feature point index of the N target feature points, the candidate scenic spot picture is obtained from the scenic spot picture database; finally, the scenic spot recognition result of the target scenic spot picture is determined based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the local features of the scenic spot can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves.

[0140] Exemplary Devices

[0141] See also Figure 5 , shows a schematic diagram of the structure of a device for scenic spot image recognition in an embodiment of the present application. In this embodiment, the device may specifically include:

[0142] A first obtaining unit 501 is used to obtain a local feature of a target scenic spot according to a target scenic spot picture and a local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local feature of the target scenic spot includes N target feature points;

[0143] A second obtaining unit 502 is used to obtain a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index;

[0144] A third obtaining unit 503 is used to obtain a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index;

[0145] The determining unit 504 is configured to determine a scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture.

[0146] In an optional implementation of the embodiment of the present application, the second obtaining unit 502 includes:

[0147] A first obtaining subunit, configured to obtain a plurality of similar feature points corresponding to each target feature point based on a similarity between each target feature point and each feature point in the feature point index;

[0148] The second obtaining subunit is used to filter the multiple similar feature points of each target feature point based on the coincidence relationship between the multiple similar feature points of each target feature point, and comprehensively obtain multiple similar feature points corresponding to the N target feature points.

[0149] In an optional implementation manner of the embodiment of the present application, the first obtaining subunit includes:

[0150] A first obtaining module, used for calculating the similarity between each target feature point and each feature point in the feature point index;

[0151] The second obtaining module is used to obtain a plurality of similar feature points corresponding to each target feature point from the feature point index according to the similarity and the similarity threshold.

[0152] In an optional implementation of the embodiment of the present application, the third obtaining unit 503 includes:

[0153] A third obtaining subunit is used to search the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index;

[0154] The fourth obtaining subunit is used to filter the multiple similar scenic spot pictures to obtain candidate scenic spot pictures according to the target scenic spot picture, the multiple similar scenic spot pictures and a sampling consistency verification algorithm.

[0155] In an optional implementation of the embodiment of the present application, the third obtaining subunit includes:

[0156] A first search module, configured to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0157] The first determining module is used to directly determine the plurality of scenic spot pictures as the plurality of similar scenic spot pictures.

[0158] In an optional implementation of the embodiment of the present application, the third obtaining subunit includes:

[0159] A second search module is used to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index;

[0160] A second determining module is used to determine the number of similar feature points included in each of the plurality of scenic spot pictures;

[0161] The third obtaining module is used to screen the plurality of scenic spot pictures to obtain the plurality of similar scenic spot pictures according to the number of similar feature points included in each scenic spot picture.

[0162] In an optional implementation of an embodiment of the present application, the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein the preset initial sample point is determined based on the similarity corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is a preset number of iterations.

[0163] In an optional implementation of the embodiment of the present application, if the candidate scenic spot picture is a plurality of candidate scenic spot pictures, the determining unit 504 includes:

[0164] A fifth obtaining subunit, configured to obtain a candidate scenic spot entry according to the scenic spot entries corresponding to the plurality of candidate scenic spot images;

[0165] The determination subunit is used to determine the scenic spot recognition result from the candidate scenic spot entries based on the correspondence between the multiple candidate scenic spot pictures and the candidate scenic spot entries and / or the number of similar feature points included in each candidate scenic spot picture in the multiple candidate scenic spot pictures.

[0166] In an optional implementation of the embodiment of the present application, the feature point index is pre-constructed for the feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures.

[0167] Through the various implementation methods provided in this embodiment, first, the target scenic spot picture is input into the scenic spot local feature model obtained by pre-training the convolutional neural network based on the attention mechanism according to the training samples, and the target scenic spot local features including N target feature points are obtained; secondly, for the N target feature points, multiple similar feature points corresponding to the N target feature points are obtained from the feature point index; then, based on the multiple similar feature points and feature point index of the N target feature points, the candidate scenic spot picture is obtained from the scenic spot picture database; finally, the scenic spot recognition result of the target scenic spot picture is determined based on the scenic spot entry corresponding to the candidate scenic spot picture. It can be seen that the use of the scenic spot local feature model to obtain the local features of the scenic spot can better mine the scenic spot features in the target scenic spot picture, and the combination of the feature point index of the scenic spot picture to obtain similar candidate scenic spot pictures can improve the search speed and accuracy, thereby increasing the efficiency and accuracy of scenic spot recognition, and significantly improving the recognition effect of scenic spot pictures taken by users themselves.

[0168] Figure 66 is a block diagram of a device 600 for scenic spot image recognition according to an exemplary embodiment. For example, the device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0169] Reference Figure 6 , the device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0170] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0171] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0172] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0173] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0174] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0175] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0176] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0177] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0178] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0180] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to execute a method for replying a conversation, the method comprising:

[0181] Obtaining local features of a target scenic spot according to a target scenic spot picture and a local feature model of the scenic spot, wherein the local feature model of the scenic spot is obtained by training a convolutional neural network based on an attention mechanism according to training samples, and the local features of the target scenic spot include N target feature points;

[0182] Obtaining a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index;

[0183] Obtaining a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index;

[0184] Based on the scenic spot entries corresponding to the candidate scenic spot pictures, a scenic spot recognition result of the target scenic spot picture is determined.

[0185] Figure 77 is a schematic diagram of the structure of the server in the embodiment of the present application. The server 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 722 (for example, one or more processors) and a memory 732, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 742 or data 744. Among them, the memory 732 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 722 can be configured to communicate with the storage medium 730 and execute a series of instruction operations in the storage medium 730 on the server 700.

[0186] The server 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 450, one or more input and output interfaces 758, one or more keyboards 756, and / or one or more operating systems 741, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0187] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0188] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0189] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0190] The above is only a preferred embodiment of the present application and does not constitute any formal limitation to the present application. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present application using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still fall within the scope of protection of the technical solution of the present application.

Claims

1. A method for identifying scenic spot pictures, characterized in that: include: Obtaining the local features of the target scenic spot according to the target scenic spot picture and the scenic spot local feature model, wherein the target scenic spot picture includes the scenic spot picture taken by the user, and the target scenic spot picture includes the non-scenic spot area, and the scenic spot local feature model is obtained by training a convolutional neural network based on the attention mechanism according to the training sample, and the target scenic spot local features include N target feature points; Calculating the similarity between each target feature point and each feature point in the feature point index; the feature point index is pre-constructed for the feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures; According to the similarity and the similarity threshold, a plurality of similar feature points corresponding to each target feature point are obtained from the feature point index; For the N target feature points, multiple similar feature points of each target feature point are filtered based on the coincidence relationship between multiple similar feature points of each target feature point, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained; the coincidence relationship means that any two similar feature points corresponding to the same scenic spot picture among the multiple similar feature points of each target feature point overlap; Based on the multiple similar feature points of the N target feature points and the feature point index, searching the scenic spot picture database to obtain multiple similar scenic spot pictures; According to the target scenic spot picture, the multiple similar scenic spot pictures and the sampling consistency verification algorithm, filtering the multiple similar scenic spot pictures to obtain candidate scenic spot pictures; Determining a scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture; If the candidate scenic spot picture is a plurality of candidate scenic spot pictures, determining the scenic spot recognition result of the target scenic spot picture based on the scenic spot entries corresponding to the candidate scenic spot pictures includes: Obtaining candidate scenic spot entries according to the scenic spot entries corresponding to the plurality of candidate scenic spot images; Based on the correspondence between the plurality of candidate scenic spot images and the candidate scenic spot entries, the scenic spot recognition result is determined from the candidate scenic spot entries.

2. The method according to claim 1, characterized in that The step of searching the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index comprises: Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points; The multiple scenic spot pictures are directly determined as the multiple similar scenic spot pictures.

3. The method according to claim 1, characterized in that The step of searching the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index comprises: Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points; Determining the number of similar feature points included in each of the plurality of scenic spot pictures; According to the number of similar feature points included in each of the scenic spot pictures, the multiple scenic spot pictures are screened to obtain the multiple similar scenic spot pictures.

4. The method according to claim 1, characterized in that: The sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein the preset initial sample point is determined based on the similarity corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is a preset number of iterations.

5. A device for identifying scenic spot pictures, characterized in that: include: A first obtaining unit is used to obtain the local features of the target scenic spot according to the target scenic spot picture and the scenic spot local feature model, wherein the target scenic spot picture includes a scenic spot picture taken by a user, and the target scenic spot picture includes a non-scenic spot area, and the scenic spot local feature model is obtained by training a convolutional neural network based on an attention mechanism according to a training sample, and the target scenic spot local feature includes N target feature points; A second obtaining unit is used to obtain a plurality of similar feature points corresponding to the N target feature points according to the N target feature points and the feature point index; the feature point index is pre-constructed for the feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures; A third obtaining unit, configured to obtain a candidate scenic spot picture from a scenic spot picture database based on a plurality of similar feature points of the N target feature points and the feature point index; A determination unit, configured to determine a scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture; The second obtaining unit comprises: A first obtaining subunit, configured to obtain a plurality of similar feature points corresponding to each target feature point based on a similarity between each target feature point and each feature point in the feature point index; A second obtaining subunit is used for filtering the multiple similar feature points of each target feature point based on the coincidence relationship between the multiple similar feature points of each target feature point, and comprehensively obtaining multiple similar feature points corresponding to the N target feature points; the coincidence relationship means that any two similar feature points corresponding to the same scenic spot picture among the multiple similar feature points of each target feature point coincide with each other; The first obtaining subunit comprises: A first obtaining module, used for calculating the similarity between each target feature point and each feature point in the feature point index; A second obtaining module, configured to obtain, from the feature point index, a plurality of similar feature points corresponding to each target feature point according to the similarity and the similarity threshold; The third obtaining unit comprises: A third obtaining subunit is used to search the scenic spot picture database to obtain a plurality of similar scenic spot pictures based on the plurality of similar feature points of the N target feature points and the feature point index; A fourth obtaining subunit is used to filter the multiple similar scenic spot pictures to obtain candidate scenic spot pictures according to the target scenic spot picture, the multiple similar scenic spot pictures and a sampling consistency verification algorithm; If the candidate scenic spot picture is a plurality of candidate scenic spot pictures, the determining unit includes: A fifth obtaining subunit, configured to obtain a candidate scenic spot entry according to the scenic spot entries corresponding to the plurality of candidate scenic spot images; The determination subunit is used to determine the scenic spot recognition result from the candidate scenic spot entries based on the correspondence between the plurality of candidate scenic spot images and the candidate scenic spot entries.

6. The device according to claim 5, characterized in that The third obtaining subunit comprises: A first search module, configured to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index; The first determining module is used to directly determine the plurality of scenic spot pictures as the plurality of similar scenic spot pictures.

7. The device according to claim 5, characterized in that The third obtaining subunit comprises: A second search module is used to search the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points based on the feature point index; A second determining module is used to determine the number of similar feature points included in each of the plurality of scenic spot pictures; The third obtaining module is used to screen the plurality of scenic spot pictures to obtain the plurality of similar scenic spot pictures according to the number of similar feature points included in each scenic spot picture.

8. The device according to claim 5, characterized in that The sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein the preset initial sample point is determined based on the similarity corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is a preset number of iterations.

9. A device for identifying scenic spot pictures, characterized in that: The system comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: Obtaining the local features of the target scenic spot according to the target scenic spot picture and the scenic spot local feature model, wherein the target scenic spot picture includes the scenic spot picture taken by the user, and the target scenic spot picture includes the non-scenic spot area, and the scenic spot local feature model is obtained by training a convolutional neural network based on the attention mechanism according to the training sample, and the target scenic spot local features include N target feature points; Calculating the similarity between each target feature point and each feature point in the feature point index; the feature point index is pre-constructed for the feature points included in the local features of the scenic spot pictures in the scenic spot picture database and the corresponding relationship between the feature points and the scenic spot pictures; According to the similarity and the similarity threshold, a plurality of similar feature points corresponding to each target feature point are obtained from the feature point index; For the N target feature points, multiple similar feature points of each target feature point are filtered based on the coincidence relationship between multiple similar feature points of each target feature point, and multiple similar feature points corresponding to the N target feature points are comprehensively obtained; the coincidence relationship means that any two similar feature points corresponding to the same scenic spot picture among the multiple similar feature points of each target feature point overlap; Based on the multiple similar feature points of the N target feature points and the feature point index, searching the scenic spot picture database to obtain multiple similar scenic spot pictures; According to the target scenic spot picture, the multiple similar scenic spot pictures and the sampling consistency verification algorithm, filtering the multiple similar scenic spot pictures to obtain candidate scenic spot pictures; Determining a scenic spot recognition result of the target scenic spot picture based on the scenic spot entry corresponding to the candidate scenic spot picture; If the candidate scenic spot picture is a plurality of candidate scenic spot pictures, determining the scenic spot recognition result of the target scenic spot picture based on the scenic spot entries corresponding to the candidate scenic spot pictures includes: Obtaining candidate scenic spot entries according to the scenic spot entries corresponding to the plurality of candidate scenic spot images; Based on the correspondence between the plurality of candidate scenic spot images and the candidate scenic spot entries, the scenic spot recognition result is determined from the candidate scenic spot entries.

10. The device according to claim 9, wherein the step of searching the scenic spot image database to obtain a plurality of similar scenic spot images based on the plurality of similar feature points of the N target feature points and the feature point index comprises: Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points; The multiple scenic spot pictures are directly determined as the multiple similar scenic spot pictures.

11. The device according to claim 9, wherein the step of searching the scenic spot image database to obtain a plurality of similar scenic spot images based on the plurality of similar feature points of the N target feature points and the feature point index comprises: Based on the feature point index, searching the scenic spot picture database for a plurality of scenic spot pictures corresponding to a plurality of similar feature points of the N target feature points; Determining the number of similar feature points included in each of the plurality of scenic spot pictures; According to the number of similar feature points included in each of the scenic spot pictures, the multiple scenic spot pictures are screened to obtain the multiple similar scenic spot pictures.

12. The device according to claim 9, wherein the sampling consistency verification algorithm is a preset sampling consistency verification algorithm, wherein: The preset initial sample points are determined according to the similarities corresponding to the similar feature points included in the similar scenic spot pictures; and the number of iterations is the preset number of iterations.

13. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes a device to execute the method for scenic spot image recognition according to one or more of claims 1 to 4.