A method and device for generating landmark images

Abstract

The embodiment of the present invention provides a method and device for generating a landmark image, which relates to the field of data processing technology. The method includes: obtaining landmark images of a target landmark; classifying the landmark images according to the image acquisition perspective; selecting a category in which the number of landmark images in each category is less than the number of preset images as the target category corresponding to the isolated perspective; performing image augmentation processing based on the landmark images included in the target category to generate a new landmark image of the target landmark. Applying the solution provided by the embodiment of the present invention, an image of a landmark can be generated.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for generating a landmark image. Background Art

[0002] When users browse pictures or watch videos, places that users are not aware of may appear in the content of the picture. In order to help users know the places in the content of the picture, landmark recognition can be performed based on the image, so that users can quickly understand the places that appear in the content of the picture.

[0003] In the prior art, landmark images of various landmarks are usually collected in advance, which are called existing images. In this way, when performing landmark recognition based on images, candidate landmarks corresponding to existing images that match the image to be recognized can be first determined, and then the landmark corresponding to the image to be recognized can be determined from the candidate landmarks according to the number of matching images corresponding to each candidate landmark, thereby completing landmark recognition. The greater the number of matching images corresponding to each candidate landmark, the greater the possibility that the landmark is the landmark corresponding to the image to be recognized.

[0004] However, it is difficult to collect landmark images of some landmarks, and the number of collected landmark images is small. Therefore, when performing image matching, the number of matching images corresponding to these landmarks will also be small, which may cause landmark recognition errors and lead to low accuracy of landmark recognition. Summary of the invention

[0005] The purpose of the embodiment of the present invention is to provide a method and device for generating a landmark image, so as to generate an image of a landmark. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for generating a landmark image, the method comprising:

[0007] Obtaining landmark images of target landmarks;

[0008] Classify landmark images according to the image acquisition perspective;

[0009] Select the category in which the number of landmark images in each category is less than the preset number of images as the target category corresponding to the isolated viewpoint;

[0010] Image augmentation processing is performed based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0011] In one embodiment of the present invention, the classifying of landmark images according to image acquisition viewing angles includes:

[0012] Obtaining global image features of each landmark image;

[0013] Clustering the obtained global image features to obtain feature clusters;

[0014] The landmark images corresponding to the global image features belonging to the same feature cluster are divided into a category corresponding to an image acquisition perspective.

[0015] In one embodiment of the present invention, the image augmentation processing is performed based on the landmark image included in the target classification to generate a new landmark image of the target landmark, including:

[0016] Determining the target number of images to be generated according to the number of landmark images included in the target classification and the number of preset images;

[0017] Image augmentation processing is performed based on the landmark images included in the target classification to generate new landmark images of the target landmarks of the target number.

[0018] In one embodiment of the present invention, the image augmentation processing is performed based on the landmark image included in the target classification to generate a new landmark image of the target landmark, including:

[0019] Performing target processing on the landmark images included in the target classification, and generating a new landmark image of the target landmark based on the processed image, wherein the target processing includes at least one of random cropping processing, random affine processing and random color parameter adjustment.

[0020] In a second aspect, an embodiment of the present invention provides a landmark image generating device, the device comprising:

[0021] An image acquisition module, used to obtain images of each landmark of the target landmark;

[0022] An image classification module, used to classify landmark images according to the image acquisition perspective;

[0023] A target classification determination module, used to select a classification in which the number of landmark images in each classification is less than a preset number of images as a target classification corresponding to an isolated perspective;

[0024] The image generation module is used to perform image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0025] In one embodiment of the present invention, the image classification module includes:

[0026] An image feature acquisition submodule is used to obtain the global image features of each landmark image;

[0027] A feature clustering submodule is used to cluster the obtained global image features to obtain feature clusters;

[0028] The image classification submodule is used to classify the landmark images corresponding to the global image features belonging to the same feature cluster into a category corresponding to an image acquisition perspective.

[0029] In one embodiment of the present invention, the image generation module is specifically used to determine the target number of images to be generated based on the number of landmark images included in the target classification and the number of preset images; perform image augmentation processing based on the landmark images included in the target classification to generate new landmark images of the target number of target landmarks.

[0030] In one embodiment of the present invention, the image generation module is specifically used to perform target processing on the landmark images included in the target classification, and generate a new landmark image of the target landmark based on the processed image, wherein the target processing includes: at least one of random cropping processing, random affine processing and random color parameter adjustment.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0032] Memory, used to store computer programs;

[0033] The processor is used to implement the landmark image generation method described in the first aspect when executing the program stored in the memory.

[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the landmark image generation method described in the first aspect is implemented.

[0035] In a fifth aspect, an embodiment of the present invention provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the landmark image generation method described in the first aspect.

[0036] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to generate landmark images, the landmark images of the target landmark are divided into categories according to the image acquisition perspective, and the target category corresponding to the isolated perspective is determined based on the number of landmark images in each category. In this way, image augmentation processing is performed based on the landmark images included in the above target category, so that a new landmark image of the target landmark can be generated, which effectively increases the number of images of the target landmark.

[0037] In addition, it can be seen that when the solution provided by the embodiment of the present invention is applied to generate landmark images, it is not to directly perform image augmentation processing based on the landmark images of the target landmark to generate new landmark images, but to perform image augmentation processing only on the landmark images included in the target classification to generate new landmark images, so that the newly generated landmark images belong to the target classification, and the target classification corresponds to the isolated perspective, which increases the number of landmark images belonging to the isolated perspective. Since the number of images belonging to other image acquisition perspectives remains unchanged, the number of landmark images belonging to each image acquisition perspective is relatively balanced. On the basis of the above, after adding the landmark images of the isolated perspective, when performing landmark recognition on the image to be recognized, even if the image to be recognized is actually an image belonging to the above isolated perspective, there will be no situation where the landmark recognition to which the above image to be recognized belongs is wrong due to the small number of landmark images belonging to the isolated perspective, thereby improving the accuracy of landmark recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0039] Figure 1 A schematic diagram of a flow chart of a first landmark image generation method provided by an embodiment of the present invention;

[0040] Figure 2 A schematic flow chart of a second landmark image generation method provided by an embodiment of the present invention;

[0041] Figure 3 A schematic structural diagram of a first landmark image generating device provided by an embodiment of the present invention;

[0042] Figure 4 A schematic structural diagram of a first landmark image generating device provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Since it is difficult to collect landmark images of some landmarks, which may result in a small number of landmark images being collected, the embodiments of the present invention provide a landmark image generation method, device, equipment and storage medium to generate landmark images, which are introduced below.

[0046] In one embodiment of the present invention, a method for generating a landmark image is provided, the method comprising:

[0047] Obtaining landmark images of target landmarks;

[0048] Classify landmark images according to the image acquisition perspective;

[0049] Select the category in which the number of landmark images in each category is less than the preset number of images as the target category corresponding to the isolated viewpoint;

[0050] Image augmentation processing is performed based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0051] It can be seen from the above that when the solution provided by the embodiment of the present invention is used to generate an image, the landmark images of the target landmark are divided into categories according to the image acquisition perspective, and the target category corresponding to the isolated perspective is determined based on the number of landmark images in each category. In this way, image augmentation processing is performed based on the landmark images included in the above target category, so that a new landmark image of the target landmark can be generated, which effectively increases the number of images of the target landmark.

[0052] In addition, it can be seen that when the solution provided by the embodiment of the present invention is applied to generate landmark images, it is not to directly perform image augmentation processing based on the landmark images of the target landmark to generate new landmark images, but to perform image augmentation processing only on the landmark images included in the target classification to generate new landmark images, so that the newly generated landmark images belong to the target classification, and the target classification corresponds to the isolated perspective, which increases the number of landmark images belonging to the isolated perspective. Since the number of images belonging to other image acquisition perspectives remains unchanged, the number of landmark images belonging to each image acquisition perspective is relatively balanced. On the basis of the above, after adding the landmark images of the isolated perspective, when performing landmark recognition on the image to be recognized, even if the image to be recognized is actually an image belonging to the above isolated perspective, there will be no situation where the landmark recognition to which the above image to be recognized belongs is wrong due to the small number of landmark images belonging to the isolated perspective, thereby improving the accuracy of landmark recognition.

[0053] The landmark image generation method provided by the embodiment of the present invention is described in detail below through specific embodiments.

[0054] See also Figure 1 , Figure 1This is a flow chart of a first landmark image generation method provided by an embodiment of the present invention. The method includes the following steps S101-S104.

[0055] Step S101: Obtaining landmark images of target landmarks.

[0056] The target landmarks mentioned above may be iconic buildings in a city, or any areas or places, such as natural landscapes, artificial buildings, historical sites, etc.

[0057] Specifically, each landmark image of the target landmark can be pre-collected and stored in a pre-established image database, so that each landmark image of the target landmark can be read from the image database. The landmark images pre-stored in the image database can be obtained by staff using image acquisition equipment, such as a camera, a video camera, etc., to take pictures at the target landmark, or by using a web crawler engine to extract images from a web page containing target landmark information. The web page containing target landmark information can be a tourism promotion web page of the target landmark, etc.

[0058] Step S102: classifying landmark images according to image acquisition viewing angles.

[0059] The above-mentioned image acquisition angle of view is the lens angle of view of the image acquisition device that acquires the landmark image. It is understandable that different image acquisition angles can cover different physical areas, and thus the landmark images acquired by the image acquisition device are also different.

[0060] The physical area covered by the above image acquisition perspective may include the entire physical area occupied by the target landmark, or may include some representative physical areas of the target landmark. For example, if the target landmark is an ancient building complex A, the above image acquisition perspective may cover all the occupied areas of the building complex, or may only cover the area where the gate a in the building complex is located, the area where the palace b is located, etc.; if the target landmark is a natural scenic area B, the above image acquisition perspective may cover the area where the river c in the natural scenic area is located, the area where the lake d is located, etc.

[0061] Since there are multiple image acquisition perspectives for the target landmark, the landmark images acquired at the same image acquisition perspective often have small differences, while the landmark images acquired at different image acquisition perspectives often have large differences. Therefore, in an embodiment of the present invention, landmark images belonging to the same image acquisition perspective are divided into the same category based on the image acquisition perspective to which the landmark images belong.

[0062] Specifically, the landmark images may be classified according to the image acquisition perspective in the following manner.

[0063] In one implementation, the shooting location of the landmark image can be obtained, and the preset area to which the shooting location belongs can be determined as the area covered by the image acquisition perspective, and then the landmark images with the same area covered by the image acquisition perspective can be determined as images belonging to the same image acquisition perspective, and then the images belonging to the same image acquisition perspective can be classified into the same category. For example, if the coordinates of the shooting locations of landmark image P and landmark image Q are (x1, y1) and (x2, y2), respectively, and the above coordinates belong to the area where the preset palace b is located, then the area where the palace b is located can be determined as the area covered by the image acquisition perspective of landmark image P and landmark image Q, and then landmark image P and landmark image Q can be determined as images belonging to the same image acquisition perspective, and landmark image P and landmark image Q can be classified into the same category.

[0064] The following two methods are used as examples to introduce how to obtain the shooting location of the above landmark image.

[0065] The first method is to read the shooting location recorded in the image file of the above-mentioned landmark image. The attribute information of the image file of the landmark image captured by the image acquisition device contains various information of the landmark image, such as image resolution, image size, shooting time, etc. In some cases, when an image acquisition device with a satellite positioning function is used to capture images, the attribute information of the image file of the captured landmark image also records the location of the above-mentioned device when the landmark image was created as the shooting location of the landmark image. In this case, the shooting location in the attribute information of the image file of the landmark image can be read.

[0066] In the second method, when the staff uses the image acquisition device to acquire the landmark image, they can manually record and store the shooting location of the landmark image, so that the electronic device can read the shooting location information to obtain the shooting location of each landmark image.

[0067] In another implementation, the global image features of each landmark image can be first obtained, the obtained global image features can be clustered to obtain feature clusters, and the landmark images corresponding to the global image features belonging to the same feature cluster can be divided into a category corresponding to an image acquisition perspective. For details of the implementation, see Figure 2 Steps S202-S204 in the illustrated embodiment are not described in detail here.

[0068] Step S103: selecting a category in which the number of landmark images in each category is less than the preset number of images as the target category corresponding to the isolated perspective.

[0069] The above-mentioned isolated perspectives are image acquisition perspectives whose corresponding number of landmark images is significantly different from that of other image acquisition perspectives. When pre-collecting images of target landmarks, some image acquisition perspectives belonging to the target landmarks may result in fewer landmark images belonging to the above image acquisition perspectives compared to other image acquisition perspectives due to the greater difficulty in image collection and / or the lower visibility of the areas covered by the image acquisition perspectives. For example, the target landmark C is a mountain range, and the area covered by image acquisition perspective c is a cliff area of ​​the landmark. Since the landmark images belonging to image acquisition perspective c are more difficult to collect, there is only one existing landmark image belonging to image acquisition perspective c, while the number of images from other image acquisition perspectives is 5. It can be seen that the number of landmark images belonging to the above image acquisition perspective c is significantly different from that of other image acquisition perspectives, and image acquisition perspective c can be determined as an isolated perspective.

[0070] Since each classification obtained by the division corresponds to each image acquisition perspective, the number of landmark images in each classification is the number of landmark images belonging to each image acquisition perspective. In this way, according to the number of landmark images included in each image acquisition perspective, the number of corresponding landmark images of the isolated perspective with a large difference from other image acquisition perspectives can be determined, and then the target classification corresponding to the isolated perspective can be determined.

[0071] Specifically, the number of preset images can be set by the staff according to the number of landmark images included in each category of the target landmark. For example, the landmark images of the target landmark C are divided into 10 categories, of which 8 categories include 5 landmark images, and 2 categories include 2 and 3 landmark images respectively. In this case, the number of preset images can be 4, so that the categories including 2 and 3 landmark images respectively will be determined as the target categories corresponding to the isolated perspective. In this way, the categories with less landmark images than the preset number of images are used as the target categories corresponding to the isolated perspective, so that the target categories corresponding to the isolated perspective can be accurately determined.

[0072] In one embodiment of the present invention, this step can also be implemented in the following manner:

[0073] First, the average number of landmark images in each category is calculated, and the category whose number of landmark images is less than the above average is selected as the target category corresponding to the isolated view.

[0074] In this way, the category whose number of landmark images is less than the average number of landmark images in each category can be used as the target category corresponding to the isolated perspective. After the landmark images belonging to the isolated perspective are subsequently augmented to generate new landmark images, the number of images belonging to the isolated perspective can be made close to the above average value, so that the number of images included in each image acquisition perspective of the target landmark is relatively balanced. Similarly, the median and mode of the number of landmark images in each category can be calculated, and the category whose number of landmark images in each category is less than the above value can be selected as the target category corresponding to the isolated perspective.

[0075] Step S104: performing image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0076] It can be seen from the aforementioned step S103 that, since the target classification corresponding to the isolated perspective has been determined, the landmark image included in the target classification can be augmented to generate a new landmark image of the target landmark.

[0077] Specifically, a new landmark image of the target landmark may be generated in the following manner.

[0078] In a first implementation manner, the landmark images included in the target classification may be randomly cropped, and the processed images may be used as new landmark images of the target landmarks.

[0079] For example, a cropping size smaller than the original size of the landmark image included in the target classification is randomly determined, and an image having a size of the cropping size is cropped from the landmark image to obtain a new landmark image.

[0080] In a second implementation manner, random affine processing may be performed on the landmark images included in the target classification, and the processed images are used as new landmark images of the target landmarks.

[0081] The random affine processing of the landmark images included in the target classification includes random scaling, translation, rotation, reflection or shearing of the landmark images.

[0082] In a third implementation manner, the color parameters of the landmark images included in the target classification may be randomly adjusted, and the adjusted images may be used as new landmark images of the target landmarks.

[0083] The color parameters may include brightness, contrast, hue, saturation, etc. A new image may be generated by randomly adjusting the color parameters of the landmark image. For example, a floating value may be pre-set, and within the range of the floating value, the original color parameters of the landmark image may be randomly adjusted, and the adjusted image may be used as a new landmark image of the target landmark.

[0084] Of course, the method of generating a new landmark image of the target landmark can also be the superposition of at least two of the above three implementations, and the embodiment of the present invention does not limit the number and order of superposition of the above implementations. For example, after performing random affine processing on the landmark images included in the target classification, the image obtained after the above processing can also be subjected to random color parameter transformation, and the processed image can be used as the new landmark image of the target landmark.

[0085] By performing at least one of a plurality of target processing on the landmark images included in the target classification, new landmark images with more diverse target landmarks can be generated based on the processed images.

[0086] In one embodiment of the present invention, a new landmark image of the target landmark may be generated through the following steps A and B.

[0087] Step A: Determine the target number of images to be generated according to the number of landmark images included in the target classification and the aforementioned preset number of images.

[0088] Specifically, the difference between the number of preset images and the number of landmark images included in the target classification can be calculated, and the difference can be determined as the target number of images to be generated. For example, if the number of preset images is 3 and the number of landmark images included in the target classification is 1, the difference between the number of preset images and the number of landmark images included in the target classification is 3-1=2, that is, the target number of images to be generated is 2. In this way, after the new landmark image is generated, the number of landmark images included in the classification corresponding to the isolated perspective reaches the number of preset images, so that the number of landmark images included in each classification is more balanced.

[0089] Step B: Perform image augmentation processing based on the landmark images included in the target classification to generate new landmark images of the target number of target landmarks.

[0090] The above step B can refer to the image augmentation method mentioned in the above embodiment, the only difference is that the number of new landmark images generated is the target number, which will not be repeated here. In this way, for the target classification corresponding to the isolated perspective, after image augmentation processing is performed on the landmark images included in the above target classification, new landmark images of the target number of target landmarks can be obtained, which increases the number of landmark images included in the target classification corresponding to the isolated perspective, making the number of images belonging to each image acquisition perspective more balanced.

[0091] In another embodiment of the present invention, the product of the average value of the number of landmark images in the above-mentioned categories and a preset coefficient can be calculated, and the difference between the above-mentioned product and the number of landmark images included in the target category can be determined as the target number of images to be generated, and image augmentation processing is performed based on the landmark images included in the target category to generate new landmark images of the target number of target landmarks.

[0092] For example, when the preset coefficient is 1, if the average value of the number of landmark images in each category is 5, and the number of landmark images included in the target category is 2, then the target number of images to be generated is 5-2=3. In this way, after the new landmark image is generated, the number of landmark images included in the category corresponding to the isolated perspective reaches the average value, so that the number of landmark images included in each category is more balanced; when the preset coefficient is 2, if the average value of the number of landmark images in each category is 5, and the number of landmark images included in the target category is 2, then the target number of images to be generated is 5*2-2=8. Since the preset number of images is the average value, twice the preset number is the maximum value of the number of landmark images in each category. In this way, the difference is determined as the target number of images to be generated, so that after the new landmark image is generated, the number of landmark images included in the category corresponding to the isolated perspective reaches the maximum value, so that the number of landmark images included in each category is more balanced.

[0093] It can be seen from the above that through the above steps S101-S104, image augmentation processing is performed on the landmark images included in the target classification corresponding to the isolated perspective, and new images of the target landmarks are generated, thereby increasing the number of images of the target landmarks.

[0094] The following describes the effect of using the solution provided by the embodiment of the present invention to generate landmark images and expand the map image on the accuracy of landmark recognition.

[0095] It is understandable that for a certain landmark, the image to be identified at a certain image acquisition angle of the landmark has a high similarity with the existing image belonging to the image acquisition angle, and they often match each other. In this way, if there are fewer existing images belonging to a certain image acquisition angle of the landmark, then when performing landmark recognition on the image to be identified at the image acquisition angle of the landmark, the number of matching images corresponding to the landmark is small, which can easily lead to errors in landmark recognition of the image to be identified.

[0096] The following example specifically illustrates the effect of the number of images in a certain image acquisition perspective on the accuracy of landmark recognition:

[0097] When performing landmark recognition on an image to be recognized, the landmark to which the image to be recognized belongs can be determined based on the confidence score of each landmark for the image to be recognized, and the landmark with the highest confidence score is used as the landmark to which the image to be recognized belongs. The confidence score is (the number of matching images of the landmark / the total number of existing images of the landmark+1)*the average value of the similarity of the matching images.

[0098] For example, the following takes landmark recognition of an image S to be recognized that actually belongs to the image acquisition viewing angle a of the landmark A as an example for explanation.

[0099] If there are 20 existing images of landmark A, among which image acquisition angle a includes 1 existing image, and the similarity between the existing image and the image to be identified S is 80%; landmark B also has 20 existing images, among which image acquisition angle b includes 10 existing images, and the similarity between these existing images and the image to be identified S is 60%. Then when performing landmark identification, if the landmark image with a similarity greater than 50% with the image to be identified is a matching image, then the number of matching images between landmark A and the image to be identified is 1, and the confidence score of landmark A for the image to be identified is (1 / 20+1)*80%=0.84; the number of matching images between landmark B and the image to be identified is 6, and the confidence score of landmark B for the landmark to which the image to be identified belongs is (10 / 20+1)*60%=0.9. It can be seen that the above confidence score of landmark B is higher than that of landmark A, so the image to be identified will be determined to belong to landmark B, and from the previous description, it can be seen that the image to be identified S actually belongs to landmark A, which shows that there is an error in landmark identification.

[0100] Analysis of the above process shows that the existing images included in the image acquisition perspective a of landmark A have a higher similarity with image S. However, since the number of existing images included in the image acquisition perspective a is too small, the above confidence score of landmark A is smaller than that of landmark B, which in turn leads to errors in the above landmark recognition of image S to be identified.

[0101] When the solution provided by the embodiment of the present invention is applied to generate landmark images, since the number of existing images corresponding to the above-mentioned image acquisition perspective a is 1, the above-mentioned image acquisition perspective a can be determined as an isolated perspective by setting the above-mentioned preset number of images, etc. Therefore, image augmentation processing will be performed based on the landmark image corresponding to the above-mentioned image acquisition perspective a. For example, 4 new landmark images are generated based on the existing landmark images of the image acquisition perspective a. After the above-mentioned image augmentation processing, the landmark image corresponding to the above-mentioned isolated perspective a is expanded. When landmark recognition is performed based on the expanded landmark image, the number of matching images between landmark A and the image to be recognized is 5, and the confidence score of landmark A for the image to be recognized is (5 / 20+1)*80%=1, which is greater than the confidence score of landmark B of 0.9. In this way, the above-mentioned image to be recognized will be determined to belong to landmark A. From the previous description, it can be seen that the image to be recognized S actually belongs to landmark A, which shows that the landmark recognition is correct.

[0102] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to generate landmark images, the landmark images of the target landmark are divided into categories according to the image acquisition perspective, and the target category corresponding to the isolated perspective is determined based on the number of landmark images in each category. In this way, image augmentation processing is performed based on the landmark images included in the above target category, so that a new landmark image of the target landmark can be generated, which effectively increases the number of images of the target landmark.

[0103] In addition, it can be seen that when the solution provided by the embodiment of the present invention is applied to generate landmark images, it is not to directly perform image augmentation processing based on the landmark images of the target landmark to generate new landmark images, but to perform image augmentation processing only on the landmark images included in the target classification to generate new landmark images, so that the newly generated landmark images belong to the target classification, and the target classification corresponds to the isolated perspective, which increases the number of landmark images belonging to the isolated perspective. Since the number of images belonging to other image acquisition perspectives remains unchanged, the number of landmark images belonging to each image acquisition perspective is relatively balanced. On the basis of the above, after adding the landmark images of the isolated perspective, when performing landmark recognition on the image to be recognized, even if the image to be recognized is actually an image belonging to the above isolated perspective, there will be no situation where the landmark recognition to which the above image to be recognized belongs is wrong due to the small number of landmark images belonging to the isolated perspective, thereby improving the accuracy of landmark recognition.

[0104] Since image classification can be achieved in a variety of ways during the process of generating new landmark images, the landmark image generation process is described below in conjunction with one of the image classification methods.

[0105] See also Figure 2 , Figure 2 This is a flow chart of a second landmark image generation method provided by an embodiment of the present invention. The method includes the following steps S201-S206.

[0106] Step S201: Obtain each landmark image of the target landmark.

[0107] The above step S201 is the same as the above Figure 1 Step S101 in the illustrated embodiment is the same and will not be described again here.

[0108] Step S202: Obtaining global image features of each landmark image.

[0109] Since the global image features of landmark images belonging to the same image acquisition perspective are often highly similar, each landmark image can be divided into categories corresponding to different image acquisition perspectives based on the global image features of each landmark image of the target landmark.

[0110] Specifically, the global image features of each landmark image can be obtained in the following manner.

[0111] In the first embodiment, the global image features extracted by the pre-trained landmark recognition model can be obtained. When the landmark recognition model performs landmark recognition on a landmark image, it first extracts the global image features of the landmark image through the feature extraction layer, and then uses the global image features as the input features of the fully connected layer, and finally the fully connected layer outputs the classification result of the landmark image. As can be seen from the above, all the global image features extracted by the landmark recognition model in the feature extraction stage can be obtained, and then the global image features corresponding to the landmark image can be determined as the global image features of the landmark image.

[0112] In a second implementation manner, an image feature database storing global image features of existing landmark images of various landmarks may be pre-established. In this case, the global image features may be read from the pre-established image feature database.

[0113] In a third implementation, the global image features may be extracted based on feature extraction algorithms such as an edge extraction operator, a texture feature extraction algorithm, and a convolutional neural network algorithm.

[0114] Step S203: clustering the obtained global image features to obtain feature clusters.

[0115] Specifically, the following uses three clustering methods as examples to illustrate how to obtain feature clusters.

[0116] In the first implementation, the obtained global image features can be clustered using a hierarchical clustering algorithm, such as the BIRCH (Balanced Iterative Reducing and Clustering Using Hierarchies) algorithm, the CURE (Clustering Using Representative) algorithm, or the Chameleon algorithm and other hierarchical clustering algorithms, and the above-mentioned feature clusters are generated after clustering is completed.

[0117] In a second implementation manner, a K-Means algorithm may be used to cluster the obtained global image features, and the above-mentioned feature clusters may be generated after clustering is completed.

[0118] In a third implementation, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) can be used to cluster the obtained global image features, and the values ​​of ε-neighborhood and min_samples (minimum number of sample points) are pre-set. After clustering is completed, the above-mentioned feature clusters are generated.

[0119] Step S204: classify the landmark images corresponding to the global image features belonging to the same feature cluster into a category corresponding to an image acquisition perspective.

[0120] Since the global image features included in the same feature cluster are similar features, the landmark images corresponding to the above-mentioned global image features are also similar landmark images, and similar landmark images often belong to the same image acquisition perspective. Therefore, it can be considered that the feature cluster corresponds to the image acquisition perspective, and then the landmark images corresponding to the global image features belonging to the same feature cluster can be divided into a category corresponding to an image acquisition perspective.

[0121] Step S205: selecting a category in which the number of landmark images in each category is less than the preset number of images as the target category corresponding to the isolated perspective.

[0122] Step S206: performing image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0123] The above steps S205 and S206 are the same as the above steps. Figure 1 In the illustrated embodiment, step S103 is the same as step S104 and will not be described in detail herein.

[0124] As can be seen from the above, since the global image features of landmark images belonging to the same image acquisition perspective are often highly similar, the global image features of each landmark image can be clustered. The result of clustering is that the global image features with high similarity are generated into a feature cluster. At this time, the landmark images corresponding to the global image features under the feature cluster can be understood as images belonging to the same image acquisition perspective. In this way, the landmark images corresponding to the global image features belonging to the same feature cluster are divided into a category corresponding to an image acquisition perspective, and each landmark image can be accurately classified according to the image acquisition perspective.

[0125] In one embodiment of the present invention, after generating a new landmark image of the target landmark, the global image features of the new landmark image can also be obtained, and then the landmark image of the target landmark and the global image features of the new landmark image are stored in an image feature database, so that the existing landmark recognition scheme can directly read the image features of the existing landmark image from the image feature database when identifying the landmark by feature comparison, which is convenient for the execution of the landmark recognition scheme. The specific method for obtaining the global image features of the new landmark image can refer to the method for obtaining the global image features mentioned in the aforementioned step S202, which will not be repeated here.

[0126] Corresponding to the above-mentioned landmark image generating method, an embodiment of the present invention further provides a landmark image generating device.

[0127] See also Figure 3 , Figure 3 This is a schematic structural diagram of a first landmark image generating device provided by an embodiment of the present invention. The device includes the following modules 301-304.

[0128] An image acquisition module 301 is used to obtain each landmark image of the target landmark;

[0129] An image classification module 302, for classifying landmark images according to image acquisition viewing angles;

[0130] The target classification determination module 303 is used to select a classification in which the number of landmark images in each classification is less than a preset number of images as the target classification corresponding to the isolated perspective;

[0131] The image generation module 304 is used to perform image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0132] It can be seen from the above that when the solution provided by the embodiment of the present invention is applied to generate landmark images, the landmark images of the target landmark are divided into categories according to the image acquisition perspective, and the target category corresponding to the isolated perspective is determined based on the number of landmark images in each category. In this way, image augmentation processing is performed based on the landmark images included in the above target category, so that a new landmark image of the target landmark can be generated, which effectively increases the number of images of the target landmark.

[0133] In addition, it can be seen that when the solution provided by the embodiment of the present invention is applied to generate landmark images, it is not to directly perform image augmentation processing based on the landmark images of the target landmark to generate new landmark images, but to perform image augmentation processing only on the landmark images included in the target classification to generate new landmark images, so that the newly generated landmark images belong to the target classification, and the target classification corresponds to the isolated perspective, which increases the number of landmark images belonging to the isolated perspective. Since the number of images belonging to other image acquisition perspectives remains unchanged, the number of landmark images belonging to each image acquisition perspective is relatively balanced. On the basis of the above, after adding the landmark images of the isolated perspective, when performing landmark recognition on the image to be recognized, even if the image to be recognized is actually an image belonging to the above isolated perspective, there will be no situation where the landmark recognition to which the above image to be recognized belongs is wrong due to the small number of landmark images belonging to the isolated perspective, thereby improving the accuracy of landmark recognition.

[0134] In one embodiment of the present invention, the image generation module 304 is specifically used to determine the target number of images to be generated based on the number of landmark images included in the target classification and the number of preset images; perform image augmentation processing based on the landmark images included in the target classification to generate new landmark images of the target number of target landmarks.

[0135] In this way, for the target classification corresponding to the isolated perspective, after image augmentation processing is performed on the landmark images included in the above target classification, new landmark images of the target number of target landmarks can be obtained, which increases the number of landmark images included in the target classification corresponding to the isolated perspective, making the number of images belonging to each image acquisition perspective more balanced.

[0136] In one embodiment of the present invention, the image generation module 304 is specifically used to perform target processing on the landmark images included in the target classification, and generate a new landmark image of the target landmark based on the processed image, wherein the target processing includes: at least one of random cropping processing, random affine processing and random color parameter adjustment.

[0137] By performing at least one of a plurality of target processing on the landmark images included in the target classification, new landmark images with more diverse target landmarks can be generated based on the processed images.

[0138] See also Figure 4 , Figure 4 This is a schematic structural diagram of a second landmark image generating device provided by an embodiment of the present invention. The device includes the following modules 401-406.

[0139] An image acquisition module 401 is used to obtain each landmark image of the target landmark;

[0140] An image feature acquisition submodule 402 is used to obtain global image features of each landmark image;

[0141] The feature clustering submodule 403 is used to cluster the obtained global image features to obtain feature clusters;

[0142] The image classification submodule 404 is used to classify the landmark images corresponding to the global image features belonging to the same feature cluster into a category corresponding to an image acquisition perspective.

[0143] The target classification determination module 405 is used to select a classification in which the number of landmark images in each classification is less than the preset number of images as the target classification corresponding to the isolated perspective.

[0144] The image generation module 406 is used to perform image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark.

[0145] As can be seen from the above, since the global image features of landmark images belonging to the same image acquisition perspective are often highly similar, the global image features of each landmark image can be clustered. The result of clustering is that the global image features with high similarity are generated into a feature cluster. At this time, the landmark images corresponding to the global image features under the feature cluster can be understood as images belonging to the same image acquisition perspective. In this way, the landmark images corresponding to the global image features belonging to the same feature cluster are divided into a category corresponding to an image acquisition perspective, and each landmark image can be accurately classified according to the image acquisition perspective.

[0146] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0147] Memory 503, used for storing computer programs;

[0148] The processor 501 is used to implement the steps of the landmark image generation method provided in the above method embodiment when executing the program stored in the memory 503.

[0149] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0150] The communication interface is used for communication between the above terminal and other devices.

[0151] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0152] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0153] In another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the landmark image generation method described in any of the above embodiments is implemented.

[0154] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the landmark image generating method described in any one of the above embodiments.

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0156] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0157] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A landmark image generation method, characterized in that: The method comprises: Obtaining landmark images of target landmarks; Classify landmark images according to the image acquisition perspective; Select the category in which the number of landmark images in each category is less than the preset number of images as the target category corresponding to the isolated viewpoint; Image augmentation processing is performed based on the landmark images included in the target classification to generate a new landmark image of the target landmark, wherein the landmark image of the target landmark is used to match with the image to be identified, and the possibility that the target landmark is the landmark corresponding to the image to be identified is determined based on the number of matching images and a first similarity, wherein the first similarity is: the similarity between the matching image and the landmark image of the target landmark.

2. The method according to claim 1, characterized in that The classifying of landmark images according to image acquisition perspectives includes: Obtaining global image features of each landmark image; Clustering the obtained global image features to obtain feature clusters; The landmark images corresponding to the global image features belonging to the same feature cluster are divided into a category corresponding to an image acquisition perspective.

3. The method according to claim 1 or 2, characterized in that: The performing image augmentation processing based on the landmark image included in the target classification to generate a new landmark image of the target landmark includes: Determining the target number of images to be generated according to the number of landmark images included in the target classification and the number of preset images; Image augmentation processing is performed based on the landmark images included in the target classification to generate new landmark images of the target landmarks of the target number.

4. The method according to claim 1 or 2, characterized in that: The performing image augmentation processing based on the landmark image included in the target classification to generate a new landmark image of the target landmark includes: Performing target processing on the landmark images included in the target classification, and generating a new landmark image of the target landmark based on the processed image, wherein the target processing includes at least one of random cropping processing, random affine processing and random color parameter adjustment.

5. A landmark image generating device, characterized in that: The device comprises: An image acquisition module, used to obtain images of each landmark of the target landmark; An image classification module, used to classify landmark images according to the image acquisition perspective; A target classification determination module, used to select a classification in which the number of landmark images in each classification is less than a preset number of images as a target classification corresponding to an isolated perspective; An image generation module is used to perform image augmentation processing based on the landmark images included in the target classification to generate a new landmark image of the target landmark, wherein the landmark image of the target landmark is used to match the image to be identified, and to determine the possibility that the target landmark is the landmark corresponding to the image to be identified based on the number of matching images and a first similarity, wherein the first similarity is: the similarity between the matching image and the landmark image of the target landmark.

6. The device according to claim 5, characterized in that The image classification module comprises: An image feature acquisition submodule is used to obtain the global image features of each landmark image; A feature clustering submodule is used to cluster the obtained global image features to obtain feature clusters; The image classification submodule is used to classify the landmark images corresponding to the global image features belonging to the same feature cluster into a category corresponding to an image acquisition perspective.

7. The device according to claim 5 or 6, characterized in that The image generation module is specifically used to determine the target number of images to be generated based on the number of landmark images included in the target classification and the number of preset images; perform image augmentation processing based on the landmark images included in the target classification to generate new landmark images of the target number of target landmarks.

8. The device according to claim 5 or 6, characterized in that The image generation module is specifically used to perform target processing on the landmark images included in the target classification, and generate a new landmark image of the target landmark based on the processed image, wherein the target processing includes: at least one of random cropping processing, random affine processing and random color parameter adjustment.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 4 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images