Merchant authenticity verification system and method based on street view image recognition

Abstract

The present invention relates to a merchant authenticity verification system and method thereof. The system comprises: a street view roaming module, which calculates a translation scale for translating the coordinate position of an image acquisition point and calculates the coordinate position of the image acquisition point based on the translation scale, and interacts with a street view image system to obtain a corresponding street view image according to the coordinate position of the image acquisition point; an image comparison module, which matches the obtained merchant door head image with the street view image obtained by the street view roaming module to calculate the merchant image similarity; and a text recognition module, which is used to identify the merchant name from the street view image obtained by the street view roaming module; a text comparison module, which is used to match the identified merchant name with the merchant name obtained from the merchant information platform to calculate the merchant name similarity, and judge whether a specified threshold is reached. If it is judged that the merchant name similarity reaches the specified threshold, the merchant authenticity verification is successful. According to the present invention, the accuracy of merchant verification can be improved.

Description

Technical Field

[0001] The present invention relates to computer technology, and in particular to a merchant authenticity verification system based on street view image recognition and a merchant authenticity verification method based on street view image recognition. Background Art

[0002] In order to obtain real merchant information, a unified merchant information platform is currently built to regularize the information of connected merchants. However, the following problems exist in the specific construction process:

[0003] On the merchant information platform where merchants register, merchant information and business information are mixed, and there are omissions and misreporting in the information submission process. In addition, due to changes in business conditions such as merchant relocation and bankruptcy, there may be differences between merchant information and actual conditions;

[0004] The storefront photos uploaded by the merchant when joining the network may have been photoshopped or forged; and

[0005] The storefront photo uploaded by the merchant is inconsistent with the specific location registered, and it is possible that they are not in the same area at all.

[0006] Therefore, considering the above problems, there is an urgent need for effective merchant verification methods to clean merchant data. In order to verify the authenticity of merchants, existing methods in the prior art include, for example, verification methods by manual field inspections, and verification methods combining search engines and natural language processing technologies.

[0007] Among them, although the manual verification method has the highest accuracy rate for merchant verification, its verification efficiency is low and the verification cost is extremely high.

[0008] On the other hand, by combining search engines and natural language processing technology to verify merchants, when an address or longitude and latitude are entered to search for a merchant name, the names of multiple merchants around the location will be returned. In this case, there will also be certain problems. For example, merchants with the same name in the same area cannot be distinguished, and if the merchant address is filled in incorrectly, the merchant information cannot be verified. Summary of the invention

[0009] In view of the above problems, the present invention aims to provide a merchant authenticity verification system and a merchant authenticity verification method based on street view image recognition that do not require human intervention and can improve accuracy.

[0010] A merchant authenticity verification method according to one aspect of the present invention is characterized in that the method verifies the authenticity of a merchant based on a merchant location, a merchant name, and a merchant doorhead image provided by a merchant information platform and a street view image provided by a street view image system, and the method comprises the following steps:

[0011] Calculating a translation scale based on a given reference image, wherein the translation scale is used to translate the coordinate position of the image acquisition point;

[0012] Based on the merchant location information obtained from the merchant information platform, a first street view image corresponding to the merchant location information is obtained from the street view image system, a merchant door head image obtained from the merchant information platform is matched with the first street view image for feature points, and the first street view image is translated along the road direction according to the translation scale to obtain the coordinate position of the image acquisition point;

[0013] Acquire a second street view image corresponding to the coordinate position of the image acquisition point from the street view image system according to the coordinate position of the image acquisition point;

[0014] intercepting a portion of the image from the second street view image to obtain a third street view image;

[0015] Matching the merchant storefront image obtained from the merchant information platform with the third street view image for feature points to calculate merchant image similarity, determining whether the merchant image similarity reaches a specified threshold, and continuing with the following steps if it is determined that the merchant image similarity reaches the specified threshold; and

[0016] The merchant name is identified from the third street view image and matched with the merchant name obtained from the merchant information platform to calculate the merchant name similarity, and it is determined whether the merchant name similarity reaches a specified threshold. If it is determined that the merchant name similarity reaches the specified threshold, the merchant authenticity verification is successful.

[0017] Optionally, calculating the translation scale based on a given reference image includes:

[0018] Given two reference images I1 and I2, wherein the reference images I1 and I2 contain calibration points A; and

[0019] The shooting coordinate position of the reference image I1 is (x1, y1), and the shooting coordinate position of the reference image I2 is (x2, y2) (x2>x1). The horizontal coordinates of the calibration point A in the reference image I1 and the reference image I2 are p1 and p2 (p2-p1), respectively. The translation scale is

[0020] Optionally, before calculating the translation scale based on a given reference image, the method further includes:

[0021] According to the merchant location information obtained from the merchant information platform, a search range of the image acquisition point corresponding to the merchant location information is obtained.

[0022] Optionally, before translating the first street view image along the road direction to obtain the coordinate position of the image acquisition point, it is further determined whether the coordinate position of the image acquisition point is within the search range of the image acquisition point. If it is determined that the coordinate position is within the search range of the image acquisition point, the subsequent steps are continued; otherwise, the subsequent steps are stopped.

[0023] Optionally, the search range for obtaining the image acquisition point corresponding to the merchant location information includes:

[0024] According to the merchant location information obtained from the merchant information platform, the latitude and longitude range corresponding to the merchant location information and the merchant list C{(cxi, cyi)} are obtained from the merchant information platform.

[0025] The search range is then expressed as: between (min(cxi), min(cyi)) and (max(cxi), max(cyi)).

[0026] Optionally, matching feature points of the storefront image obtained from the store information platform with the first street view image and translating the first street view image along the road direction according to the translation scale to obtain the coordinate position of the image acquisition point includes:

[0027] The shooting position of the first street view image is (sx, sy);

[0028] Acquire a set of matching feature points P={p1, p2, ..., pn} in the store front image and the first street view image by a specified feature matching algorithm, where pi is the horizontal coordinate of the matching feature point;

[0029] If the number of feature points n in the matching feature point set P is greater than the threshold m, then the average L of the differences between the center position f of the merchant door head image and the matching feature point set P is calculated; if n is less than the threshold m, then L is set as the minimum moving distance l; and

[0030] The coordinate position of the image acquisition point is N, N is (sx+Lkcos(t), sy+kLsin(t)),

[0031] Wherein, k is the translation scale, t is the angle between the street direction and the longitude line,

[0032] The angle t is expressed as follows:

[0033] t=argtan((mean(cxi)-min(cxi)) / (mean(cyi)-min(cyi))).

[0034] Optionally, the specified feature matching algorithm includes any one of the following:

[0035] SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

[0036] Optionally, intercepting a portion of the image from the second street view image to obtain a third street view image includes:

[0037] Acquire a set of matching feature points Q={(qx1, qy1), (qx2, qy2), ..., (qxn, qyn)} in the store front image and the second street view image by a prescribed feature matching algorithm, wherein (qxi, qyi) are the horizontal and vertical coordinates of the matching feature points; and

[0038] A portion of the image is intercepted from the second street view image in the following manner to obtain the third street view image:

[0039] So that the upper left and lower right coordinates of the third street view image are

[0040] {(min(qxi)-a,min(qyi)-a),(max(qxi)+a,max(qyi)+a)},

[0041] Where a is the pixel distance, ranging from 10 to 100.

[0042] Optionally, the specified feature matching algorithm includes any one of the following:

[0043] SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

[0044] Optionally, the merchant image similarity is calculated using any of the following matching methods:

[0045] Matching methods based on image grayscale, feature-based matching methods, model-based matching methods, and transform domain-based matching methods.

[0046] Optionally, the merchant name similarity is calculated by any of the following comparison methods:

[0047] There are three types of text similarity calculation methods: string-based text similarity calculation method, corpus-based text similarity calculation method, and world knowledge-based text similarity calculation method.

[0048] A merchant authenticity verification system according to one aspect of the present invention is characterized in that the system verifies the authenticity of a merchant based on a merchant location, merchant name, and merchant doorhead image provided by a merchant information platform and a street view image provided by a street view image system, and the system comprises:

[0049] A street view roaming module obtains the merchant location, merchant name and merchant door head image from the merchant information platform, calculates a translation scale for translating the coordinate position of the image acquisition point and the coordinate position of the image acquisition point obtained by calculating the moving street view image based on the translation scale, and interacts with the street view image system to obtain a corresponding street view image according to the coordinate position of the image acquisition point;

[0050] An image comparison module matches the merchant storefront image obtained from the merchant information platform with the street view image obtained by the street view roaming module to calculate the merchant image similarity, and determines whether the merchant image similarity reaches a specified threshold; and

[0051] A text recognition module, used to recognize business names from the street view image obtained by the street view roaming module;

[0052] The text comparison module is used to match the merchant name identified by the text recognition module with the merchant name obtained from the merchant information platform to calculate the merchant name similarity, and determine whether the merchant name similarity reaches a specified threshold. If it is determined that the merchant name similarity reaches the specified threshold, the merchant authenticity verification is successful.

[0053] Optionally, the street view roaming module obtains the merchant location, merchant name and merchant storefront image from the merchant information platform, calculates the translation scale based on a given reference image, obtains a first street view image corresponding to the merchant location information from the street view image system based on the merchant location information obtained from the merchant information platform, translates the first street view image along the road direction to obtain the coordinate position of the image acquisition point based on the merchant storefront image obtained from the merchant information platform and the feature point matching of the first street view image and the translation scale, obtains a second street view image corresponding to the coordinate position of the image acquisition point from the street view image system according to the coordinate position of the image acquisition point, and captures a portion of the image from the second street view image to obtain a third street view image.

[0054] The image comparison module matches the merchant door head image obtained from the merchant information platform with the third street view image obtained by the street view roaming module to calculate the merchant image similarity.

[0055] The text recognition module recognizes a business name from the third street view image.

[0056] Optionally, calculating the translation scale based on a given reference image includes:

[0057] Given two reference images I1 and I2, the reference images I1 and I2 contain calibration point A, the shooting coordinate position of reference image I1 is (x1, y1), the shooting coordinate position of reference image I2 is (x2, y2) (x2>x1), the horizontal coordinates of calibration point A in the reference images I1 and I2 are p1 and p2 (p2-p1), respectively, then the translation scale is

[0058] Optionally, the street view roaming module further obtains a search range of image acquisition points corresponding to the merchant location information based on the merchant location information obtained from the merchant information platform.

[0059] Optionally, before translating the first street view image along the road direction to obtain the coordinate position of the image acquisition point, the street view roaming module further determines whether the coordinate position of the image acquisition point is within the search range of the image acquisition point, and continues with subsequent steps if it is determined to be within the search range of the image acquisition point, otherwise stops the subsequent steps.

[0060] Optionally, the search range for obtaining the image acquisition point corresponding to the merchant location information includes:

[0061] According to the merchant location information obtained from the merchant information platform, the latitude and longitude range corresponding to the merchant location information and the merchant list C{(cxi, cyi)} are obtained from the merchant information platform.

[0062] The search range is then expressed as: between (min(cxi), min(cyi)) and (max(cxi), max(cyi)).

[0063] Optionally, based on the matching of feature points between the merchant door head image obtained from the merchant information platform and the first street view image and the translation scale, the first street view image is translated along the road direction to obtain the coordinate position of the image acquisition point, including:

[0064] The shooting position of the first street view image is (sx, sy);

[0065] Acquire a set of matching feature points P={p1, p2, ..., pn} in the store front image and the first street view image by a specified feature matching algorithm, where pi is the horizontal coordinate of the matching feature point;

[0066] If the number of feature points n in the matching feature point set P is greater than the threshold m, then the average L of the differences between the center position f of the merchant door head image and the matching feature point set P is calculated; if n is less than the threshold m, then L is set as the minimum moving distance l; and

[0067] The coordinate position of the image acquisition point is N, N is (sx+Lkcos(t), sy+kLsin(t)),

[0068] Wherein, k is the translation scale, t is the angle between the street direction and the longitude line, and the angle t is expressed by the following formula:

[0069] t=argtan((mean(cxi)-min(cxi)) / (mean(cyi)-min(cyi))).

[0070] Optionally, the specified feature matching algorithm includes any one of the following:

[0071] SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

[0072] Optionally, intercepting a portion of the image from the second street view image to obtain a third street view image includes:

[0073] Acquire a set of matching feature points Q={(qx1, qy1), (qx2, qy2), ..., (qxn, qyn)} in the store front image and the second street view image by a prescribed feature matching algorithm, wherein (qxi, qyi) are the horizontal and vertical coordinates of the matching feature points; and

[0074] A portion of the image is intercepted from the second street view image in the following manner to obtain the third street view image:

[0075] So that the upper left and lower right coordinates of the third street view image are

[0076] {(min(qxi)-a,min(qyi)-a),(max(qxi)+a,max(qyi)+a)},

[0077] Where a is the pixel distance, ranging from 10 to 100.

[0078] Optionally, the specified feature matching algorithm includes any one of the following:

[0079] SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

[0080] Optionally, the image comparison module adopts any one of the following matching methods:

[0081] Matching methods based on image grayscale, feature-based matching methods, model-based matching methods, and transform domain-based matching methods.

[0082] Optionally, the text comparison module adopts any of the following comparison methods:

[0083] There are three types of text similarity calculation methods: string-based text similarity calculation method, corpus-based text similarity calculation method and world knowledge-based text similarity calculation method.

[0084] A computer-readable medium according to one aspect of the present invention stores a computer program thereon, wherein the computer program implements the merchant authenticity verification method when executed by a processor.

[0085] A computer device according to one aspect of the present invention comprises a storage module, a processor, and a computer program stored in the storage module and executable on the processor, wherein the processor implements the merchant authenticity verification method when executing the computer program.

[0086] The computer readable medium of the present invention stores a computer program thereon, characterized in that:

[0087] When the computer program is executed by a processor, the merchant authenticity verification method is implemented.

[0088] The computer device of the present invention comprises a storage module, a processor and a computer program stored in the storage module and executable on the processor, wherein the processor implements the merchant authenticity verification method when executing the computer program.

[0089] As described above, in the merchant authenticity verification system and the merchant authenticity verification method based on street view image recognition of the present invention, since the street view image databases of companies such as Baidu are relatively complete, there is no need to build a street view database by themselves. The roaming search of street view images combined with the use of image comparison technology can greatly improve the merchant verification accuracy, and no manual participation is required, saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a structural block diagram showing the merchant authenticity verification system of the present invention.

[0091] Figure 2 It is a schematic diagram showing the flow of the merchant authenticity verification method of the present invention.

[0092] Figure 3 This is a schematic diagram showing the use of the SIFT feature matching algorithm. DETAILED DESCRIPTION

[0093] The following describes some of the various embodiments of the present invention, which are intended to provide a basic understanding of the present invention, but are not intended to identify the key or decisive elements of the present invention or to limit the scope of protection.

[0094] For the purpose of brevity and illustration, the principles of the present invention are described herein mainly with reference to its exemplary embodiments. However, those skilled in the art will readily recognize that the same principles are equally applicable to and can be implemented in all types of merchant authenticity verification systems and merchant authenticity verification methods based on street view image recognition, and any such changes do not depart from the true spirit and scope of the present patent application.

[0095] Moreover, in the following description, reference is made to the accompanying drawings, which illustrate specific exemplary embodiments. Electrical, mechanical, logical and structural changes may be made to these embodiments without departing from the spirit and scope of the present invention. In addition, although the features of the present invention are disclosed in conjunction with only one of several implementations / embodiments, this feature may be combined with one or more other features of other implementations / embodiments as may be desirable and / or advantageous for any given or identifiable function. Therefore, the following description should not be regarded as limiting, and the scope of the present invention is defined by the appended claims and their equivalents.

[0096] Terms such as “having” and “including” indicate that in addition to the units (modules) and steps directly and explicitly stated in the specification and claims, the technical solution of the present invention does not exclude the situation where it has other units (modules) and steps that are not directly or explicitly stated.

[0097] Figure 1 It is a structural block diagram showing the merchant authenticity verification system of the present invention.

[0098] like Figure 1 As shown, the merchant authenticity verification system 100 of the present invention verifies the authenticity of the merchant based on the merchant location, merchant name, merchant door head image provided by the merchant information platform 200 and the street view image provided by the street view image system 300.

[0099] The merchant authenticity verification system 100 of the present invention comprises:

[0100] The street view roaming module 110 interacts with the street view image system 300, calculates the road coordinate range, calculates the translation scale, moves the data collection point along the road direction, and obtains the street view image data;

[0101] An image comparison module 120 performs matching through feature points and calculates the similarity of images;

[0102] A text recognition module 130 for recognizing the business name of the corresponding business in the street view image;

[0103] The text comparison module 140 matches the merchant name identified by the text recognition module with the merchant name filled in by the merchant and calculates the similarity of the merchant names.

[0104] Here, the merchant information platform 200 can provide information such as the merchant location, merchant name, and merchant door image. The street view image system 300 can provide a street view image of a corresponding location based on the coordinate location provided by the street view roaming module 110. The street view image system 300 can be implemented using existing technologies. For example, the street view image databases of companies such as Baidu Maps and Amap are relatively complete, and there is no need to build a street view database separately.

[0105] As an example, it can be set that the similarity of the image obtained by the image comparison module 120 first reaches above a specified threshold and then the similarity of the merchant name is calculated by the text comparison module 140, and when the similarity of the merchant name calculated by the text comparison module 140 reaches the specified threshold, it is determined that the merchant authenticity verification is successful.

[0106] Figure 2 It is a schematic diagram showing the flow of the merchant authenticity verification method of the present invention.

[0107] like Figure 2 As shown, the merchant authenticity verification method of the present invention mainly includes the following steps:

[0108] Step S1: the street view roaming module 110 determines the search scope according to the merchant location and merchant name obtained from the merchant information platform, wherein the merchant information platform can provide information such as merchant name, merchant location, merchant door head image uploaded by the merchant;

[0109] Step S2: the street view navigation module 110 calculates the translation scale based on a given reference image;

[0110] Step S3: the street view roaming module 110 obtains the coordinate position of the data collection point (starting point) of the street view roaming based on the business door head image F and the street view image S uploaded by the business to the business information platform by the business using a prescribed feature matching algorithm based on the translation scale;

[0111] Step S4: acquiring the corresponding street view image S1 from the street view image system 200 according to the coordinate position of the data collection point (starting point);

[0112] Step S5: obtaining the coordinate positions of the matching points in the storefront image F and the street view image S1 according to a specified feature matching algorithm, performing a specified calculation on the coordinate positions of the matching points to obtain the calculated coordinate positions, and obtaining the intercepted street view image S2 according to the calculated coordinate positions;

[0113] Step S6: the image comparison module 120 matches the feature points of the merchant storefront image F uploaded by the user and the street view image S2 and calculates the merchant image similarity;

[0114] Step S7: the image comparison module 120 determines whether the merchant image similarity reaches a specified threshold, and if so, proceeds to step S8, otherwise, jumps to step S13;

[0115] Step S8: the text recognition module 130 recognizes the business name from the street view image S2;

[0116] Step S9: the text comparison module 140 matches the merchant name identified by the text recognition module 130 with the merchant name uploaded by the merchant and calculates the merchant name similarity;

[0117] Step S10: the text comparison module 140 determines whether the merchant name similarity reaches a specified threshold, if so, proceeds to step S11, if not, jumps to step S13;

[0118] Step S11: Merchant verification is successful;

[0119] Step S12: The search ends and the search and verification results are returned.

[0120] Step S13: the street view roaming module 110 obtains the next data collection point;

[0121] Step S14: the street view roaming module 110 determines whether the next data collection point obtained is beyond the search range, and if it is not beyond the search range, returns to step S4; if it is beyond the search range, jumps to step S12.

[0122] Hereinafter, a specific implementation of the merchant authenticity verification system and merchant authenticity verification method based on street view image recognition of the present invention will be described.

[0123] First, how the street view navigation module 110 obtains the search range is described.

[0124] The street view roaming module 110 obtains the merchant location and merchant name from the merchant information platform. The merchant location and merchant name on the merchant information platform are information uploaded by the merchant when registering. The street view roaming module 110 obtains the latitude and longitude range corresponding to the location information (for example, the merchant can be selected within 1 km) and the merchant list C{(cxi, cyi)} according to the location information registered by the merchant.

[0125] Therefore, the search range is obtained as: (min(cxi), min(cyi)) to (max(cxi), max(cyi)).

[0126] Thus, the angle t between the street and the latitude line is expressed as follows:

[0127] t=argtan((mean(cxi)-min(cxi)) / (mean(cyi)-min(cyi))).

[0128] Next, the specific process of calculating the translation scale of the street view image by the street view navigation module 110 is described.

[0129] In order to identify the text content of the storefront in the street view by using the image text recognition technology, it is necessary to obtain the front image of the storefront as much as possible by translating the latitude and longitude coordinates of the image acquisition point.

[0130] In actual street view images, due to the large differences in street width and shooting distance, the translation scale and the scale between the pixel movement distance in the image and the actual movement distance need to be calculated for each street.

[0131] Given two reference images I1 and I2, the reference images I1 and I2 contain calibration point A, the shooting position of reference image I1 is (x1, y1), the shooting position of reference image I2 is (x2, y2) (x2>x1), the horizontal coordinates of A in the two images are p1 and p2 (p2-p1), then the translation scale is

[0132] As the two reference images I1 and I2 mentioned above, two images containing a common area randomly selected from the street view image are used, and the distance between the shooting positions of the two images is known.

[0133] Next, the specific steps of how the street view navigation module 110 obtains the next candidate data collection point are described.

[0134] The street view roaming module 110 obtains the next candidate data collection point through the following steps:

[0135] (1) Obtaining a storefront image F and a street view image S from the store information platform. The shooting position of the street view image S is (sx, sy). The shooting position of the street view image S is (sx, sy) and can be obtained from the street view image system 300;

[0136] (2) Obtain a set of matching feature points P = {p1, p2, ..., pn} in the storefront image F and the street view image S through the SIFT feature matching algorithm, where pi is the horizontal coordinate of the matching feature point;

[0137] (3) If the number of feature points n in P is greater than the threshold m, the mean L of the differences between the center position f of F and each value in P is calculated. If n is less than the threshold m, L is set to the minimum moving distance l; and

[0138] (4) The next candidate collection point is N, whose position is (sx+Lkcos(t),sy+kLsin(t)), where k is the translation scale and t is the angle between the street direction and the longitude line.

[0139] Figure 3 is a schematic diagram showing the use of SIFT feature matching algorithm. Figure 3 In the figure, the left side is the storefront image F, and the right side is the street view image S.

[0140] The SIFT feature matching algorithm is a scale-invariant feature transformation, which is used to detect local features of an image, find extreme points in the spatial scale, and extract the position, scale, and rotation invariants of this point. These key points are very prominent points that will not change due to factors such as lighting and noise, such as corner points, edge points, bright spots in dark areas, and dark spots in bright areas. Therefore, they are independent of the size and rotation of the image, and have a high tolerance for changes in light, noise, and viewing angle.

[0141] SIFT is a similarity invariant in theory, that is, it is invariant to image scale changes and rotations. However, due to the special processing of many details when constructing SIFT features, SIFT has a strong adaptability to complex deformations and illumination changes of images, and the operation speed is relatively fast and the positioning accuracy is relatively high. For example, the DOG operator is used to detect key points in multi-scale space, which greatly speeds up the operation speed compared to the traditional detection method based on the LOG operator; the precise positioning of key points not only improves the accuracy, but also greatly improves the stability of key points.

[0142] In this embodiment, the SIFT feature matching algorithm is cited as an example, which is only an example. In addition, other image feature matching algorithms can also be used. For example, SURF (Speeded Up Robust Feature) algorithm, SIFT (Scale Invariant Feature Transform) algorithm, ORB (ORiented Brief) algorithm, FAST (Features from Accelerated Segment Test) algorithm, Harris (corner point) algorithm, etc.

[0143] The following describes how the street view navigation module 110 crops the image of the candidate collection point.

[0144] The street view roaming module 110 implements cropping of candidate acquisition point images through the following steps:

[0145] (1) Given a merchant’s storefront image F uploaded by the merchant;

[0146] (2) The data collection position is translated to the candidate collection point position N, and the street view image S1 is obtained again from the street view image system 200;

[0147] (3) Obtaining a set of matching feature points Q = {(qx1, qy1), (qx2, qy2), ..., (qxn, qyn)} in the storefront image F and the street view image S1 by using the SIFT feature matching algorithm, where (qxi, qyi) are the horizontal and vertical coordinates of the matching feature points; and

[0148] (4) A portion of the image is intercepted from the street image S1 to obtain a street view image S2, whose upper left and lower right coordinates are {(min(qxi)-a, min(qyi)-a), (max(qxi)+a, max(qyi)+a)}, where a is the pixel distance, and its value range is 10 to 100. In addition, the purpose of intercepting the street view image S2 from the street image S1 is to filter out the redundant images other than the key feature points to prevent the occurrence of similarity matching between large and small images, thereby improving the accuracy of image matching.

[0149] The image comparison module 120 matches the feature points of the merchant storefront image F and the street view image S2 uploaded by the user and calculates the merchant image similarity. Here, the image similarity calculation method adopted by the image comparison module 120 can adopt, for example, a matching method commonly used in existing computer vision, such as a matching method based on image grayscale, a matching method based on features, a matching method based on models, and a matching method based on transform domains. As an example, please refer to:

[0150] https: / / wenku.baidu.com / view / 019ec260657d27284b73f242336c1eb91a3733 0f? fr=xueshu20001

[0151] The text comparison module 140 matches the merchant name identified by the text recognition module 130 with the merchant name uploaded by the merchant and calculates the merchant name similarity. Here, the merchant name similarity calculation method adopted by the text comparison module 140 can adopt, for example, existing text similarity calculation methods, such as string-based methods, corpus-based methods, and world knowledge-based methods. As an example, please refer to:

[0152] https: / / blog.csdn.net / gamer_gyt / article / details / 102916791

[0153] As described above, in the merchant authenticity verification system and the merchant authenticity verification method based on street view image recognition of the present invention, since the street view image databases of companies such as Baidu Maps and Amap are relatively complete, there is no need to build a self-built street view database. The roaming search of street view images combined with the use of image comparison technology can greatly improve the merchant verification accuracy, and no manual participation is required, saving labor costs.

[0154] The above examples mainly illustrate the merchant authenticity verification system and the merchant authenticity verification method based on street view image recognition of the present invention. Although only some of the specific embodiments of the present invention are described, it should be understood by those skilled in the art that the present invention can be implemented in many other forms without departing from its subject matter and scope. Therefore, the examples and embodiments shown are regarded as illustrative rather than restrictive, and the present invention may cover various modifications and substitutions without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A merchant authenticity verification method, characterized in that: The method verifies the authenticity of a merchant based on the merchant location, merchant name, merchant doorhead image provided by the merchant information platform and the street view image provided by the street view image system. The method includes the following steps: Calculating a translation scale based on two given reference images, wherein the translation scale is used to translate the coordinate position of the image acquisition point; Based on the merchant location information obtained from the merchant information platform, a first street view image corresponding to the merchant location information is obtained from the street view image system, a merchant door head image obtained from the merchant information platform is matched with the first street view image for feature points, and the first street view image is translated along the road direction according to the translation scale to obtain the coordinate position of the image acquisition point; Acquire a second street view image corresponding to the coordinate position of the image acquisition point from the street view image system according to the coordinate position of the image acquisition point; intercepting a portion of the image from the second street view image to obtain a third street view image; Matching the merchant storefront image obtained from the merchant information platform with the third street view image for feature points to calculate merchant image similarity, determining whether the merchant image similarity reaches a specified threshold, and continuing with the following steps if it is determined that the merchant image similarity reaches the specified threshold; and The merchant name is identified from the third street view image and matched with the merchant name obtained from the merchant information platform to calculate the merchant name similarity, and it is determined whether the merchant name similarity reaches a specified threshold. If it is determined that the merchant name similarity reaches the specified threshold, the merchant authenticity verification is successful.

2. The merchant authenticity verification method according to claim 1, characterized in that: The calculation of the translation scale based on two given reference images comprises: Given two reference images I1 and I2, wherein the reference images I1 and I2 contain calibration points A; and The shooting coordinate position of the reference image I1 is (x1, y1), and the shooting coordinate position of the reference image I2 is (x2, y2), where x2 > x1. The horizontal coordinates of the calibration point A in the reference image I1 and the reference image I2 are p1 and p2 respectively, where p2 < p1. Then the translation scale is 3. The merchant authenticity verification method according to claim 2, characterized in that: Before calculating the translation scale based on a given reference image, further comprising: According to the merchant location information obtained from the merchant information platform, a search range of the image acquisition point corresponding to the merchant location information is obtained.

4. The merchant authenticity verification method according to claim 3, characterized in that: Before translating the first street view image along the road direction to obtain the coordinate position of the image acquisition point, it is further determined whether the coordinate position of the image acquisition point is within the search range of the image acquisition point. If it is determined that the coordinate position is within the search range of the image acquisition point, the subsequent steps are continued; otherwise, the subsequent steps are stopped.

5. The merchant authenticity verification method according to claim 4, characterized in that: The search range for obtaining the image acquisition point corresponding to the merchant location information includes: According to the merchant location information obtained from the merchant information platform, the latitude and longitude range corresponding to the merchant location information and the merchant list C{(cxi, cyi)} are obtained from the merchant information platform. The search range is then expressed as: between (min(cxi), min(cyi)) and (max(cxi), max(cyi)).

6. The merchant authenticity verification method according to claim 5, characterized in that: Matching feature points of the storefront image obtained from the store information platform with the first street view image and translating the first street view image along the road direction according to the translation scale to obtain the coordinate position of the image acquisition point includes: The shooting position of the first street view image is (sx, sy); Acquire a set of matching feature points P={P1, P2, ..., Pn} in the store front image and the first street view image by a specified feature matching algorithm, wherein Pi is the horizontal coordinate of the matching feature point in the first street view image; If the number of feature points n in the matching feature point set P is greater than the threshold m, then the average L of the differences between the center position f of the merchant door head image and the matching feature point set P is calculated; if n is less than the threshold m, then L is set as the minimum moving distance l; and The coordinate position of the image acquisition point is N, N is (sx+Lkcos(t), sy+kLsin(t)), Wherein, k is the translation scale, t is the angle between the street direction and the longitude line, The angle t is expressed as follows: t=argtan((mean(cxi)-min(cxi)) / (mean(cyi)-min(cyi))).

7. The merchant authenticity verification method according to claim 6, characterized in that: The specified feature matching algorithm includes any one of the following: SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

8. The merchant authenticity verification method according to claim 6, characterized in that: Cutting out a portion of the image from the second street view image to obtain a third street view image comprises: Acquire a set of matching feature points Q={(qx1, qy1), (qx2, qy2), ..., (qxn, qyn)} in the store front image and the second street view image by a prescribed feature matching algorithm, wherein (qxi, qyi) are the horizontal and vertical coordinates of the matching feature points in the second street view image; and A portion of the image is intercepted from the second street view image in the following manner to obtain the third street view image: So that the upper left and lower right coordinates of the third street view image are: {(min(qxi)- a,min(qyi)-a),(max(qxi)+a,max(qyi)+a)}, Where a is the pixel distance, ranging from 10 to 100.

9. The merchant authenticity verification method according to claim 8, characterized in that: The specified feature matching algorithm includes any one of the following: SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

10. The merchant authenticity verification method according to claim 1, characterized in that: The merchant image similarity calculation adopts any of the following matching methods: Matching methods based on image grayscale, feature-based matching methods, model-based matching methods, and transform domain-based matching methods.

11. The merchant authenticity verification method according to claim 1, characterized in that: The calculation of merchant name similarity adopts any of the following comparison methods: There are three types of text similarity calculation methods: string-based text similarity calculation method, corpus-based text similarity calculation method, and world knowledge-based text similarity calculation method.

12. A merchant authenticity verification system, characterized in that: The system verifies the authenticity of merchants based on the merchant location, merchant name, merchant door image provided by the merchant information platform and the street view image provided by the street view image system. The system includes: A street view roaming module obtains the merchant location, merchant name and merchant door head image from the merchant information platform, calculates a translation scale for translating the coordinate position of the image acquisition point and the coordinate position of the image acquisition point obtained by calculating the moving street view image based on the translation scale, and interacts with the street view image system to obtain a corresponding street view image according to the coordinate position of the image acquisition point; An image comparison module matches the merchant storefront image obtained from the merchant information platform with the street view image obtained by the street view roaming module to calculate the merchant image similarity, and determines whether the merchant image similarity reaches a specified threshold; and A text recognition module, used to recognize business names from the street view image obtained by the street view roaming module; The text comparison module is used to match the merchant name identified by the text recognition module with the merchant name obtained from the merchant information platform to calculate the merchant name similarity, and determine whether the merchant name similarity reaches a specified threshold. If it is determined that the merchant name similarity reaches the specified threshold, the merchant authenticity verification is successful, wherein the street view roaming module obtains the merchant location, merchant name and merchant door image from the merchant information platform, calculates the translation scale based on the given two reference images, obtains a first street view image corresponding to the merchant location information from the street view image system based on the merchant location information obtained from the merchant information platform, and translates the first street view image along the road direction to obtain the coordinate position of the image acquisition point based on the feature point matching between the merchant door image obtained from the merchant information platform and the first street view image and the translation scale, and obtains a second street view image corresponding to the coordinate position of the image acquisition point from the street view image system according to the coordinate position of the image acquisition point, and intercepts a part of the image from the second street view image to obtain a third street view image, The image comparison module matches the merchant door head image obtained from the merchant information platform with the third street view image obtained by the street view roaming module to calculate the merchant image similarity. The text recognition module recognizes a business name from the third street view image.

13. The merchant authenticity verification system according to claim 12, characterized in that: The calculation of the translation scale based on two given reference images comprises: Given two reference images I1 and I2, the reference images I1 and I2 contain a calibration point A. The shooting coordinate position of the reference image I1 is (x1, y1), and the shooting coordinate position of the reference image I2 is (x2, y2), where x2 > x1. The horizontal coordinates of the calibration point A in the reference image I1 and the reference image I2 are p1 and p2 respectively, where p2 < p1. Then the translation scale is 14. The merchant authenticity verification system according to claim 13, characterized in that: The street view roaming module further obtains a search range of image acquisition points corresponding to the merchant location information based on the merchant location information obtained from the merchant information platform.

15. The merchant authenticity verification system according to claim 14, characterized in that: Before translating the first street view image along the road direction to obtain the coordinate position of the image acquisition point, the street view roaming module further determines whether the coordinate position of the image acquisition point is within the search range of the image acquisition point. If it is determined that the coordinate position is within the search range of the image acquisition point, the subsequent steps are continued, otherwise the subsequent steps are stopped.

16. The merchant authenticity verification system according to claim 15, characterized in that: The search range for obtaining the image acquisition point corresponding to the merchant location information includes: According to the merchant location information obtained from the merchant information platform, the latitude and longitude range corresponding to the merchant location information and the merchant list C{(cxi, cyi)} are obtained from the merchant information platform. Then the search range is expressed as: Between (min(cxi), min(cyi)) and (max(cxi), max(cyi)).

17. The merchant authenticity verification system according to claim 16, characterized in that: Based on the feature point matching between the merchant door head image obtained from the merchant information platform and the first street view image and the translation scale, the first street view image is translated along the road direction to obtain the coordinate position of the image acquisition point, including: The shooting position of the first street view image is (sx, sy); Acquire a set of matching feature points P={P1, P2, ..., Pn} in the store front image and the first street view image by a specified feature matching algorithm, wherein Pi is the horizontal coordinate of the matching feature point in the first street view image; If the number of feature points n in the matching feature point set P is greater than the threshold m, then the average L of the differences between the center position f of the merchant door head image and the matching feature point set P is calculated; if n is less than the threshold m, then L is set as the minimum moving distance l; and The coordinate position of the image acquisition point is N, N is (sx+Lkcos(t), sy+kLsin(t)), Wherein, k is the translation scale, t is the angle between the street direction and the longitude line, and the angle t is expressed by the following formula: t=argtan((mean(cxi)-min(cxi)) / (mean(cyi)-min(cyi))).

18. The merchant authenticity verification system according to claim 17, characterized in that: The specified feature matching algorithm includes any one of the following: SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

19. The merchant authenticity verification system according to claim 17, characterized in that: Cutting out a portion of the image from the second street view image to obtain a third street view image comprises: Acquire a set of matching feature points Q={(qx1, qy1), (qx2, qy2), ..., (qxn, qyn)} in the store front image and the second street view image by a prescribed feature matching algorithm, wherein (qxi, qyi) are the horizontal and vertical coordinates of the matching feature points in the second street view image; and A portion of the image is intercepted from the second street view image in the following manner to obtain the third street view image: So that the upper left and lower right coordinates of the third street view image are {(min(qxi)-a,min(qyi)-a),(max(qxi)+a,max(qyi)+a)}, Where a is the pixel distance, ranging from 10 to 100.

20. The merchant authenticity verification system according to claim 19, characterized in that: The specified feature matching algorithm includes any one of the following: SIFT algorithm, SURF algorithm, SIFT algorithm, ORB algorithm, FAST algorithm, Harri algorithm.

21. The merchant authenticity verification system according to claim 20, characterized in that: The image comparison module adopts any of the following matching methods: Matching methods based on image grayscale, feature-based matching methods, model-based matching methods, and transform domain-based matching methods.

22. The merchant authenticity verification system according to claim 20, characterized in that: The text comparison module adopts any of the following comparison methods: There are three types of text similarity calculation methods: string-based text similarity calculation method, corpus-based text similarity calculation method and world knowledge-based text similarity calculation method.

23. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the merchant authenticity verification method described in any one of claims 1 to 11 is implemented.

24. A computer device comprising a storage module, a processor, and a computer program stored in the storage module and executable on the processor, characterized in that: When the processor executes the computer program, the merchant authenticity verification method described in any one of claims 1 to 11 is implemented.

Images