Shop authenticity verification method and device, storage medium and computer equipment
By performing multi-faceted detection and text recognition of the target image of the store, the authenticity of the store information submitted by the merchant is automatically verified, and the problem of time-consuming and low accuracy of store photo verification in the existing technology is solved, achieving efficient and accurate Verification of authenticity of merchant stores.
Patent Information
- Application Number
- CN202410902025.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-12-06
AI Technical Summary
During the merchant’s online review process, verification of the authenticity of store photos takes a lot of time and manpower, and has low accuracy.
By obtaining the target image and related information of the store, store door head detection, moiré detection, screenshot detection and watermark detection are carried out, combined with text recognition and feature extraction, the authenticity of images and information is automatically verified, and the intelligent authenticity verification of the store is realized. .
It reduces the workload of manual review, improves the efficiency and accuracy of authenticity verification of merchant stores, realizes a dual-factor authentication mechanism for images and information, and reduces labor and time costs.
Smart Images

Figure CN119090568A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a method, device, storage medium and computer equipment for verifying the authenticity of a store. Background Art
[0002] Merchant network access review is an important part of the risk control system of third-party payment institutions and is crucial to building a healthy, safe and efficient payment ecosystem. During the merchant network access review, a lot of time and manpower are required to identify the authenticity of the store photos provided by users, and the accuracy of store photo verification is low. Summary of the invention
[0003] In view of this, the present application provides a store authenticity verification method, device, storage medium and computer equipment to realize automated and intelligent authenticity verification of store information submitted by users.
[0004] According to one aspect of the present application, a method for verifying the authenticity of a store is provided, comprising:
[0005] Get the target image of the store and the first store information;
[0006] Performing detection processing of image detection items on the target image to obtain detection results of the image detection items, wherein the image detection items include at least one of the following: store front detection, moiré detection, screenshot detection, or watermark detection;
[0007] When the detection result of the image detection item meets the preset condition, identifying the second store information in the target image;
[0008] The authenticity verification result of the store is determined based on the similarity between the first store information and the second store information.
[0009] Optionally, the image detection project includes store front detection, and the performing detection processing of the image detection project on the target image specifically includes:
[0010] Performing mirror-inversion processing on the target image to obtain a flipped image;
[0011] Performing text recognition processing on the target image and the flipped image respectively, and determining the confidence of the first text in the target image and the confidence of the second text in the flipped image;
[0012] When the confidence level of the first text is less than the confidence level of the second text, replacing the target image with the mirror image;
[0013] The target image is input into a storefront detection model to obtain a detection result that the target image includes a storefront, wherein the storefront detection model is trained based on a first sample image and a storefront label, and the first sample image includes a storefront image.
[0014] Optionally, the image detection item includes moiré detection, and the performing the detection process of the image detection item on the target image specifically includes:
[0015] Obtain a moiré image and a store template image;
[0016] Extracting moiré features contained in the moiré image;
[0017] Performing perspective transformation processing on the store template image according to the moiré feature to form a second sample image and an edge map of the moiré feature in the second sample image;
[0018] training a moiré recognition model according to the second sample image and the edge map;
[0019] Inputting the target image into a moiré recognition model to obtain a probability that the target image contains moiré;
[0020] A detection result of the moiré detection is determined according to the probability.
[0021] Optionally, the image detection project includes screenshot detection, and the performing the detection process of the image detection project on the target image specifically includes:
[0022] Read the image file format information associated with the target image, and determine the detection result of the screenshot detection according to the reading completion status of the image file format information, wherein the reading completion status includes reading success or reading failure, and the image file format information includes: shooting time, device model, focal length parameter, aperture parameter at least one.
[0023] Optionally, the image detection project includes screenshot detection, and the performing the detection process of the image detection project on the target image specifically includes:
[0024] The detection result of the screenshot detection is determined according to the size relationship between the resolution of the target image and the preset screenshot resolution.
[0025] Optionally, the image detection project includes screenshot detection, and the performing the detection process of the image detection project on the target image specifically includes:
[0026] The aspect ratio similarity between the aspect ratio of the target image and a plurality of preset aspect ratios is calculated by using the least common multiple method, and the detection result of the screenshot detection is determined according to the size relationship between the similarity and the preset similarity.
[0027] Optionally, the image detection project includes screenshot detection, and the performing the detection process of the image detection project on the target image specifically includes:
[0028] The target image is input into an interface element recognition model to obtain interface element features in the target image, and based on the interface element features, a detection result of the screenshot detection is determined, wherein the interface element recognition model is trained based on a third sample image and interface element labels, and the interface element features include information prompt bar features, application identification features, and screen boundary features.
[0029] Optionally, before inputting the target image into the interface element recognition model, the method further includes:
[0030] Inputting the third sample image into a feature extraction model to obtain a global feature vector and a local feature vector of the third sample image, wherein the feature extraction model includes a global feature extraction channel and a local feature extraction channel;
[0031] Determining a channel attention weight according to the global feature vector and the local feature vector;
[0032] Update the third sample image according to the channel attention weight.
[0033] Optionally, the image detection project includes watermark detection, and the performing the detection process of the image detection project on the target image specifically includes:
[0034] The target image is input into a watermark detection model to obtain a detection result that the target image contains a watermark, wherein the watermark detection model is trained based on a fourth sample image and a watermark label, and the fourth sample image includes a watermark image.
[0035] Optionally, determining the authenticity verification result of the store based on the similarity between the first store name and the second store name specifically includes:
[0036] When the similarity between the first store information and the second store information is greater than a similarity threshold, determining that the authenticity verification result is passed;
[0037] When the similarity between the first store information and the second store information is less than or equal to a similarity threshold, sending the first store information to a review node;
[0038] In response to a confirmation instruction fed back by the review node, determining that the authenticity verification result is passed;
[0039] If no confirmation instruction is received from the review node within a preset period of time, the authenticity verification result is determined to be failed.
[0040] Optionally, the method further comprises:
[0041] When the detection result of any of the image detection items does not meet the preset condition, the authenticity verification result is determined to be failed.
[0042] According to another aspect of the present application, a store authenticity verification device is provided, comprising:
[0043] An acquisition module, used for a target image of the store and first store information;
[0044] An image verification module, used to perform detection processing of image detection items on the target image to obtain detection results of the image detection items, wherein the image detection items include at least one of the following: store front detection, moiré detection, screenshot detection or watermark detection;
[0045] A recognition module, configured to recognize second store information in the target image when a detection result of the image detection item meets a preset condition;
[0046] The information verification module is used to determine the authenticity verification result of the store based on the similarity between the first store information and the second store information.
[0047] Optionally, the image detection project includes store front detection, and the device further includes:
[0048] An image flipping module, used for performing mirror-reversal processing on the target image to obtain a flipped image;
[0049] The recognition module is further used to perform text recognition processing on the target image and the flipped image respectively, and determine the confidence of the first text in the target image and the confidence of the second text in the flipped image;
[0050] The image verification module is specifically used to replace the target image with the mirror image when the confidence of the first text is less than the confidence of the second text; and input the target image into a store head detection model to obtain a detection result that the target image contains a store head, wherein the store head detection model is trained based on a first sample image and a store head label, and the first sample image includes a store head image.
[0051] Optionally, the image detection item includes moiré detection, and the device further includes:
[0052] A first feature processing module is used to extract moiré features contained in the moiré image; and perform perspective transformation processing on the store template image according to the moiré features to form a second sample image and an edge map of the moiré features in the second sample image;
[0053] A training module, used for training a moiré recognition model according to the second sample image and the edge map;
[0054] The image verification module is specifically used to obtain a moiré image and a store template image; input the target image into a moiré recognition model to obtain the probability that the target image contains moiré; and determine the detection result of the moiré detection based on the probability.
[0055] Optionally, the image detection project includes screenshot detection, and the image verification module is specifically used to read the image file format information associated with the target image, and determine the detection result of the screenshot detection based on the reading completion status of the image file format information, wherein the reading completion status includes reading success or reading failure, and the image file format information includes: shooting time, device model, focal length parameter, aperture parameter at least one.
[0056] Optionally, the image detection project includes screenshot detection, and the image verification module is specifically used to determine the detection result of the screenshot detection according to the size relationship between the resolution of the target image and a preset screenshot resolution.
[0057] Optionally, the image detection project includes screenshot detection, and the image verification module is specifically used to calculate the aspect ratio similarity between the aspect ratio of the target image and multiple preset aspect ratios using the least common multiple method, and determine the detection result of the screenshot detection based on the size relationship between the similarity and the preset similarity.
[0058] Optionally, the image detection project includes screenshot detection, and the image verification module is specifically used to input the target image into an interface element recognition model to obtain interface element features in the target image, and determine the detection result of the screenshot detection based on the interface element features, wherein the interface element recognition model is trained based on a third sample image and interface element labels, and the interface element features include information prompt bar features, application identification features and screen boundary features.
[0059] Optionally, the device further comprises:
[0060] A second feature processing module is configured to input the third sample image into a feature extraction model to obtain a global feature vector and a local feature vector of the third sample image, wherein the feature extraction model includes a global feature extraction channel and a local feature extraction channel; and determine a channel attention weight according to the global feature vector and the local feature vector;
[0061] An updating module is used to update the third sample image according to the channel attention weight.
[0062] Optionally, the image detection project includes watermark detection, and the image verification module is specifically used to input the target image into a watermark detection model to obtain a detection result that the target image contains a watermark, wherein the watermark detection model is trained based on a fourth sample image and a watermark label, and the fourth sample image includes a watermark image.
[0063] Optionally, the information verification module is specifically used to determine that the authenticity verification result is passed when the similarity between the first store information and the second store information is greater than a similarity threshold; send the first store information to a review node when the similarity between the first store information and the second store information is less than or equal to a similarity threshold; determine that the authenticity verification result is passed in response to a confirmation instruction fed back by the review node; determine that the authenticity verification result is failed when no confirmation instruction is received from the review node within a preset time period; the image verification module is also used to determine that the authenticity verification result is failed when the detection result of any of the image detection items does not meet the preset conditions.
[0064] According to another aspect of the present application, a readable storage medium is provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the authenticity verification method of the above-mentioned store are implemented.
[0065] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and the processor implements the steps of the authenticity verification method of the above-mentioned store when executing the program.
[0066] By means of the above technical solution, when the authenticity of the store information submitted by the user needs to be verified, the system obtains the target image of the store submitted by the user and the first store information related to the store. The system first verifies the authenticity of the target image from multiple aspects, such as whether it is a store door image, whether it is a real scene shot, whether it is a device screenshot, and whether it is a reprinted image, through at least one of the following detection items: store door detection, moiré detection, screenshot detection, and watermark detection. When the detection result of the image detection item meets the preset conditions, that is, after the verification is passed, the system can determine that the target image is a real scene image of the store taken by the user. At this time, text recognition can be performed on the target image to obtain the second store information displayed in the target image. By comparing the second store information in the image with the first store information submitted by the user, if the comparison is passed, it can be determined that the user is a real merchant, and the subsequent process can be executed. Thus, through the automated store image verification and information verification process, a dual authentication mechanism for images and information is realized. On the one hand, manual intervention in the audit process is reduced, labor costs and time costs are reduced, and the efficiency of authenticity verification of merchant stores is greatly improved. On the other hand, it combines a variety of detection technologies to perform intelligent authenticity verification from multiple aspects, greatly improving the merchant verification accuracy.
[0067] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0069] Figure 1 A schematic diagram showing a flow chart of a method for verifying the authenticity of a store provided in an embodiment of the present application;
[0070] Figure 2 A structural block diagram of a store authenticity verification device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0071] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0072] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0073] Those skilled in the art will appreciate that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, or there may be intermediate elements. In addition, the "connection" or "connection" used herein may include wireless connection or wireless fusion. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0074] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present application thorough and complete, and to fully convey the concepts of these exemplary embodiments to those of ordinary skill in the art.
[0075] In this embodiment, a method for verifying the authenticity of a store is provided. Figure 1 As shown, the method includes:
[0076] Step 101, obtaining a target image of a store and first store information;
[0077] Among them, the first store information is the store related information submitted by the user, such as store name, address, contact information, etc.
[0078] The authenticity verification method of the store provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the intent recognition method, etc., but is not limited to the above forms.
[0079] Step 102, performing detection processing of image detection items on the target image to obtain detection results of the image detection items;
[0080] The image detection items include at least one of the following: store front detection, moiré detection, screenshot detection or watermark detection. Store front detection is used to detect whether the target image is a store front image containing store information. Moiré detection is used to detect whether the target image contains moiré that affects clarity, so as to determine whether the target image is an image obtained by shooting the device screen and whether it is clear. Screenshot detection is used to detect whether the target image is a device screenshot. Watermark detection is used to detect whether the target image contains a watermark, so as to determine whether the target image is an image obtained by reprinting.
[0081] In this embodiment, when it is necessary to verify the authenticity of the store information submitted by the user, the system obtains the target image of the store submitted by the user and the first store information related to the store. The system first verifies the authenticity of the target image from multiple aspects, such as whether it is a store front image, whether it is a real-life shot, whether it is a device screenshot, and whether it is a reproduced image, through at least one of the following detection items: store front detection, moiré detection, screenshot detection, and watermark detection. The target image is judged to be a real-life image of the store taken by the user through the detection results of the image detection item. Not only does the automated image detection save the workload of manual review, but it also combines multiple detection technologies to perform intelligent authenticity verification from multiple aspects, which can greatly improve the merchant verification accuracy.
[0082] In actual application scenarios, for shop door detection, the target image is detected and processed, specifically including: mirror-reversing the target image to obtain a flipped image; performing text recognition on the target image and the flipped image respectively to determine the confidence of the first text in the target image and the confidence of the second text in the flipped image; when the confidence of the first text is less than the confidence of the second text, replacing the target image with a mirror image; inputting the target image into the door detection model to obtain a detection result that the target image contains the shop door.
[0083] Among them, the door head detection model is trained based on the first sample image and the door head label, and the first sample image includes the store door head image. For example, the door head detection model is a model trained based on the YOLOv7 target detection algorithm. Specifically, a data set containing real scene image data, network door head photo image data and open source data is collected, and the input image is first uniformly scaled to a uniform size. Random cropping, flipping, rotation, translation, brightness, contrast, deduplication, mixup data enhancement and mosaic data enhancement are also used for data enhancement preprocessing. After preprocessing, the data set is manually annotated using an annotation tool, which includes images and corresponding annotations. The annotation specifies the location and category of the object in the image, and the annotation uses the YOLO format (class, x, y, w, h). The annotated data is divided into a training set, a validation set, and a test set according to a specified ratio to construct a training data set. YOLOv7 is used for training, and the training set image data and the corresponding annotations are input into the initial model. The model predicts the bounding box and category probability, calculates the loss based on the prediction and annotation, and back-propagates to update the weight of the model to minimize the loss until the set number of training iterations is reached. Try different learning rates, optimization algorithms, and other hyperparameters to ensure that the model converges effectively, and fine-tune and optimize the model. Then test the performance of the trained model on an independent validation dataset to avoid overfitting, and continuously optimize the model based on the validation results. After the model training is completed, use the test set to test the model performance, and determine whether further optimization is needed based on the map of the test results. If the map meets the requirements, the door head detection model can be output for use.
[0084] In this embodiment, by comparing the confidence of the text in the same text box in the target image and the flipped image, the text that is more consistent with the text semantics and word order under the two perspectives is determined to determine whether the target image is a mirror image. When the confidence of the first text is greater than or equal to the confidence of the second text, it means that the credibility of the first text of the target image is higher, then the target image submitted by the user can be determined to be a non-mirror image, and the subsequent door head detection can be directly performed. On the contrary, when the confidence of the first text is less than the confidence of the second text, it means that the credibility of the second text obtained after flipping is higher and more consistent with natural language, then the target image submitted by the user can be determined to be a mirror image. At this time, the target image is first covered with the mirror image, and the door head detection is performed through the updated target image. Thereby, the mirror image problem of the image is discovered in time, and the text recognition error caused by the image direction problem is eliminated by using the result of the mirror processing, and accurate regional information is provided for subsequent analysis and processing. Moreover, even if the user submits a mirror image, the verification system can correct it, which reduces the requirements for the user to submit the target image and facilitates user operation.
[0085] For example, taking the merchant network access verification scenario as an example, merchants generally use mobile phones as shooting devices for mirror shooting. In the process of using mobile phones to shoot, it is often the case that mirror images are taken due to the use of the front camera or some configurations. Since many texts may become non-existent characters or other characters after flipping, the recognized text and the actual meaning are seriously deviated. To this end, after reading the store image (origin_image) submitted by the merchant, a copy can be made and flipped to obtain a flipped image (flip_image). Based on the store image and the flipped image, text box detection, text detection box text recognition, and the confidence mean of all text box text recognition are performed in parallel. For example, the first text "electric vehicle" and "123456789" are recognized in the store image, and the confidences are 0.9991 and 0.9043 respectively, and the second text "Dongchuangdian" and "ertyufgvh" are recognized in the flipped image, and the confidences are 0.3077 and 0.2268 respectively. The statistical mean confidence of all first texts is 0.9517, and the mean confidence of all second texts is 0.26725. 3. Comparing the sizes of a and b, the mean confidence of the first text is greater than the mean confidence of the second text, indicating that the original is a store picture and is not flipped. The original picture is sent to the door detection model. Based on the information output by the door detection model, it is judged whether there is a door in the current picture. Specifically, if there is a door and the confidence is greater than the preset threshold, the current picture is considered to be a door photo, otherwise it is considered that the current picture is not a door photo, and the merchant-related information is reported for review by the business personnel. After verification, it is returned to the initiator and the processing is terminated.
[0086] For moiré detection, the target image is detected and processed, specifically including: obtaining a moiré image and a store template image; extracting moiré features contained in the moiré image; performing perspective transformation on the store template image based on the moiré features to form a second sample image and an edge map of the moiré features in the second sample image; training a moiré recognition model based on the second sample image and the edge map; inputting the target image into the moiré recognition model to obtain the probability that the target image contains moiré; and determining the detection result of the moiré detection based on the probability.
[0087] It should be noted that moiré is a common optical phenomenon in the fields of image processing, printing, textiles, photography, etc. Moiré is an interference pattern caused by the superposition of two or more periodic structures (such as pixel arrays, fences, lines, grids or ripples, etc.) due to their similar or slightly different frequencies. When these repeating patterns overlap and cannot be perfectly aligned, the human eye or sensor will perceive a new, usually irregular periodic stripe or ripple structure, which is called moiré.
[0088] Furthermore, the moiré image can be a moiré image on a pure color background, so as to facilitate the segmentation of the moiré. The color difference can be used to reduce the interference of the background on the moiré, and the segmentation algorithm can be used for further filtering, and finally the screening algorithm can be used for fine-grained moiré extraction.
[0089] In this embodiment, the store template image is perspectively transformed by the moiré features in the moiré image using the idea of positive overlay to generate a second sample image with moiré and store background, and an edge map of the moiré features in the second sample image is constructed. The second sample image is used as a sample, and the edge map is used as a note to train the moiré recognition model. The probability that the target image contains moiré can be determined by the moiré recognition model. When the probability is greater than the preset probability, it can be determined that the target image contains moiré, which is a non-real scene photo of the electronic screen. Therefore, on the basis of achieving accurate and automated moiré detection, actual camera shooting can be simulated, and the background and moiré features are retained to the maximum extent, so that the final synthesized second sample image not only achieves pixel-level correspondence with the moiré layer, but also approximates to the real store picture, and can quickly realize high-quality, large-scale, low-cost moiré training data preparation, thereby improving the training efficiency and difficulty of the moiré recognition model.
[0090] For example, for a given store template image B∈[0,255] m×n×3 And the original moiré feature layer M, use the pixel-by-pixel multiplication of the perspective transformation matrix of the store template image and the original moiré feature layer M to synthesize the second sample image I∈[0,255] m×n×3 and the corresponding moiré edge map D∈[0,255] m×n , so that the second sample image can retain the texture and color features of M. Using the second sample image I and the corresponding moiré edge map D as training data, a high-level encoder (High-Level Encoder), a low-level encoder (Low-Level Encoder) and a spatial encoder (SpatialEncoder) are used to fully capture the feature values of moiré at different granularities, different sizes and different positions. For the above three encoders, three corresponding loss functions are also proposed to effectively guide feature extraction and model generalization, and further improve the model training efficiency and moiré detection accuracy.
[0091] The calculation formula of I is as follows:
[0092]
[0093] Where Tra(·) represents the perspective transformation operation, t represents the perspective transformation matrix used to calculate the target mask M on the rectangular area of size [m, n], and t can be changed to stretch, squeeze, rotate, shear, and reflect according to experience to imitate different camera poses and camera shooting distances.
[0094] For screenshot detection, the target image is detected and processed, and the specific implementation method is as follows:
[0095] Method 1: read the image file format information associated with the target image, and determine the detection result of the screenshot detection according to the reading completion status of the image file format information.
[0096] The read completion status includes read success or read failure, and the image file format information (EXIF) includes: at least one of shooting time, device model, focal length parameter, and aperture parameter.
[0097] In this embodiment, if the image file format information associated with the target image is not successfully read, it means that the target image lacks these shooting-related metadata, or the information is incomplete, and the target image can be roughly determined to be a screenshot. On the contrary, if the target image contains EXIF data, it means that the target image must be a photographed image.
[0098] Method 2: Determine the detection result of the screenshot detection according to the size relationship between the resolution of the target image and the preset screenshot resolution.
[0099] The screenshot resolution in a mobile device is the device screen resolution. The preset screenshot resolution can be set to the device screen resolution. The device screen resolution is usually less than 720x1280.
[0100] In this embodiment, the resolution of the target image can be compared with the device screen resolution to determine whether the image is a screenshot. If the resolution of the target image is less than the preset screenshot resolution, the target image can be determined to be a screenshot.
[0101] Method three, using the least common multiple method to calculate the aspect ratio similarity between the aspect ratio of the target image and multiple preset aspect ratios, and determining the detection result of the screenshot detection based on the size relationship between the similarity and the preset similarity.
[0102] The preset aspect ratios of regular screenshots are basically between 2:1 and 21:9, namely 2:1, 19:9, 16:9, 19.5:9, 15.8:9, 21:9, etc.
[0103] In this embodiment, the aspect ratio of the target image is compared with common aspect ratios one by one, and the maximum similarity is taken as the aspect ratio similarity. If the aspect ratio similarity is greater than the preset similarity, it is considered that the aspect ratio of the target image is consistent with a certain common aspect ratio, and it is likely to be a screenshot.
[0104] Method 4: Input the target image into the interface element recognition model to obtain the interface element features in the target image, and determine the detection result of the screenshot detection based on the interface element features.
[0105] The interface element recognition model is trained based on the third sample image and the interface element label, and the interface element features include information prompt bar features, application identification features and screen boundary features.
[0106] In this embodiment, screenshots usually contain screen-specific content such as status bars, navigation bars, and application interface elements, which do not appear in ordinary photos. Sometimes, even after cropping, the edges may still retain subtle pixel traces or unnatural screen boundaries. The deep learning model can be used in advance to automatically learn and extract interface element features from the third sample image, thereby effectively capturing and modeling complex nonlinear feature relationships in the image. The interface element recognition model is then used to determine whether the target image contains interface element features. If so, the target image can be determined to be a screenshot.
[0107] Exemplarily, the convolutional neural network structure EfficientNet can be used as an interface element recognition model framework.
[0108] Furthermore, before inputting the target image into the interface element recognition model, the store authenticity verification method also includes: inputting the third sample image into the feature extraction model to obtain the global feature vector and local feature vector of the third sample image; determining the channel attention weight based on the global feature vector and the local feature vector; and updating the third sample image based on the channel attention weight.
[0109] Among them, the feature extraction model includes a global feature extraction channel and a local feature extraction channel.
[0110] In this embodiment, the global feature vector and the local feature vector of the third sample image are extracted based on the attention mechanism, so that the global feature vector can capture the information of the whole image, while the local feature vector can capture the details and local features in the image. Then the third sample image is updated by the channel attention weight obtained according to the global feature vector and the local feature vector. In this way, the contribution of different channels to the features can be dynamically adjusted, and only the most useful channels for the current image are retained. By dynamically weighting the feature map in the channel and spatial dimensions, the network's attention to key information is enhanced, thereby improving the performance and interpretability of the model, especially when processing unseen images. It has better performance. It can also reduce the calculation of useless feature channels, thereby reducing the calculation amount and storage requirements of the model.
[0111] Exemplarily, the original third sample image is subjected to GAP (Global Average Pooling) and GMP (Global Max Pooling) operations respectively, and the third sample image is compressed into a 1×1×C vector, where C is the number of channels, so as to capture the global and local features between channels. The global feature vector and local feature vector obtained above are passed through two independent fully connected layers to learn the correlation between channels, and the output of the fully connected layer is normalized by the Sigmoid function to obtain a channel attention weight between 0 and 1. The output of the Sigmoid function is applied to each channel of the original third sample image, and the feature maps processed by channel attention and spatial attention are multiplied to obtain a weighted third sample image that takes into account both the importance of the channel and the importance of the spatial position.
[0112] For watermark detection, the target image is subjected to detection processing, specifically including: inputting the target image into a watermark detection model to obtain a detection result that the target image contains a watermark.
[0113] The watermark detection model is trained based on the fourth sample image and the watermark label, and the fourth sample image includes the watermark image.
[0114] In this embodiment, the watermark detection model can be used to detect whether the target image contains a watermark. If the target image contains a watermark, it can be determined that the target image may have been tampered with or reproduced, thereby improving the reliability and credibility of the information.
[0115] Exemplarily, the watermark detection model includes a YOLOV7 detection model and a large language model. Internet pictures usually carry watermarks, so the authenticity of the current target picture can be judged based on whether the picture contains a watermark. The specific implementation process is basically the same as the door head photo detection principle. The YOLOV7 detection model is also used, but the training data is a large public watermark data set manually annotated, and the training process is not repeated. Furthermore, when the YOLOV7 model detects that there is a watermark in the picture and the confidence is less than the preset confidence, it means that the blurred part of the picture is not sure whether it is a watermark or there is no watermark in the picture, and the large language model will be used for detection. Utilizing the powerful semantic understanding and thinking ability of the large model, the content in the picture can be analyzed and the corresponding results can be given. Based on the large language model, high-precision detection effect is achieved.
[0116] It is worth mentioning that, considering the different requirements for the authenticity of store information in different scenarios, the image detection items that need to be performed can be matched according to different application scenarios. For example, strict control is required in scenarios such as merchant network access, merchant inspection, and customer risk control, so four tests can be performed: store door detection, moiré detection, screenshot detection, and watermark detection. In the customer management scenario, the purpose is to determine whether the merchant's information has changed, so only store door detection and moiré detection are required to ensure the accuracy of information recognition in the image.
[0117] Step 103, when the detection result of the image detection item meets the preset condition, identifying the second store information in the target image;
[0118] In this embodiment, when the detection result of the image detection item meets the preset conditions, that is, after the verification is passed, the system can determine that the target image is a real-life image of the store taken by the user. At this time, text recognition can be performed on the target image to obtain the second store information displayed by the target image. On the contrary, when the detection result of any image detection item does not meet the preset conditions, the authenticity verification result is determined to be failed. Thus, when the image is guaranteed to be authentic and reliable and the text can be accurately identified, the subsequent store information recognition function is triggered, which not only uses image detection to perform the first authenticity verification, optimizes the authenticity verification process, and improves the verification efficiency, but also can ensure the recognition accuracy of the store information.
[0119] Exemplarily, the door head part in the target image is segmented to obtain a picture of the door head area. The door head picture is recognized using the OCR algorithm, and the recognition results are spliced. The OCR recognition result is input into the UIE (Unified Information Extraction Framework) extraction model to obtain the merchant name. Among them, the UIE extraction model is implemented based on the PaddleNLP tool library, which supports automatic extraction of structured information from unstructured or semi-structured text. The main tasks include entity recognition, relationship extraction, event extraction, sentiment analysis, comment extraction and other tasks. The UIE extraction model can be pre-trained based on the PaddleNLP tool library and fine-tuned through real scene data to adapt to the current scene.
[0120] Step 104: Determine the authenticity verification result of the store based on the similarity between the first store information and the second store information.
[0121] The authenticity verification method of the store provided in the embodiment of the present application, when it is necessary to verify the authenticity of the store information submitted by the user, the system obtains the target image of the store submitted by the user and the first store information related to the store. The system first verifies the authenticity of the target image from multiple aspects such as whether it is a store door head image, whether it is a real scene shooting, whether it is a device screenshot, whether it is a reprinted image, etc. through at least one detection item in the store door detection, moiré detection, screenshot detection and watermark detection. After the detection result of the image detection item meets the preset conditions, that is, after the verification is passed, the system can determine that the target image is a real scene image of the store taken by the user. At this time, text recognition can be performed on the target image to obtain the second store information displayed by the target image. By comparing the second store information in the image with the first store information submitted by the user, if the comparison is passed, it can be determined that the user is a real merchant, and the subsequent process can be executed. Thus, through the automated store image verification and information verification process, a dual authentication mechanism for images and information is realized. On the one hand, manual intervention in the audit process is reduced, labor costs and time costs are reduced, and the authenticity verification efficiency of merchant stores is greatly improved. On the other hand, it combines a variety of detection technologies to perform intelligent authenticity verification from multiple aspects, greatly improving the merchant verification accuracy.
[0122] In a specific application scenario, step 104, that is, determining the authenticity verification result of the store based on the similarity between the first store information and the second store information, specifically includes: when the similarity between the first store information and the second store information is greater than the similarity threshold, determining that the authenticity verification result is passed; when the similarity between the first store information and the second store information is less than or equal to the similarity threshold, sending the first store information to the review node; in response to a confirmation instruction fed back by the review node, determining that the authenticity verification result is passed; when no confirmation instruction is received from the review node within a preset time period, determining that the authenticity verification result is failed.
[0123] Among them, the similarity threshold can be reasonably set according to the required verification accuracy, and the embodiments of the present application do not make specific limitations.
[0124] In this embodiment, if the similarity between the first store information and the second store information is greater than the similarity threshold, the authenticity verification result can be determined to be passed, indicating that the store information provided by the user and the store information identified by the image are highly similar, and the store information provided by the user may be true. On the contrary, if the similarity between the first store information and the second store information is small, the first store information is sent to the review node, and the staff corresponding to the review node can manually review and verify the first store information to ensure its authenticity. After the staff confirms that it is correct, the system can be informed that the first store information is true by issuing a confirmation instruction. If the staff has doubts about the first store information, a return instruction can be issued or no operation can be performed. In this way, after the system detects that no confirmation instruction feedback from the review node is received within a preset period of time, it determines that the authenticity verification result is not passed. Thereby, not only the automatic verification of the credibility of the store information is realized, the workload of manual review is reduced, but also the erroneous verification results caused by errors in automatic identification and judgment are avoided, and the rigor and credibility of the verification process are increased. In addition, manual review will be triggered only when the system determines that the similarity does not meet the requirements, which reduces the data transmission channels, thereby reducing the risk of data leakage and enhancing data security and privacy protection.
[0125] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0126] Furthermore, if Figure 2 As shown, as a specific implementation of the authenticity verification method of the above-mentioned store, an embodiment of the present application provides a store authenticity verification device 200, and the store authenticity verification device 200 includes: an acquisition module 201, an image verification module 202, an identification module 203 and an information verification module 204.
[0127] Wherein, the acquisition module 201 is used for the target image of the store and the first store information;
[0128] The image verification module 202 is used to perform detection processing of image detection items on the target image to obtain detection results of the image detection items, wherein the image detection items include at least one of the following: store front detection, moiré detection, screenshot detection or watermark detection;
[0129] The recognition module 203 is used to recognize the second store information in the target image when the detection result of the image detection item meets the preset condition;
[0130] The information verification module 204 is used to determine the authenticity verification result of the store based on the similarity between the first store information and the second store information.
[0131] Furthermore, the image detection project includes store front detection, and the store authenticity verification device 200 also includes: an image flipping module (not shown in the figure), which is used to perform mirror-reversal processing on the target image to obtain a flipped image; a recognition module 203, which is also used to perform text recognition processing on the target image and the flipped image respectively, and determine the confidence of the first text in the target image and the confidence of the second text in the flipped image; an image verification module 202, which is specifically used to replace the target image with a mirror image when the confidence of the first text is less than the confidence of the second text; and, input the target image into a door head detection model to obtain a detection result that the target image contains a store front, wherein the door head detection model is trained based on a first sample image and a door head label, and the first sample image includes a store front image.
[0132] Furthermore, the image detection project includes moiré detection, and the authenticity verification device 200 of the store also includes: a first feature processing module (not shown in the figure), the first feature processing module is used to extract the moiré features contained in the moiré image; and, according to the moiré features, the store template image is perspective transformed to form a second sample image and an edge map of the moiré features in the second sample image; a training module (not shown in the figure), the training module is used to train the moiré recognition model based on the second sample image and the edge map; an image verification module 202, specifically used to obtain the moiré image and the store template image; input the target image into the moiré recognition model to obtain the probability that the target image contains moiré; and, determine the detection result of the moiré detection based on the probability.
[0133] Furthermore, the image detection project includes screenshot detection, and the image verification module 202 is specifically used to read the image file format information associated with the target image, and determine the detection result of the screenshot detection based on the reading completion status of the image file format information, wherein the reading completion status includes reading success or reading failure, and the image file format information includes: shooting time, device model, focal length parameter, aperture parameter, at least one of.
[0134] Furthermore, the image detection project includes screenshot detection, and the image verification module 202 is specifically used to determine the detection result of the screenshot detection according to the size relationship between the resolution of the target image and the preset screenshot resolution.
[0135] Furthermore, the image detection project includes screenshot detection, and the image verification module 202 is specifically used to calculate the aspect ratio similarity between the aspect ratio of the target image and multiple preset aspect ratios using the least common multiple method, and determine the detection result of the screenshot detection based on the size relationship between the similarity and the preset similarity.
[0136] Furthermore, the image detection project includes screenshot detection, and the image verification module 202 is specifically used to input the target image into the interface element recognition model, obtain the interface element features in the target image, and determine the detection result of the screenshot detection based on the interface element features, wherein the interface element recognition model is trained based on the third sample image and the interface element label, and the interface element features include information prompt bar features, application identification features and screen boundary features.
[0137] Furthermore, the authenticity verification device 200 of the store also includes: a second feature processing module (not shown in the figure), the second feature processing module is used to input the third sample image into the feature extraction model to obtain the global feature vector and local feature vector of the third sample image, wherein the feature extraction model includes a global feature extraction channel and a local feature extraction channel; and, based on the global feature vector and the local feature vector, determining the channel attention weight; an updating module (not shown in the figure), the updating module is used to update the third sample image based on the channel attention weight.
[0138] Furthermore, the image detection project includes watermark detection, and the image verification module 202 is specifically used to input the target image into the watermark detection model to obtain a detection result that the target image contains a watermark, wherein the watermark detection model is trained based on the fourth sample image and the watermark label, and the fourth sample image includes the watermark image.
[0139] Furthermore, the information verification module 204 is specifically used to determine that the authenticity verification result is passed when the similarity between the first store information and the second store information is greater than a similarity threshold; send the first store information to the review node when the similarity between the first store information and the second store information is less than or equal to the similarity threshold; determine that the authenticity verification result is passed in response to a confirmation instruction fed back by the review node; and determine that the authenticity verification result is failed when no confirmation instruction is received from the review node within a preset time period; the image verification module 202 is also used to determine that the authenticity verification result is failed when the detection result of any image detection item does not meet the preset conditions.
[0140] For the specific definition of the authenticity verification device of the store, please refer to the definition of the authenticity verification method of the store above, which will not be repeated here. Each module in the authenticity verification device of the above store can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0141] Based on the above Figure 1The method shown in the embodiment of the present application accordingly provides a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned Figure 1 The authenticity verification method of the store shown.
[0142] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.
[0143] Based on the above Figure 1 The method shown, and Figure 2 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The authenticity verification method of the store shown.
[0144] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.
[0145] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.
[0146] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.
[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware platforms, or by hardware to obtain the target image and the first store information of the store; perform detection processing of the image detection project on the target image to obtain the detection result of the image detection project, wherein the image detection project includes at least one of the following: store door detection, moiré detection, screenshot detection or watermark detection; if the detection result of the image detection project meets the preset conditions, identify the second store information in the target image; determine the authenticity verification result of the store based on the similarity between the first store information and the second store information. In the embodiment of the present application, when it is necessary to verify the authenticity of the store information submitted by the user, the system obtains the target image of the store submitted by the user and the first store information related to the store. The system first verifies the authenticity of the target image from multiple aspects such as whether it is a store door image, whether it is a real scene shooting, whether it is a device screenshot, and whether it is a reprinted image through at least one detection project of store door detection, moiré detection, screenshot detection and watermark detection. When the detection result of the image detection project meets the preset conditions, that is, after the verification is passed, the system can determine that the target image is a real scene image of the store taken by the user. At this time, text recognition can be performed on the target image to obtain the second store information displayed in the target image. By comparing the second store information in the image with the first store information submitted by the user, if the comparison is successful, it can be determined that the user is a real merchant and the subsequent process can be executed. Thus, through the automated store image verification and information verification process, a dual authentication mechanism for images and information is realized. On the one hand, it reduces manual intervention in the audit process, reduces labor costs and time costs, and greatly improves the efficiency of authenticity verification of merchant stores. On the other hand, it combines a variety of detection technologies to perform intelligent authenticity verification from multiple aspects, which greatly improves the accuracy of merchant verification.
[0148] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.
[0149] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.
Claims
1. A method for verifying the authenticity of a store, characterized in that: The method comprises: Get the target image of the store and the first store information; Performing detection processing of image detection items on the target image to obtain detection results of the image detection items, wherein the image detection items include at least one of the following: store front detection, moiré detection, screenshot detection, or watermark detection; When the detection result of the image detection item meets the preset condition, identifying the second store information in the target image; The authenticity verification result of the store is determined based on the similarity between the first store information and the second store information.
2. The authenticity verification method of a store according to claim 1, characterized in that: The image detection project includes store front detection, and the detection processing of the target image for the image detection project specifically includes: Performing mirror-inversion processing on the target image to obtain a flipped image; Performing text recognition processing on the target image and the flipped image respectively, and determining the confidence of the first text in the target image and the confidence of the second text in the flipped image; When the confidence level of the first text is less than the confidence level of the second text, replacing the target image with the mirror image; The target image is input into a storefront detection model to obtain a detection result that the target image includes a storefront, wherein the storefront detection model is trained based on a first sample image and a storefront label, and the first sample image includes a storefront image.
3. The authenticity verification method of a store according to claim 1, characterized in that: The image detection item includes moiré detection, and the detection processing of the image detection item on the target image specifically includes: Obtain a moiré image and a store template image; Extracting moiré features contained in the moiré image; Performing perspective transformation processing on the store template image according to the moiré feature to form a second sample image and an edge map of the moiré feature in the second sample image; training a moiré recognition model according to the second sample image and the edge map; Inputting the target image into a moiré recognition model to obtain a probability that the target image contains moiré; A detection result of the moiré detection is determined according to the probability.
4. The authenticity verification method of a store according to claim 1, characterized in that: The image detection project includes screenshot detection, and the detection process of the target image for the image detection project specifically includes: Reading the image file format information associated with the target image, and determining the detection result of the screenshot detection according to the reading completion status of the image file format information, wherein the reading completion status includes reading success or reading failure, and the image file format information includes: shooting time, device model, focal length parameter, aperture parameter at least one; and / or, Determining the detection result of the screenshot detection according to the size relationship between the resolution of the target image and the preset screenshot resolution; and / or, Calculating the aspect ratio similarity between the aspect ratio of the target image and a plurality of preset aspect ratios using the least common multiple method, and determining the detection result of the screenshot detection according to the size relationship between the similarity and the preset similarities; and / or, The target image is input into an interface element recognition model to obtain interface element features in the target image, and based on the interface element features, a detection result of the screenshot detection is determined, wherein the interface element recognition model is trained based on a third sample image and interface element labels, and the interface element features include information prompt bar features, application identification features, and screen boundary features.
5. The authenticity verification method of a store according to claim 4, characterized in that: Before inputting the target image into the interface element recognition model, the method further includes: Inputting the third sample image into a feature extraction model to obtain a global feature vector and a local feature vector of the third sample image, wherein the feature extraction model includes a global feature extraction channel and a local feature extraction channel; Determining a channel attention weight according to the global feature vector and the local feature vector; Update the third sample image according to the channel attention weight.
6. The authenticity verification method of a store according to claim 1, characterized in that: The image detection project includes watermark detection, and the detection process of the image detection project on the target image specifically includes: The target image is input into a watermark detection model to obtain a detection result that the target image contains a watermark, wherein the watermark detection model is trained based on a fourth sample image and a watermark label, and the fourth sample image includes a watermark image.
7. The authenticity verification method of a store according to any one of claims 1 to 6, characterized in that: Determining the authenticity verification result of the store based on the similarity between the first store name and the second store name specifically includes: When the similarity between the first store information and the second store information is greater than a similarity threshold, determining that the authenticity verification result is passed; When the similarity between the first store information and the second store information is less than or equal to a similarity threshold, sending the first store information to a review node; In response to a confirmation instruction fed back by the review node, determining that the authenticity verification result is passed; If no confirmation instruction is received from the verification node within a preset period of time, determining that the authenticity verification result is failed; The method further comprises: When the detection result of any of the image detection items does not meet the preset condition, the authenticity verification result is determined to be failed.
8. A store authenticity verification device, characterized in that: The device comprises: An acquisition module, used to acquire a target image of a store and first store information; An image verification module, used to perform detection processing of image detection items on the target image to obtain detection results of the image detection items, wherein the image detection items include at least one of the following: store front detection, moiré detection, screenshot detection or watermark detection; A recognition module, configured to recognize second store information in the target image when a detection result of the image detection item meets a preset condition; The information verification module is used to determine the authenticity verification result of the store based on the similarity between the first store information and the second store information.
9. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the authenticity verification method for a store as described in any one of claims 1 to 7 are implemented.
10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the program, the authenticity verification method of a store according to any one of claims 1 to 7 is implemented.
