Methods and non-transitory computer readable media for counterfeit product detection

Abstract

A method, system, and non-transient computer-readable medium for accurately and efficiently identifying digital images depicting counterfeit products are disclosed. The disclosed system can store key values ​​and attributes of authentic graphic elements (logos, trademarks, designs, etc.). For example, the disclosed system can determine authentic graphic features, including authentic shape features, authentic color features, and authentic text features corresponding to the graphic elements. The disclosed system can also extract graphic elements from the digital image for comparison with authentic graphic elements. For example, the disclosed system can extract graphic features of the graphic elements, including shape features, color features, and text features derived from the graphic elements. The disclosed system can also determine whether the digital image depicts a counterfeit product or a genuine product based on a comparison of authentic graphic features with the extracted graphic features.

Description

Background Technology

[0001] In recent years, hardware and software platforms for buying and selling products electronically have improved significantly. For example, developers have created technologies to modify or improve e-commerce platforms to provide information about products and facilitate product sales. To illustrate this, in addition to displaying product details, including price, specifications, offers, and other information, traditional e-commerce systems typically display product images to assist buyers. For instance, traditional e-commerce systems often upload product images taken and submitted by the seller. Unfortunately, while e-commerce systems have become more popular and allow for wider reach and easier product sales, they have also expanded the market for counterfeit products. In fact, it is common for counterfeit products to be sold online as genuine. Despite the availability of product images, detecting counterfeit products is not a simple or easy task. Summary of the Invention

[0002] One or more embodiments of systems, methods, and non-transient computer-readable storage media that accurately and efficiently identify counterfeit products in digital images based on various features within the digital image provide benefits and / or solve one or more problems in the art. Typically, the disclosed system stores a key value calculated based on the shape and other attributes of an authentic logo. For illustration, the disclosed system extracts a logo image including the identified logo from a digital image. The disclosed system generates an edge map of the authentic logo in the logo image to define the shape representation of the authentic logo. The disclosed system also registers authentic color shading and gradients at each portion of the authentic logo. Furthermore, the disclosed system registers valid text styles and other attributes for the text used in the logo image. The disclosed system similarly extracts attributes from the logo within an uploaded image. The disclosed system then determines whether the uploaded image depicts a genuine or counterfeit product by comparing features extracted from the authentic logo with features extracted from the logo within the uploaded image.

[0003] Additional features and advantages of one or more embodiments of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing such exemplary embodiments. Attached Figure Description

[0004] Various embodiments will be described and explained in detail below with the help of the accompanying drawings.

[0005] Figure 1 An environment in which a counterfeit identification system operates according to one or more embodiments of the present disclosure is shown.

[0006] Figure 2An overview diagram of a digital image depicting a forged object is shown according to one or more embodiments of the present disclosure.

[0007] Figure 3 An example region suggestion neural network for extracting graphical elements is shown according to one or more embodiments of the present disclosure.

[0008] Figure 4 An overview of generating shape features for graphic elements according to one or more embodiments of the present disclosure is shown.

[0009] Figure 5 An overview of generating color features for graphic elements according to one or more embodiments of the present disclosure is shown.

[0010] Figure 6 An overview of generating text features for graphical elements according to one or more embodiments of the present disclosure is shown.

[0011] Figures 7A to 7B An overview of comparing real graphic features with graphic features according to one or more embodiments of the present disclosure is shown.

[0012] Figures 8A to 8B The present disclosure illustrates the learning of parameters for a forgery detection model and the application of the forgery detection model according to one or more embodiments of the present disclosure.

[0013] Figure 9 A schematic diagram of an example architecture of a counterfeit identification system according to one or more embodiments of the present disclosure is shown.

[0014] Figure 10 A series of actions for determining a digital image depicting a counterfeit product, according to one or more embodiments of the present disclosure, are illustrated.

[0015] Figure 11 A block diagram of an example computing device according to one or more embodiments of the present disclosure is shown. Detailed Implementation

[0016] One or more embodiments of this disclosure include a counterfeit identification system that automatically determines whether a digital image depicts a counterfeit product by extracting and comparing authentic graphic features with graphic features from a digital image. Specifically, the counterfeit identification system locates a genuine logo image from a genuine digital image. The counterfeit identification system extracts authentic logo features from the authentic logo image. Authentic graphic features reflect attributes of the authentic logo image, including authentic shape features, authentic color features, and authentic text features. The counterfeit identification system also locates and extracts the logo image from a digital image depicting a product. The counterfeit identification system also extracts graphic features of graphic elements, including shape features, color features, and text features. The counterfeit identification system verifies graphic features by comparing the graphic features with authentic graphic features.

[0017] For illustration, in some embodiments, the forgery identification system extracts genuine graphic elements from a genuine digital image. The forgery identification system determines genuine graphic features for the genuine graphic elements. For example, the forgery identification system determines genuine graphic features, including one or more features such as genuine shape features, genuine color features, or genuine text features. In some embodiments, the forgery identification system extracts graphic elements from the digital image to be analyzed. The forgery identification system also generates graphic features of the graphic elements. For example, similar to genuine graphic features, the forgery identification system generates graphic features, including one or more features such as shape features, color features, or text features. In some embodiments, the forgery identification system determines that the digital image depicts a forged product based on a comparison of graphic features and genuine graphic features.

[0018] As described above, in some embodiments, the forgery identification system extracts authentic graphic elements, such as logos, from authentic digital images depicting genuine products. Similarly, the forgery identification system also extracts graphic elements from digital images depicting the product to be analyzed. In some embodiments, the forgery identification system locates and extracts authentic graphic elements and graphic features by utilizing a convolutional neural network trained using digital images of various graphic elements. Specifically, the forgery identification system generates bounding boxes, including graphic element boundaries, mask regions, and confidence scores indicating the probability that the bounding box includes a graphic element.

[0019] The forgery identification system also determines authentic graphic features for genuine graphic elements. Typically, authentic graphic features include key attributes of the genuine graphic element. Authentic graphic features include authentic shape features, authentic color features, authentic text features, etc. In one or more implementations, the forgery identification system determines authentic shape features by calculating the edge map of the genuine graphic element. In some embodiments, the forgery identification system generates authentic color features by mapping pixels from the genuine graphic element to a color space to define a range of acceptable color values. Furthermore, the forgery identification system generates authentic text features by determining various text attributes such as position, style, distortion, and color.

[0020] The forgery identification system also generates graphic features for graphic elements. Typically, the forgery identification system identifies the attributes of graphic elements to be compared with genuine graphic elements. In some embodiments, the forgery identification system generates graphic features that mirror genuine graphic features. For illustration, the forgery identification system generates shape features, color features, and text features of the graphic elements. In one or more implementations, the forgery identification system uses the same methods and processes used to generate genuine graphic features to generate the graphic features.

[0021] The counterfeit identification system also analyzes graphic features and authentic graphic features to determine whether a digital image depicts a counterfeit product. In some embodiments, the counterfeit identification system compares graphic features with authentic graphic features to determine whether a digital image depicts a counterfeit product. In one example, the counterfeit identification system determines that a digital image depicts a counterfeit product based on the determination that any of a shape feature, color feature, or text feature falls outside an acceptable threshold range for authentic graphic features. For example, the counterfeit identification system determines that a digital image depicts a counterfeit product based on the determination that the shape of a graphic element differs from the shape of an authentic graphic element.

[0022] In some embodiments, the counterfeit identification system utilizes a counterfeit detection model to intelligently analyze graphic features and genuine graphic features. Specifically, the counterfeit identification system trains the counterfeit detection model using training images depicting both counterfeit and genuine products. The counterfeit identification system also trains the counterfeit detection model to generate one or more similarity confidence scores indicating the similarity between graphic elements and genuine graphic elements. The counterfeit identification system utilizes the counterfeit detection model to analyze the graphic features of graphic elements and generate similarity confidence scores. Based on determining that at least one confidence score corresponding to a digital image is below a threshold similarity value, the counterfeit identification system determines that the digital image depicts a counterfeit product.

[0023] Compared to traditional systems, counterfeit identification systems offer several technological advantages. Specifically, traditional e-commerce systems are often inaccurate and inefficient. Specifically, most products displayed and sold through traditional e-commerce systems are uploaded by third-party sellers unrelated to the product manufacturers or the e-commerce system itself. Some traditional systems attempt to identify and remove counterfeit products; however, most cannot do so accurately. For example, some traditional systems use templates or image matching techniques to label images as counterfeit. Other traditional e-commerce systems analyze the text descriptions of products. However, many counterfeit products often mimic genuine products or corresponding text so closely that traditional systems frequently mislabel products as either counterfeit or genuine.

[0024] Partly due to inaccuracies, traditional systems often inefficiently utilize computing, storage, and communication resources. Traditional e-commerce systems expend computing resources to display and process transactions involving counterfeit products. Traditional systems often cannot identify a product as counterfeit until the buyer receives it and reports it. Therefore, traditional systems often expend additional communication resources to receive notifications of counterfeit products. However, additional computing power must be used to remove counterfeit products.

[0025] Even traditional systems that attempt to identify counterfeit products before release are often inefficient. For example, as mentioned above, some traditional systems rely on template or image matching to identify counterfeit products. Template matching often requires traditional systems to process large amounts of image data before identifying the target image or template.

[0026] Compared to traditional systems, counterfeit product identification systems improve accuracy. Counterfeit identification systems identify authentic graphic features that reflect a variety of aesthetics and other visual attributes of genuine graphic elements. Unlike traditional systems that frequently rely on rigid and unrepresentative characteristics, counterfeit identification systems evaluate a wide range of attributes of authentic graphic features, including authentic shape features, authentic color features, and authentic text features. Counterfeit identification systems also compare authentic graphic features with corresponding graphic features for graphic elements from digital images to assess whether the digital image depicts a counterfeit or genuine product.

[0027] Compared to traditional e-commerce systems, counterfeit identification systems also improve efficiency. They effectively identify counterfeit digital images depicting counterfeit products by analyzing digital images. Therefore, instead of expending computational resources to render digital images depicting counterfeit products and identifying them based on buyer feedback, the system automatically tags counterfeit products before they are published.

[0028] Furthermore, forgery identification systems improve efficiency by extracting and selectively analyzing authentic graphic elements. In one or more implementations, instead of analyzing the entire digital image, the forgery identification system uses machine learning models to intelligently identify portions of the digital image and real images that contain authentic graphic elements or graphic elements. The forgery identification system also performs analysis on the identified authentic graphic elements or graphic elements. Therefore, the forgery identification system reduces the amount of computational resources required to generate authentic graphic features and graphic characteristics.

[0029] As discussed above, this disclosure uses various terms to describe the features and advantages of the disclosed methods. Additional details regarding the meaning of these terms are now provided. For example, as used herein, the term "realistic graphic element" refers to a unique visual item. Specifically, a realistic graphic element is a symbol or other design used by an organization to identify its products. For example, realistic graphic elements include logos, symbols, designs, or trademarks specific to a given organization. Realistic graphic elements are often attached to products depicted within realistic digital images.

[0030] Accordingly, as used herein, the term "graphic element" refers to a visual item attached to a product within a digital image. Specifically, graphic elements include logos, symbols, designs, or trademarks that resemble genuine graphic elements. For example, graphic elements are often attached to a product depicted within a digital image. Counterfeit graphic elements are designed to mimic genuine graphic elements.

[0031] As used herein, the term "authentic digital image" refers to an electronic representation of an authentic subject, such as through a set of pixels. Specifically, the term "authentic digital image" refers to a digital image depicting an authentic product from a particular organization. Authentic digital images include any suitable format such as bitmap or JPEG file formats. Authentic digital images include product images depicting branded products originating from a branding organization. More specifically, authentic digital images depict authentic products identified by authentic graphic elements.

[0032] As used herein, the term "digital image" refers to an electronic representation of a counterfeit or genuine subject matter. Specifically, a digital image may include a digital image depicting a counterfeit product that resembles a genuine product from a given organization. Alternatively, a digital image may depict a genuine product. For example, a digital image may be uploaded by a user (e.g., a third-party seller) to an online content management system to indicate that the user is selling or considering purchasing an item. Relatedly, the term "counterfeit digital image" refers to a digital image depicting a counterfeit product.

[0033] As used herein, the term "realistic graphic feature" refers to the characteristics or attributes of a real graphic element. Specifically, realistic graphic features include a variety of unique characteristics that reflect the attributes of a real graphic element. For example, realistic graphic features include key values ​​calculated based on the shape, color, text, or other components of a real graphic element.

[0034] Accordingly, as used herein, the term "graphic feature" refers to the characteristics or attributes of graphic elements within a digital image. Specifically, graphic features include a variety of attributes of graphic elements. For example, graphic features include values ​​calculated based on the shape, color, text, or other components of a graphic element. In one or more implementations, graphic features may resemble, but are not identical to, true graphic features.

[0035] As used herein, the term "realistic shape feature" refers to an attribute relating to the shape of a real graphic element. Specifically, a realistic shape feature includes a shape representation indicating the location of the edges of a real graphic element. For example, in some embodiments, a realistic shape feature includes a shape vector / list containing points indicating the pixel locations of the edges of a graphic element. Relatedly, the term "shape feature" refers to a shape representation indicating the location of the edges of a graphic element within a digital image.

[0036] As used herein, the term "true color feature" refers to the unique color attributes of a true graphic element. Specifically, true color features include color values ​​within a true graphic element and their corresponding pixel locations. True color features include color values ​​from the true graphic element for each pixel location in its tonal saturation (HSV), tonal saturation light (HSL), red-blue-green (RBG), or other color spaces. Relatedly, the term "color feature" refers to the color attributes of a graphic element. For example, color features also include color values ​​from the graphic element for each pixel location in its HSV, HSL, RBG, or other color spaces.

[0037] As used herein, the term "realistic text features" refers to the unique textual attributes of a real graphic element. Specifically, realistic text features reflect various textual attributes of text found within a real graphic element. For example, text features indicate text position, text style, text warp, text color, and other attributes of text within a real graphic element. Relatedly, the term "text feature" refers to the textual attributes of text found within a graphic element. Text features include one or more of text position, text style, text warp, text color, or other attributes of text within a graphic element.

[0038] As used herein, the term "counterfeit product" refers to a forgery or unauthorized copy of a genuine product. Specifically, a counterfeit product is a good that is manufactured or sold under a given brand name without the authorization of the brand owner. For example, counterfeit products include goods with graphic elements designed to mimic genuine graphic elements.

[0039] The following disclosure provides additional details regarding a counterfeit identification system, including illustrative drawings depicting exemplary embodiments and implementations of the system. For example, Figure 1 A schematic diagram of a system environment (or "environment") 100 in which a forgery identification system 106 operates according to one or more embodiments is shown. As shown, environment 100 includes one or more server devices 102 connected to user client device 108 via network 112. Although Figure 1 An embodiment of the counterfeit identification system 106 is shown, but alternative embodiments and configurations are also possible.

[0040] like Figure 1 As shown, server devices 102 and user client devices 108 are connected via network 112. As illustrated, in one or more implementations, each component of environment 100 communicates via network 112. Network 112 includes a suitable network on which computing devices can communicate. (See below for reference.) Figure 11 The example network will be discussed in more detail.

[0041] As shown in the figure, environment 100 includes server devices 102. The server devices 102 generate, store, receive, and / or transmit digital content, including digital video, digital images, digital audio, metadata, etc. Specifically, in one or more implementations, the server devices 102 provide digital content to devices such as user client devices 108 via web pages or local applications. The server devices 102 are capable of communicating with user client devices 108 via network 112. For example, the server devices 102 collect and / or receive digital images from user client devices 108, including real digital images and / or digital images. The server devices 102 can also present digital images and real digital images at user client devices 108. In some embodiments, the server devices 102 include distributed servers, wherein the server devices 102 include multiple server devices distributed across network 112 and located in different physical locations. The server devices 102 may optionally include content servers, application servers, communication servers, web hosting servers, or digital content management servers.

[0042] like Figure 1As further shown, server devices 102 include an online content management system 104. In one or more embodiments, the online content management system 104 includes an e-commerce management system that facilitates the online purchase of products via network 112. The online content management system 104 also performs various backend functions associated with the online presence of sellers to facilitate the online purchase of products. In some embodiments, the online content management system 104 verifies the authenticity of products submitted to it. For example, the online content management system 104 identifies digital images depicting counterfeit products. Additionally, the online content management system 104 may optionally perform other backend functions associated with the online presence of sellers. For example, the online content management system 104 generates web pages or other types of web content that are provided to user client devices 108, offering options for purchase, rental, download, leasing, or other forms of consumption as described.

[0043] like Figure 1 As shown, the counterfeit identification system 106 is implemented as part of the online content management system 104. Typically, the counterfeit identification system 106 intelligently identifies digital images depicting counterfeit products by comparing graphic features in a digital image with real graphic features. Specifically, the counterfeit identification system 106 extracts real graphic elements from a real digital image and determines real graphic features from these real graphic elements, including real shape features, real color features, and real text features. The counterfeit identification system 106 also extracts graphic elements from the digital image and generates graphic features for these graphic elements. The counterfeit identification system 106 further determines whether a digital image depicts a counterfeit product based on comparing the graphic features with real graphic features.

[0044] like Figure 1 As shown, environment 100 includes user client device 108. User client device 108 is capable of generating, storing, receiving, and transmitting digital data. For example, user client device 108 communicates with server device(s) 102 via network 112. Figure 1 The user client device 108 shown may include various types of client devices. For example, in some embodiments, the user client device 108 is a mobile device, such as a laptop computer, tablet computer, mobile phone, smartphone, etc. In other embodiments, the user client device 108 includes non-mobile devices such as desktop computers or servers, or other types of client devices. Reference will be made below. Figure 11 Further details regarding the computing device are discussed; the user client device 108 is an implementation of the computing device.

[0045] User client device 108 may optionally be associated with a user or user account of an e-commerce platform managed by online content management system 104. For example, user client device 108 may be associated with a consumer of a product. Furthermore, user client device 108 may optionally be associated with a user browsing and viewing products listed by online content management system 104. As described above, user client device 108 communicates with server devices(s) 102. Specifically, user client device 108 uploads and sends digital data, including digital images (e.g., user-submitted images), to server devices(s) 102 via network 112. Additionally, user client device 108 displays a graphical user interface including product images to the user associated with user client device 108.

[0046] Additionally or alternatively, user client device 108 (or another client device) is associated with a seller or marketer of the product. For example, user client device 108 sends information about a product for sale to server devices 102, including digital images depicting the product. In some embodiments, user client device 108 sends authentic digital images depicting genuine products to server devices 102. In some examples, user client device 108 is the associated authentic seller or vendor. The authentic seller sends authentic graphic features to device 102 via user client device 108, and the counterfeit identification system 106 uses these authentic graphic features to evaluate digital images that may contain counterfeit products.

[0047] like Figure 1 As shown, user client device 108 includes application 110. Application 110 can be a web application or a native application (e.g., a mobile application, desktop application, etc.) on user client device 108. Application 110 interfaces with counterfeit identification system 106 to provide digital content, including product information such as digital images, to devices (multiple) 102. In one or more implementations, application 110 is a browser that draws a graphical user interface on the display of user client device 108. For example, application 110 draws a series of graphical user interfaces for uploading product information and managing the association between product information and promotional content. Additionally, application 110 may optionally present a simulation of a webpage from the perspective of accessing the webpage from a customer's client device. Simulating the webpage to preview content about the product allows sellers to review product information. Furthermore, in some embodiments, application 110 draws indications as to whether a product has been marked as counterfeit or genuine.

[0048] although Figure 1A forgery identification system 106 is shown located on device(s) 102, but in some embodiments, the forgery identification system 106 is implemented via (e.g., wholly or partially located) one or more other components of environment 100. For example, the forgery identification system 106 may be implemented wholly or partially on user client device 108. For example, device(s) 102 and / or user client device 108 have digital images or real digital images stored thereon.

[0049] Although environment 100 includes a single user client device 108, in one or more embodiments, environment 100 includes multiple user client devices and client devices. For example, environment 100 includes a first user client device 108 associated with a buyer viewing a webpage displaying product information for a product. Environment 100 may also optionally include a second user client device 108 associated with a seller or distributor who has uploaded product information including digital images and / or physical digital images.

[0050] Furthermore, user client device 108 may optionally communicate directly with counterfeit product identification system 106, bypassing network 112. Additionally, counterfeit identification system 106 may have access to one or more databases (e.g., digital image databases) hosted on server device(s) 102 or elsewhere in environment 100. Furthermore, counterfeit identification system 106 may optionally include one or more machine learning models (e.g., neural networks), and counterfeit identification system 106 may be implemented across server device(s) 102, network 112, and user client device 108 in various different ways.

[0051] Specifically, in some implementations, the counterfeit identification system 106 on server devices 102 supports applications on user client devices 108. For example, the counterfeit identification system 106 on server devices 102 generates or trains the counterfeit identification system 106. Server devices 102 provide the trained counterfeit identification system 106 to user client devices 108. In other words, user client devices 108 obtain (e.g., download) the counterfeit identification system 106 from server devices 102. At this point, user client devices 108 can use the counterfeit identification system 106 independently of server devices 102 to replace digital images used to detect counterfeit goods.

[0052] In an alternative embodiment, the forgery identification system 106 includes a web-hosted application that allows a user client device 108 to interact with content and services hosted on server devices 102(s). For illustration, in one or more implementations, the user client device 108 accesses web pages supported by server devices 102(s). The user client device 108 provides input to server devices 102(s) to perform forgery detection, and in response, the forgery identification system 106 on server devices 102(s) performs an operation. Server devices 102(s) then provide the output or result of the operation to the user client device 108.

[0053] although Figure 1 An example environment in which the counterfeit identification system 106 operates is shown, but according to one or more embodiments, the following figures and corresponding discussion provide additional details about how the counterfeit identification system 106 determines that a digital image depicts a counterfeit product. For example, Figure 2 A general overview of a counterfeit identification system 106 for determining digital images depicting counterfeit products, according to one or more embodiments, is shown. Specifically, Figure 2 The diagram illustrates a counterfeit identification system 106 that generates and compares authentic graphic features from a genuine digital image with graphic features from the digital image to determine if the digital image depicts a counterfeit product. More specifically, Figure 2 A series of actions 200 are shown, including actions 202 of extracting real graphic elements, actions 204 of determining real graphic features, actions 206 of extracting graphic elements, actions 208 of generating graphic features, and actions 210 of determining that the digital image depicts the counterfeit product.

[0054] like Figure 2 As shown, the series of actions 200 includes an action 202 of extracting real graphic elements. Specifically, action 202 includes extracting real graphic elements 216 from a real digital image 212. In some embodiments, as shown, the real digital image 212 includes a digital image depicting a real product (e.g., a t-shirt). In other embodiments, the real digital image 212 includes digital images of real graphic elements 216. For example, in some embodiments, the real digital image includes a real graphic element digital image 214.

[0055] The counterfeit identification system 106 receives a genuine digital image 212 and / or a genuine graphic element digital image 214 from a client device associated with the genuine seller. For example, the genuine seller possesses a genuine graphic element 216. Additionally or alternatively, the genuine seller is a third party authorized by the organization to use the genuine graphic element 216. The genuine seller typically has readily available genuine graphic element digital images (e.g., logo files), and the counterfeit identification system 106 receives and stores these genuine graphic element digital images in a repository of genuine graphic elements. In some embodiments, the counterfeit identification system 106 retrieves the genuine graphic element 216 by accessing the repository of genuine graphic elements.

[0056] When the forgery identification system 106 receives a real digital image 212 instead of a real graphic element digital image 214, the forgery identification system 106 extracts the real graphic element 216 from the real digital image 212. Specifically, the forgery identification system 106 uses a machine learning model to generate a bounding box, a mask region, and a confidence score that includes the boundaries of the real graphic element 216. Figure 3 The corresponding paragraphs describe how, according to one or more embodiments, the forgery identification system 106 uses a machine learning model to extract real graphic elements 216.

[0057] like Figure 2 As further shown, the series of actions 200 includes action 204 for determining real graphic features. Specifically, action 204 includes determining real graphic features for real graphic element 216, wherein real graphic features include real shape features 218, real color features 220, and real text features 222.

[0058] Furthermore, in some embodiments, the forgery identification system 106 performs action 204 by generating a realistic shape feature 218 based on the analysis of the real graphic element 216. Specifically, the forgery identification system 106 generates the realistic shape feature 218 by generating an edge map of the real graphic element 216 using a Canny edge filter together with adaptive hysteresis. Figure 4 The corresponding discussion provides additional details on how the counterfeit identification system 106 determines the true shape feature 218 according to one or more embodiments.

[0059] like Figure 2As further shown, the forgery identification system 106 generates authentic color features 220 based on the analysis of authentic graphic elements 216. Typically, the forgery identification system 106 analyzes pixels within the authentic graphic element 216 to determine the effective color shading and gradations of the authentic graphic element 216. For example, the forgery identification system 106 maps the color values ​​for each pixel within the authentic graphic element 216 to a color space such as the HSV color space. The forgery identification system 106 then uses the resulting values ​​as the authentic color feature 220. In some embodiments, the authentic color feature 220 includes a range of effective color values ​​for each pixel location. Specifically, the forgery identification system 106 considers color variations due to lighting conditions of the captured digital image by determining the range of effective color values. Figure 5 The corresponding paragraphs provide additional details on how the counterfeit identification system 106 determines the true color feature 220 according to one or more embodiments.

[0060] As execution Figure 2 In the action 204 shown, the forgery identification system 106 also analyzes the real graphic element 216 to generate authentic text features 222. Specifically, the forgery identification system 106 extracts the text and its style used within the real graphic element 216. For example, the forgery identification system 106 analyzes the attributes of the text, including the position, style, distortion, and color of the text within the real graphic element 216. Figure 6 The accompanying paragraphs provide additional details regarding how the forgery identification system 106 generates authentic text features 222, based on one or more embodiments.

[0061] Figure 2 The illustrated series of actions 200 also includes an action 206 for extracting graphic elements. Specifically, action 206 includes extracting graphic elements 228 from a digital image 226 depicting the product. The counterfeit identification system 106 may optionally utilize the same machine learning model used as part of action 202 for extracting real graphic elements to extract graphic element 228. For example, in some embodiments, the counterfeit identification system 106 utilizes a convolutional neural network (CNN)-based model trained using images of various graphic elements. According to one or more embodiments, Figure 3 The corresponding paragraphs provide additional details regarding how the forgery identification system 106 extracts graphic element 228.

[0062] like Figure 2As further shown, the series of actions 200 also includes an action 208 for generating graphic features. Specifically, the forgery identification system 106 generates graphic features for graphic element 228, wherein the graphic features include shape features 230, color features 232, and text features 234. In some embodiments, the forgery identification system 106 analyzes graphic element 228 using methods and processes similar to those used by the forgery identification system 106 to analyze genuine graphic element 216. For example, the forgery identification system 106 generates shape features 230 by utilizing a Canney edge detector and optionally adaptive hysteresis. The forgery identification system 106 also generates color features 232 by mapping color values ​​for each pixel within graphic element 228 to a color space. The forgery identification system 106 also generates text features 234 by analyzing the text attributes of text within graphic element 228, wherein text attributes include position, style, distortion, and color. Figures 3 to 6 The corresponding paragraphs provide additional details on how the forgery identification system 106 generates shape feature 230, color feature 232, and text feature 234 according to one or more embodiments.

[0063] Figure 2 The series of actions 200 shown also includes the action 210 of determining that the digital image depicts a counterfeit product. Specifically, the counterfeit identification system 106 compares genuine shape features with genuine shape features, genuine color features with genuine color features, and genuine text features with genuine text features. In some embodiments, the counterfeit identification system 106 determines that the digital image depicts a counterfeit product based on determining that at least one of the shape features, color features, or text features is different from genuine shape features, genuine color features, and genuine text features, respectively. Furthermore, in some embodiments, the counterfeit identification system 106 determines that the digital image depicts a counterfeit product based on any graphic feature being outside the range of genuine graphic features. For example, the counterfeit identification system 106 determines that the digital image depicts a counterfeit product based on determining that a color feature is outside the range of genuine color features.

[0064] Alternatively or additionally, the counterfeit identification system 106 performs action 210 by utilizing a counterfeit detection model. Specifically, the counterfeit identification system 106 may utilize the counterfeit detection model to analyze graphic features and real graphic features. Based on the analysis of graphic features and real graphic features, the counterfeit identification system 106 uses the counterfeit detection model to generate a similarity confidence score. Based on the similarity confidence score that falls below a threshold similarity value, the counterfeit identification system 106 determines that the digital image depicts a counterfeit product. Figures 8A to 8B A forgery identification system 106, which learns parameters for a forgery detection model and applies the forgery detection model according to one or more embodiments, is shown.

[0065] Figure 2The accompanying paragraphs describe an overview of a counterfeit identification system 106 for determining digital images depicting counterfeit products, according to one or more embodiments. The following figures and paragraphs further describe in detail how the counterfeit identification system 106 performs the functions described in the references. Figure 2 The described action. Specifically, Figure 3 This illustrates how the forgery identification system 106 utilizes a region proposal neural network 330 to locate and identify graphic elements 346 in a digital image 334.

[0066] As previously described, in some embodiments, the counterfeit identification system 106 utilizes a machine learning model to identify and locate graphic elements 346 within the digital image 334. The digital image 334 may represent a digital image received by the counterfeit identification system 106 from a user client device. The digital image 334 includes digital images depicting genuine or counterfeit products. For example, in some embodiments, the digital image 334 includes a digital image depicting a counterfeit product with counterfeit graphic elements 346. In another example, the digital image 334 includes a genuine digital image depicting a genuine product from which the counterfeit identification system 106 extracts the genuine graphic elements.

[0067] Figure 3 An implementation of a machine learning model that the forgery identification system 106 can utilize is shown. Specifically, Figure 3 A region proposal neural network 330 according to one or more implementations is illustrated. Typically, the region proposal neural network 330 can detect objects in an image. In one or more embodiments, the region proposal neural network 330 is a deep learning convolutional neural network (CNN). For example, in some embodiments, the region proposal neural network 330 is a region-based CNN (R-CNN) or an object detection neural network. For example, an example of a region proposal neural network can be found below: U.S. Patent Application Publication No. 2021 / 0027083, entitled Automatically Detecting User-Requested Objects In Images, the entire contents of which are incorporated herein by reference. Another example of a region proposal neural network can be found below: S. Ren, K. He, R. Girshick, and J. Sun, Faster R-CNN: Towards real- time object detection with region proposal networks (Faster R-CNN: Real-time Object Detection with Region Proposal Networks), NIPS, 2015, the entire contents of which are incorporated herein by reference.

[0068] Although Figure 3One implementation of a region proposal neural network is shown, but the forgery identification system 106 utilizes alternative machine learning models such as object detection neural networks or logo detection neural networks. For example, an object detection neural network is described in U.S. Patent Application No. 17 / 038,866, filed September 30, 2020, entitled "Generating Composite Images With Objects From Different Times," the entire contents of which are incorporated herein by reference. Similarly, a logo detection neural network known as a machine learning logo classifier is described in U.S. Patent No. 11,106,944, entitled "Selecting Logo Images Using Machine-Learning-Logo Classifiers," the entire contents of which are incorporated herein by reference.

[0069] like Figure 3 As shown, the region proposal neural network 330 includes a lower neural network layer 338 and a higher neural network layer 340. Typically, the lower neural network layer 338 collectively forms an encoder, and the higher neural network layer 340 collectively forms a decoder (or latent object detector). In one or more embodiments, the lower neural network layer 338 is a convolutional layer that encodes a digital image 334 into a feature vector, which is output from the lower neural network layer 338 and input to the higher neural network layer 340. In various implementations, the higher neural network layer 340 includes a fully connected layer that analyzes the feature vector and outputs graphical element proposals 342 (e.g., bounding boxes around latent objects) and a proposal confidence score 344.

[0070] Specifically, the lower neural network layer 338 includes convolutional layers that generate feature vectors in the form of feature maps. To generate graphical element proposals 342, the region proposal neural network 330 processes the feature maps using convolutional layers in the form of small networks that slide over small windows on the feature maps. The region proposal neural network 330 then maps each sliding window to a lower-dimensional feature. The region proposal neural network 330 then processes this feature using two separate heads of a fully connected layer. Specifically, the first head optionally includes a box regression layer and a box classification layer, the box regression layer generating graphical element proposals 342 and the box classification layer generating a proposal confidence score 344. As described above, for each region proposal, the region proposal neural network 330 generates a corresponding proposal confidence score 344.

[0071] Furthermore, in some embodiments, the graphical element proposal 342 includes a bounding box that includes the boundary of the graphical element 346 and a mask region. Typically, the mask region indicates the boundary of an object within a digital image. Specifically, the region proposal neural network 330 includes a mask R-CNN with additional branches for predicting a segmentation mask for each graphical element proposal in a pixel-to-pixel manner. Specifically, the mask R-CNN includes the region proposal neural network 330 to propose candidate graphical element bounding boxes and a binary mask classifier to generate a mask for each category. The forgery identification system 106 can utilize the region proposal neural network 330 to generate multiple regions of interest (RoIs). The forgery identification system 106 can then utilize an RoI alignment network to warp the RoIs to a fixed dimension. The forgery identification system 106 also feeds the warped features into fully connected layers for classification. The warped features are also fed into a binary mask classifier consisting of one or more CNNs to output a binary mask for each RoI.

[0072] The forgery identification system 106 may utilize a region proposal neural network 330 to generate multiple candidate graphic elements and their corresponding candidate bounding boxes, candidate mask regions, and candidate confidence scores. For example, the product depicted within the digital image 334 may optionally include multiple different designs, text, and symbols. In some embodiments, the forgery identification system 106 utilizes the confidence scores to select graphic element 346 from the candidate bounding boxes. For example, the forgery identification system 106 designates the candidate graphic element corresponding to the highest candidate confidence score as graphic element 346.

[0073] Furthermore, in some embodiments, the counterfeit identification system 106 selects graphic element 346 based on whether the suggested confidence score meets a suggested confidence score threshold. Typically, if none of the suggested confidence scores 344 associated with graphic element suggestion 342 meets the suggested confidence score threshold, the counterfeit identification system 106 will not classify the digital image as depicting a genuine product or a counterfeit product. For example, based on the determination that none of the suggested confidence scores 344 meet the suggested confidence score threshold, the counterfeit identification system 106 marks digital image 334 as unclassified or unknown.

[0074] Figure 3 The corresponding paragraphs above describe a forgery identification system 106 that extracts graphic elements from digital images using a machine learning model according to one or more embodiments. As previously stated, the forgery identification system 106 generates graphic features from the extracted graphic elements. The following figures and paragraphs also describe in detail how, according to one or more embodiments, the forgery identification system 106 generates shape features, color features, and text features for both real and real graphic elements. Specifically, Figure 4The appended paragraphs describe in detail how, according to one or more embodiments, the forgery identification system 106 generates realistic shape features for real graphic elements and shape features for graphic elements. For illustration, Figure 4 A series of actions 400 are shown, including action 402 for generating an edge image, action 404 for generating a shape representation, and optional action 406 for normalizing the shape representation.

[0075] As described above, the forgery identification system 106 generates true color features for real graphic elements and color features for graphic elements within a digital image. The forgery identification system 106 may perform a series of actions 400 as part of generating both the true color features and the color features. In some embodiments, the forgery identification system 106 performs an optional action 406 of normalizing the shape representation of the real graphic elements instead of the shape representation of the graphic elements. In other embodiments, the forgery identification system 106 performs all actions within this series of actions 400 for both the real graphic elements and the graphic elements.

[0076] like Figure 4 As shown, the series of actions 400 includes an action 402 for generating an edge image. Specifically, the forgery identification system 106 can generate an edge image 414 of the graphic element 408 by utilizing the Canney edge detector 410 and optionally adaptive hysteresis 412. As previously described, the graphic element 408 represents a real graphic element or a graphic element within a digital image. Furthermore, the graphic element 408 includes a bounding box, which includes elements referenced above. Figure 2 The described region suggests the boundaries of the graphical elements output by the neural network.

[0077] As part of the action 402 of generating the edge image, the forgery identification system 106 may utilize the Cannibal edge detector 410. Typically, the Cannibal edge detector 410 includes an edge detection algorithm that detects large-scale edges in an image. For example, the forgery identification system 106 utilizes the Cannibal edge detector 410 to generate an edge image 414 indicating both the boundaries and interior edges of the graphic element 408. In some embodiments, the forgery identification system 106 utilizes a different type of edge detection algorithm instead of the Cannibal edge detector 410. For example, the forgery identification system 106 may utilize the Deriche edge detection algorithm or various other edge detection algorithms. In some embodiments, the forgery identification system 106 utilizes the Cannibal edge detector 410 to generate the edge image 414.

[0078] In some embodiments, the forgery identification system 106 also utilizes adaptive hysteresis 412 to compute edge image 414. Typically, the forgery identification system 106 utilizes adaptive hysteresis 412 to capture minute boundary details within graphic elements 408. Specifically, adaptive hysteresis 412 includes computing an adaptive threshold while performing Canney edge detection. The use of adaptive hysteresis 412 enables the forgery identification system 106 to produce sharp boundaries or edges for edge image 414. For example, adaptive hysteresis 412 counts stripes common in edge images generated using a single threshold limit. More specifically, in systems utilizing a single threshold limit, pixels that meet the threshold limit will contain edges, while pixels below the threshold limit will not contain edges. Conversely, in one or more embodiments, adaptive hysteresis 412 uses an upper threshold and a lower threshold limit to process pixel values. The forgery identification system 106 designates pixels with values ​​above the upper threshold limit as edge pixels, but not pixels with values ​​below the lower threshold limit. When surrounding pixels have already been designated as edge pixels or still exhibit high pixel values, the forgery identification system 106 determines that pixels with values ​​between the upper and lower thresholds are edge pixels. Therefore, the use of adaptive hysteresis 412 reduces stripes in the edge image 414.

[0079] Figure 4 The series of actions 400 shown includes an action 404 of generating a shape representation. Specifically, the forgery identification system 106 generates a shape representation 418 indicating a set of pixel coordinates corresponding to the edge image 414. In some embodiments, the forgery identification system 106 determines an edge value for each pixel location of the edge image 414. For example, as... Figure 4 As shown, the forgery identification system 106 determines a binary edge value (0 or 1) for each pixel location based on whether that pixel location has an edge. An edge value matrix 416 is used to depict the edge values. Each square in the edge value matrix 416 corresponds to a pixel location in the edge image 414. The forgery identification system 106 then generates a shape representation 418 by creating a list of pixel coordinates corresponding to pixels with positive (e.g., 1) edge values.

[0080] As described above, the forgery identification system 106 may perform an optional action 406 to normalize the shape representation. Typically, the forgery identification system 106 performs the optional action 406 to make the shape representations comparable to each other. For example, the size of a real graphic element (e.g., 400 pixels × 400 pixels) may differ from the size of a graphic element (e.g., 200 pixels × 200 pixels). For example, in some embodiments, the forgery identification system 106 converts the pixel coordinates of a single pixel to normalized coordinates (x, y). In one or more embodiments, the forgery identification system 106 normalizes the pixel coordinates to a range between 0 and 1.

[0081] For example, the edge image 414 is 400 px × 400 px in size. The forgery identification system 106 converts the pixel coordinates within the shape representation 418 into normalized coordinates falling between 0 and 1. In some embodiments, the forgery identification system 106 achieves this by dividing the x-pixel coordinate by the size width and the y-coordinate by the size height. As shown, the normalized pixel coordinates for the shape representation are equal to (0.875, 0.225).

[0082] In some embodiments, the forgery identification system 106 performs optional action 406 on genuine graphic elements but not on other graphic elements. For example, in some embodiments, the forgery identification system 106 determines a normalized genuine shape representation, which it compares with the shape representations corresponding to the graphic elements within the digital image. In other embodiments, the forgery identification system 106 normalizes the shape representations of both genuine and other graphic elements for comparison.

[0083] Furthermore, the forgery identification system 106 can determine the optional action 406 to perform based on the file type of the genuine graphic element. Typically, the forgery identification system 106 can receive two forms of graphic elements 408: vector form and raster form. Specifically, vector form graphic elements include vector files constructed using mathematical formulas that establish points on a grid. Vector file types include EPS, AI, PDF, and other file types. If the graphic element 408 is in vector form, the forgery identification system 106 can easily adjust the size of the graphic element 408 for comparison with other graphic elements.

[0084] Conversely, the forgery identification system 106 can determine the optional action 406 of performing normalized shape representation based on determining that the graphic element 408 is in raster form. Specifically, a raster-form graphic element includes a raster file comprising multiple color pixels or individual building blocks. Raster file types include JPEG, GIF, PNG, and other image types. If the graphic element 408 is in raster form, the forgery identification system 106 can perform the optional action 406 of normalized shape representation.

[0085] Figure 4 The diagram illustrates how the forgery identification system 106 generates realistic shape features and shape characteristics according to one or more embodiments. As described above, the forgery identification system 106 can also generate realistic color features and color characteristics. Figure 5 The corresponding paragraphs also describe in detail how the forgery identification system 106 generates genuine color features and color characteristics according to one or more embodiments. Specifically, Figure 5A series of actions 500 are illustrated, including an action 502 to normalize pixel coordinates and an action 504 to map pixels from graphic elements to a color space. As described above, the forgery identification system 106 can perform this series of actions 500 as part of generating realistic color features associated with real graphic elements and also generating color features associated with digital images.

[0086] like Figure 5 As shown, this series of actions 500 includes an action 502 of normalizing pixel coordinates. Typically, the forgery identification system 106 performs action 502 to make color features comparable. As previously described, regarding normalized shape representation, the forgery identification system 106 converts the pixel coordinates of individual pixels into normalized coordinates (x, y). In one or more embodiments, the forgery identification system 106 normalizes pixel coordinates to values ​​within the range of 0 and 1. The forgery identification system 106 may normalize pixel coordinates based on the real graphic elements being in raster form rather than in vector form. Furthermore, in one or more embodiments, the forgery identification system 106 normalizes the pixel coordinates of the real graphic elements rather than the graphic elements themselves. In still other embodiments, the forgery identification system 106 normalizes the pixel coordinates of both the real graphic elements and the graphic elements depicted in the digital image.

[0087] like Figure 5 As further illustrated, this series of actions 500 includes an action 504 of mapping pixels from a graphic element to a color space. Specifically, the forgery identification system 106 maps pixels from a graphic element 510 to a color space to generate color features. The graphic element 510 represents a real graphic element or a graphic element depicted within a digital image. For illustration, the forgery identification system 106 maps pixel 506a at a first pixel location and pixel 506b at a second pixel location to a color space 512. The forgery identification system 106 uses the color space 512 to generate a color value for each pixel from pixel 506a to pixel 506b.

[0088] The forgery identification system 106 is capable of utilizing various color spaces as part of generating color features and genuine color features. In one or more embodiments, the forgery identification system 106 maps pixels 506a to 506b to a Hue, Saturation, Value (HSV) or Hue, Saturation, Lightness (HSL) color space. HSV and HSL color spaces are alternative representations of the RGB color model. In the HSV color space, hue is represented as a value from 0 to 360 degrees. Saturation describes the gray measure in a particular color and, optionally, is represented as a value between 0 and 100, where 0 is associated with gray and produces a faded effect. Value or lightness describes the brightness or intensity of a color and, optionally, is represented as a value between 0 and 100 percentages, where 0 is complete black and 100 is the brightest and most vibrant color. Although... Figure 5 The forgery identification system 106 is shown to map pixels 506a to 506b to the HSV color space, but the forgery identification system 106 may map pixels 506a to 506b to alternative color spaces. For example, the forgery identification system 106 may map pixels 506a to 506b to RYB, RBG, CMY, or other color spaces.

[0089] like Figure 5 As shown, the forgery identification system 106 maps pixels from graphic element 510 to a color space to generate color values ​​508a to 508b. Figure 5 As shown, color values ​​508a to 508b correspond to pixels 506a to 506b, respectively. The forgery identification system 106 also associates normalized pixel coordinates with color values ​​508a to 508b. In one or more embodiments, the color features and the true color features include a color value matrix. Specifically, each position in the color value matrix corresponds to a pixel position within the graphic element 510. Each position may correspond to one or more different color values. For example, if the pixels from the graphic element are mapped to the HSV color space, each position in the color value matrix is ​​associated with a hue value, a saturation value, and a brightness value.

[0090] Figure 5 The accompanying paragraphs describe how the forgery identification system 106 generates color features and genuine color features according to one or more embodiments. The forgery identification system 106 can also generate genuine text features and text features. Figure 6 The corresponding paragraphs illustrate how the forgery identification system 106 generates authentic text features and text characteristics according to one or more embodiments. Specifically, Figure 6 A series of actions 600 are shown, including action 602 identifying text within a graphic element, action 604 extracting text attributes, and action 606 maintaining the hash graph.

[0091] like Figure 6 As shown, the series of actions 600 includes action 602 identifying text within a graphic element. Specifically, the forgery identification system 106 utilizes a text detection machine learning model 610 to extract text 612 from a graphic element 608. The graphic element 608 includes real graphic elements or graphic elements from digital images. In some embodiments, the text detection machine learning model 610 includes a CNN-based model trained using text with various attributes. The forgery identification system 106 utilizes the text detection machine learning model 610 to analyze the image and generate a bounding box 616 including the text 612 and a corresponding text confidence score indicating the probability that the bounding box 616 contains text. Furthermore, in some embodiments, the text detection machine learning model 610 generates character bounding boxes 614 for each character within the text 612. The text detection machine learning model 610 also generates a character confidence score corresponding to the character bounding box 614, indicating the probability that a given character bounding box contains a character.

[0092] Figure 6 The series of actions 600 shown also includes an action 604 for extracting text attributes. Specifically, the forgery identification system 106 analyzes the text 612 to extract text attributes, which include at least one of position, style, distortion, and color. Specifically, the forgery identification system 106 analyzes the text 612 to extract the text position. In some embodiments, the forgery identification system 106 determines the position of the text 612 relative to the graphic element 608. For example, the forgery identification system 106 determines the normalized pixel coordinates for the top-left position (XY) of the text 612 within the graphic element 608. Additionally or alternatively, the forgery identification system 106 may also determine the size of the text 612 relative to the graphic element 608. For example, the forgery identification system 106 determines the normalized pixel coordinates corresponding to the boundaries of the text 612.

[0093] like Figure 6 As further shown, the forgery identification system 106 also determines text styles as part of extracting text attributes. Typically, text styles are resources that specify text attributes in a particular format. Example text styles include bold, italic, underline, strikethrough, overline, uppercase, normal, or other text styles.

[0094] The forgery identification system 106 can also determine text distortion as part of the extracted text attributes. Text distortion refers to a design or deformation applied to text 612 to change the shape of text 612. Examples of text distortion attributes include arcing, bulging, bulging, fisheye, and other deformations.

[0095] like Figure 6As further shown, as part of performing the action 604 of extracting text attributes, the forgery identification system 106 also determines the text color. Specifically, the forgery identification system 106 determines the color of text 612. In some embodiments, the forgery identification system 106 determines the color of text 612 by mapping pixels within text 612 to a color space (e.g., RGB, HSV, etc.). In some embodiments, the forgery identification system 106 reduces the computational load required to extract text color by selecting specific pixels within text 612 to which its color value is to be determined. For example, the forgery identification system 106 determines the color value (or range) of the pixel located at the top-left corner of text 612 and a second color value (or range) of the pixel located at the bottom-right corner of text 612. Additionally or alternatively, the forgery identification system 106 measures the text color change from one point in text 612 to another point in text 612. For example, the forgery identification system 106 determines the color difference between the pixel located at the top-left corner of text 612 and the pixel located at the bottom-right corner of text 612.

[0096] In addition, such as Figure 6 As shown, the forgery identification system 106 determines the font of text 612 as part of performing the action 604 of extracting text attributes. Typically, a font indicates the specific size, weight, and style of the letterform. Specifically, the font includes a matching set of types, where each glyph corresponds to one type, and the letterform consists of a set of fonts sharing a common overall design. For example, the forgery identification system 106 determines that text 612 is Times New Roman, EB Garamond, Lora, or another font.

[0097] Furthermore, in some embodiments, the forgery identification system 106 also extracts text kerning as part of performing action 604. Typically, kerning refers to the spacing between characters in text. The forgery identification system 106 determines the center of each character depicted by character bounding boxes 614. The forgery identification system 106 then measures the distance between the centers of each character. For example, the forgery identification system 106 measures the distance between the centers of “B” and “R”, “R” and “A”, etc. The forgery identification system 106 also determines normalized character distances between each character in the text 612. For example, the forgery identification system 106 normalizes the character distances relative to the text length.

[0098] like Figure 6As further shown, the series of actions 600 also includes an action 606 for maintaining a hash graph. In some embodiments, the forgery identification system 106 performs action 606 only when extracting genuine text attributes, not when generating text attributes. Typically, the forgery identification system 106 maintains a hash graph that stores items with various extracted genuine text attributes. Specifically, the forgery identification system 106 maps genuine brands and their corresponding genuine graphic elements(s) to their genuine text attributes within the hash graph. Thus, the forgery identification system 106 efficiently accesses and references genuine text attributes when analyzing digital images.

[0099] although Figure 6 One method is illustrated for the forgery identification system 106 to generate authentic text features, but in some embodiments, the forgery identification system 106 receives authentic text features from a user client device associated with an authentic seller. For example, the forgery identification system 106 receives one or more acceptable positions, text styles, text distortions, and text colors from the user client device associated with an authentic seller.

[0100] The preceding figures and paragraphs describe in detail how the forgery identification system 106 extracts genuine graphic features and graphic characteristics according to one or more embodiments. As previously described, the forgery identification system 106 compares genuine graphic features with graphic characteristics to determine whether a graphic element is genuine or forged. Figures 7A to 7B The corresponding discussion describes how, in one or more embodiments, the counterfeit identification system 106 compares genuine graphic features with graphic features to determine if a digital image depicts a counterfeit product. Specifically, Figures 7A to 7B A series of actions 700 are shown, including actions 702 of aligning real graphic elements and graphic elements in the same orientation plane, actions 704 of scaling the bounding box of real graphic elements to the bounding box of graphic elements, actions 706 of comparing real shape features with shape features, actions 708 of determining the range of real color features, actions 710 of comparing color features with the range of real color features, and actions 712 of comparing real text features with text features.

[0101] like Figure 7AAs shown, the series of actions 700 includes action 702 aligning the real graphic element and the graphic element 714 in the same orientation plane. Typically, graphic element 716 in a digital image may be tilted relative to the real graphic element 714. Therefore, the forgery identification system 106 adjusts the alignment of graphic element 716 to match the alignment of the real graphic element 714. In some embodiments, the forgery identification system 106 turns or orients graphic element 716 to be in the same orientation plane as the real graphic element 714. In another example, the forgery identification system 106 turns both the real graphic element 714 and graphic element 716 to align them in a common orientation plane.

[0102] The series of actions 700 also includes an action 704 of scaling the real graphic element bounding box to the size of the graphic element bounding box. Typically, the forgery identification system 106 scales the real graphic element bounding box 718 to the size of the graphic element bounding box 720. This is necessary for normalizing the bounding box so that comparisons are made at the same scale. As mentioned above, if the real graphic element 714 is in vector form, the forgery identification system 106 can simply scale the real graphic element bounding box 718. However, if the real graphic element 714 is in pixel form, the forgery identification system 106 scales the real graphic element bounding box 718 based on normalized pixel coordinates. In some embodiments, instead of scaling the real graphic element bounding box 718 to match the size of the graphic element bounding box 720, the forgery identification system 106 scales both the real graphic element bounding box 718 and the graphic element bounding box 720 to a common normalized scale. For example, the forgery identification system 106 adjusts the sizes of the real graphic element bounding box 718 and the graphic element bounding box 720 so that the pixel coordinates within both fall between values ​​of 0 and 1. In any case, the forgery identification system 106 makes the genuine graphic element bounding box 718 and the graphic element bounding box 720 the same size for comparison.

[0103] The counterfeit identification system 106 executes actions 706 to 712, as part of comparing graphic features with genuine graphic features. Specifically, such as... Figure 7A As shown, the forgery identification system 106 performs an action 706 of comparing the real shape features with the shape features. Specifically, action 706 includes comparing the real shape representation 722 with the shape representation 724. As previously described, the real shape representation 722 includes a list of pixel coordinates corresponding to the edge pixels of the real graphic element 714, and the shape representation 724 includes a list of pixel coordinates corresponding to the edge pixels of the graphic element 716. Furthermore, at least one of the real shape representation 722 and the shape representation 724 includes normalized pixel coordinates for comparison at the same scaling.

[0104] In some embodiments, the forgery identification system 106 performs action 706, namely, comparing a real shape feature with a shape feature by subtracting a shape representation 724 from a real shape representation 722, and vice versa. By subtracting the shape representation 724 from the real shape representation 722, the forgery identification system 106 can detect substantial differences between the shapes or edges of the real graphic element 714 and the graphic element 716. For example, the forgery identification system 106 determines that the shape representation 724 differs from the real shape representation 722 at edge coordinates 726. Based on the determination that the shape feature differs from the real shape feature, the forgery identification system 106 determines that the graphic element 716 is a forgery graphic element. In some embodiments, the forgery identification system 106 determines that the shape feature differs from the real shape feature based on the determination that the difference between the features meets a shape difference threshold. For example, the forgery identification system 106 determines the sum of shape differences based on the difference between shape representation 724 and the real shape representation 722. Based on the sum of shape differences meeting a shape difference threshold, the forgery identification system 106 determines that the graphic element 716 is a forgery graphic element.

[0105] like Figure 7A As shown, the forgery identification system 106 also performs the action 708 of determining the true color feature range. In some embodiments, the forgery identification system 106 determines the true color feature range for each pixel coordinate within a genuine graphic element. For illustration, the forgery identification system 106 determines the hue (H), saturation (S), and value or brightness (V) values ​​for the genuine pixel coordinate 728 of the genuine graphic element 714. The forgery identification system 106 determines the true color feature range to account for variations in lighting, image editing, and other variables in the digital image. For example, the forgery identification system 106 determines a true hue range of 234 to 240 degrees and a true saturation range of 73 to 76% for the genuine pixel coordinate 728. In some embodiments, the forgery identification system 106 determines the true color feature range for some attributes but not others. For example, the forgery identification system 106 does not determine the true value range but instead uses a true brightness value of 87% for comparison.

[0106] In some embodiments, the forgery identification system 106 performs the action 708 of determining the true color feature range by receiving the true color feature range from the genuine seller. Specifically, the forgery identification system 106 receives the true color feature range for pixels within the genuine graphic element 714 from a user client device associated with the genuine seller. In some embodiments, the forgery identification system 106 does not receive or determine pixel coordinates corresponding to the true color feature range. Instead, the forgery identification system 106 receives a set of true color feature ranges for the entire genuine graphic element 714. For example, the forgery identification system 106 receives a set of true color feature ranges acceptable for the genuine graphic element 714.

[0107] Figure 7A The document also illustrates the action 710 of the forgery identification system 106 performing a comparison of color features with a range of genuine color features. Typically, the forgery identification system 106 determines whether a color feature falls within a range of genuine color features. Specifically, the forgery identification system 106 compares the color feature associated with each pixel coordinate within graphic element 716 with the genuine color feature at the corresponding pixel coordinate within genuine graphic element 714. For illustration, the forgery identification system 106 determines the HSV value of pixel coordinate 730 in the digital image. The forgery identification system 106 compares the HSV value at pixel coordinate 730 with the range of HSV values ​​at genuine pixel coordinate 728. The forgery identification system 106 determines that the hue value (232 degrees) at pixel coordinate 730 falls outside the genuine hue range (234 to 240 degrees) of the corresponding genuine pixel coordinate 728. In some embodiments, based on the determination that the color feature falls outside the range of genuine color features, the forgery identification system 106 determines that graphic element 716 is a forgery graphic element.

[0108] As described above, in some examples, the forgery identification system 106 determines or receives a genuine set of color feature ranges associated with the genuine graphic element 714. In this case, the forgery identification system 106 compares the color features of the graphic element 716 with the genuine set of color feature ranges. For illustration, the forgery identification system 106 selects pixels at individual pixel coordinates within the graphic element 716. The forgery identification system 106 compares the color features (e.g., HSV values) at each pixel coordinate with the genuine set of color feature ranges. Based on the determination that one or more color features fall outside the genuine set of color feature ranges, the forgery identification system 106 marks the graphic element 716 as a forgery graphic element.

[0109] like Figure 7BAs further shown, the series of actions 700 includes an action 712 comparing genuine text features with text features. Specifically, the counterfeit identification system 106 compares the text features 734 of the graphic element 716 with the genuine text features 732 of the genuine graphic element 714. In some embodiments, based on any deviation between the genuine text features 732 and the text features 734, the counterfeit identification system 106 determines that the product associated with the graphic element 716 is a counterfeit product. For example, based on the determination that the text style within the text feature 734 is different from the text style within the genuine text feature 732, the counterfeit identification system 106 determines that the graphic element 716 is a counterfeit graphic element.

[0110] As described above, in some embodiments, the counterfeit identification system 106 determines that a product depicted within a digital image is a counterfeit product based on comparing authentic graphic features with the graphic features. In some embodiments, the counterfeit identification system 106 determines that a product is a counterfeit product based on any one of shape features, color features, or text features that deviate from authentic shape features, authentic color features, or authentic text features, respectively. In other embodiments, the counterfeit identification system 106 determines that a product is a counterfeit product based on any one of shape features, color features, or text features that fall outside the range of authentic shape features, authentic color features, or authentic text features.

[0111] Furthermore, in some embodiments, the counterfeit identification system 106 utilizes a counterfeit detection model to determine whether a product is genuine or counterfeit. Specifically, the counterfeit identification system 106 may use the counterfeit detection model to compare genuine graphic features with graphic features. Figures 8A to 8B The diagram illustrates that, according to one or more embodiments, the forgery identification system 106 learns parameters for a forgery detection model and utilizes the forgery detection model. Figure 8A A forgery labeling system 106 for training a forgery detection model 804 is illustrated. Typically, the forgery labeling system 106 inputs graphical elements 802 and training ground truth graphical elements 801 into the forgery detection model 804 to generate a predicted similarity confidence score 805. The forgery labeling system 106 determines the predicted ground truth 806 based on the predicted similarity confidence score 805. The forgery labeling system 106 compares the predicted ground truth 806 with the true ground truth 812 and adjusts the parameters of the forgery detection model 804 to reduce the loss 808 between the predicted ground truth 806 and the true ground truth 812.

[0112] refer to Figure 8AThe forgery identification system 106 extracts graphical elements 802 from real digital images used for training the forgery detection model 804. Specifically, the forgery identification system 106 utilizes a CNN to extract graphical elements 802 from the real digital images used for training. The graphical elements 802 include combinations of real and forged graphical elements associated with real and forged digital images, respectively.

[0113] The counterfeit identification system 106 also trains the input of authentic graphic elements 801 into the counterfeit detection model 804. Authentic graphic elements 801 include authentic graphic elements received from authentic sellers. In some embodiments, the counterfeit identification system 106 inputs authentic shape features, authentic color features, and authentic text features for various authentic graphic elements, instead of authentic graphic elements 801.

[0114] The forgery identification system 106 uses a forgery detection model 804 to generate a predicted similarity confidence score 805. As part of generating the predicted similarity confidence score 805, the forgery detection model 804 generates training shape features, training color features, and training text features corresponding to each graphic element in the graphic element 802. The forgery identification system 106 also uses the forgery detection model 804 to generate a predicted similarity confidence score 805 corresponding to each of the training shape features, training color features, and training text features. For illustration, the forgery identification system 106 uses the forgery detection model 804 to generate predicted similarity confidence scores, including shape similarity scores, color similarity scores, and text similarity scores. The predicted similarity confidence score 805 indicates the degree of similarity between the training shape features, training color features, and training text features and the real shape features, real color features, and real text features, respectively.

[0115] like Figure 8A As further shown, the forgery identification system 106 uses the predicted similarity confidence score 805 to determine the predicted authenticity 806. In some embodiments, the forgery identification system 106 uses one or more threshold similarity values ​​as part of generating the predicted authenticity 806. Specifically, based on determining that at least one of the predicted similarity confidence scores 805 falls below the threshold similarity value, the forgery identification system 106 determines that the corresponding training digital image depicts a forgery product. For illustration, in some embodiments, the threshold similarity value is equal to 0.65. Based on determining that any one of the shape similarity score, color similarity score, and text similarity score of the training graphic features is below the threshold similarity value, the forgery identification system 106 determines that the corresponding training graphic element is a forgery graphic element.

[0116] As described above, the forgery identification system 106 compares the true authenticity 812 with the predicted authenticity 806. Specifically, the forgery identification system 106 determines, evaluates, identifies, or generates a loss 808 between the predicted authenticity 806 and the true authenticity 812. The forgery identification system 106 also adjusts the parameters of the forgery detection model 804 based on the loss 808.

[0117] Figure 8A This illustrates how the forgery identification system 106 learns parameters for the forgery detection model 804, while Figure 8B The diagram illustrates the application of a forgery detection model 804 by a forgery identification system 106 according to one or more embodiments. During application, the forgery identification system 106 utilizes the forgery detection model 804 to analyze graphic elements 814 and genuine graphic elements 820. The forgery identification system 106 uses the forgery detection model 804 to generate a similarity confidence score 822 and makes a authenticity determination 818 based on the similarity confidence score 822.

[0118] The counterfeit identification system 106 extracts graphic elements 814 from a digital image depicting the product. The counterfeit identification system 106 accesses real graphic elements 820 from a repository of real graphic elements. Alternatively or additionally, the counterfeit identification system 106 may input real graphic features, including real shape features, real color features, and real text features, instead of inputting real graphic elements 820.

[0119] The forgery identification system 106 uses a forgery detection model 804 to generate graphic features of graphic element 814. Specifically, the forgery detection model 804 generates graphic features including shape features, color features, and text features of graphic element 814. The forgery detection model 804 also compares the generated graphic features with real graphic features from real graphic element 820. The forgery identification system 106 uses the forgery detection model 804 to further generate a similarity confidence score 822 based on the comparison of shape features, color features, and text features. More specifically, the similarity confidence score 822 includes shape similarity score, color similarity score, and text similarity score.

[0120] As previously described, the forgery identification system 106 makes a authenticity determination 818 based on a similarity confidence score 822 and a threshold similarity value. In some embodiments, the threshold similarity value is fixed across all similarity confidence scores 822. For example, based on determining that any one of the shape similarity score, color similarity score, or text similarity score is below the threshold similarity value, the forgery identification system 106 determines that the graphic element 814 is a forged graphic element. Additionally or alternatively, the forgery identification system 106 determines individual threshold similarity values ​​for each graphic feature in the graphic features. For example, the forgery identification system 106 determines a threshold shape similarity value of 0.65, a threshold color similarity value of 0.34, and a threshold text similarity value of 0.77.

[0121] In some embodiments, the forgery identification system 106 adjusts or modifies a threshold similarity value. The forgery identification system 106 adjusts the threshold similarity value based on the quality of the graphic element 814 or the real graphic element 820. For example, based on determining that the image quality of the graphic element 814 or the real graphic element 820 is below a resolution threshold, the forgery identification system 106 determines to reduce the threshold similarity value. In some examples, the forgery identification system 106 adjusts the threshold similarity value based on user input.

[0122] Figure 9 Additional details are provided regarding various components and capabilities of the counterfeit identification system 106 according to one or more embodiments. Typically, Figure 9 A forgery identification system 106 implemented by an online content management system 104 on a computing device 900 (e.g., user client device 108 and / or server device 102) is illustrated. As shown, the forgery identification system 106 includes, but is not limited to, a real digital image manager 902, a digital image manager 904, a graphic element extractor 906, a graphic feature comparator 908, a machine learning model manager 910, and a storage manager 912. The storage manager 912 stores real digital images 914, digital images 916, and real graphic features 918. In some embodiments, the forgery identification system 106 is implemented as part of the online content management system 104 in a distributed system of a server device for identifying forged products in digital images. Additionally or alternatively, the forgery identification system 106 may be implemented in a manner such as... Figure 1 Implemented on a single computing device of (multiple) server devices 102.

[0123] In one or more embodiments, each component of the counterfeit identification system 106 communicates with each other using any suitable communication technology. Additionally, the components of the counterfeit identification system 106 are associated with, among others, […]. Figure 1 The user client device 108 shown communicates with one or more other devices. Although the components of the forgery identification system 106 are... Figure 9The components are shown as separate, but any sub-component can be combined into fewer components, such as a single component, or divided into more components to serve a specific implementation. Furthermore, although described in conjunction with the forgery identification system 106... Figure 9 The components, however, at least some of the components used to perform operations in conjunction with the forgery identification system 106 described herein can be implemented on other devices within the environment.

[0124] Components of the forgery identification system 106 may include software, hardware, or both. For example, components of the forgery identification system 106 may include one or more instructions stored on a computer-readable storage medium and executable by a processor of one or more computing devices (e.g., user client device 108). When executed by one or more processors, the computer-executable instructions of the forgery identification system 106 may cause the computing device to perform the object clustering method described herein. Alternatively, components of the forgery identification system 106 may include hardware, such as a dedicated processing device that performs a specific function or group of functions. Additionally or alternatively, components of the forgery identification system 106 may include a combination of computer-executable instructions and hardware.

[0125] Furthermore, components of the forgery identification system 106, which perform the functions described herein with respect to the forgery identification system 106, may be implemented as, for example, parts of a standalone application, modules of an application, plugins of an application, one or more library functions that can be called by other applications, and / or cloud computing models. Therefore, components of the forgery identification system 106 may be implemented as parts of a standalone application on a personal computing device or mobile device. Alternatively or additionally, components of the forgery identification system 106 may be implemented in any application that provides image management, including but not limited to Adobe. ® Experience Cloud, such as Adobe ® MAGENTO ® Adobe ® COMMERCE CLOUD, ADOBE ® ANALYTICS, ADOBE ® MARKETING CLOUD™ and ADOBE ® ADVERTISING CLOUD. “ADOBE,” “ADOBE MAGENTO,” and “ADOBE MARKETING CLOUD” are registered trademarks of Adobe Systems Incorporated in the U.S. and / or other countries.

[0126] The counterfeit identification system 106 includes a genuine digital image manager 902. The genuine digital image manager 902 receives, accesses, and / or manages genuine digital images depicting genuine products. In some embodiments, the genuine digital image manager 902 manages genuine graphic features received by the counterfeit identification system 106.

[0127] like Figure 9 As shown, the forgery identification system 106 also includes a digital image manager 904. Typically, the digital image manager 904 receives, accesses, and / or manages digital images.

[0128] The forgery identification system 106 also includes a graphic element extractor 906. The graphic element extractor 906 communicates with the digital image manager 904 to access digital images. The graphic element extractor 906 also utilizes a CNN to extract graphic elements from the digital images. In some embodiments, the graphic element extractor 906 also extracts authentic graphic elements from genuine digital images.

[0129] Figure 9 The forgery identification system 106 shown also includes a graphic feature comparator 908. The graphic feature comparator 908 compares genuine graphic features with other graphic features. In some embodiments, the graphic feature comparator 908 manages a forgery detection model and uses the forgery detection model to compare genuine graphic elements with other graphic elements.

[0130] Figure 9 A machine learning model manager 910, which is part of the forgery identification system 106, is also shown. The machine learning model manager 910 manages various machine learning models utilized by the forgery identification system 106. In some embodiments, the machine learning model manager 910 trains and applies machine learning models. For example, the machine learning model manager 910 manages and stores forgery detection models, region proposal neural networks, and other models utilized by the forgery identification system 106.

[0131] The counterfeit identification system 106 also includes a storage manager 912. The storage manager 912 stores data via one or more storage devices. Specifically, the storage manager 912 stores a real digital image 914, a digital image 916, and a real graphic feature 918.

[0132] Figures 1 to 9 The corresponding text and examples provide various methods, systems, devices, and non-transient computer-readable media for the forgery identification system 106. In addition to the above, one or more embodiments may be described according to flowcharts including actions for achieving specific results, such as... Figure 10 As shown. Figure 10The series of actions shown can be performed with more or fewer actions. Furthermore, the actions shown can be performed in different orders. Additionally, the actions described herein can be repeated or performed in parallel with each other, or in parallel with different instances of the same or similar actions.

[0133] Figure 10 A flowchart is shown of a series of actions 1000 for determining a digital image depicting a counterfeit product according to one or more embodiments. Specifically, the series of actions 1000 includes an action 1002 of extracting real graphic elements, an action 1004 of generating real graphic features, an action 1006 of extracting graphic elements, an action 1008 of generating graphic features, and an action 1010 of determining that a digital image depicts a counterfeit product.

[0134] like Figure 10 As shown, the series of actions 1000 includes action 1002 of extracting real graphic elements. Specifically, action 1002 includes extracting real graphic elements from a real digital image. In some embodiments, action 1002 further includes extracting real graphic elements from a real digital image by accessing a repository of real graphic elements. Furthermore, in some embodiments, action 1002 includes extracting graphic elements by utilizing a machine learning model to generate a bounding box including the boundaries of the graphic elements, a mask region, and a confidence score indicating the probability that the bounding box includes the graphic elements.

[0135] Figure 10 The action 1004 also includes determining authentic graphic features. Specifically, action 1004 includes determining authentic graphic features for authentic graphic elements, wherein authentic graphic features include authentic shape features, authentic color features, and authentic text features. In some embodiments, action 1004 includes determining authentic graphic features by receiving a range of authentic graphic features from a user client device associated with an authentic seller. In some embodiments, action 1004 further includes determining authentic graphic features by determining a range of authentic graphic features.

[0136] Figure 10 The series of actions 1000 shown also includes action 1006 for extracting graphic elements. Specifically, action 1006 includes extracting graphic elements from a digital image. In some embodiments, action 1006 further includes extracting graphic elements from a digital image by utilizing a machine learning model.

[0137] like Figure 10As further shown, the series of actions 1000 includes an action 1008 for generating graphic features. Specifically, action 1008 includes generating graphic features for graphic elements, wherein the graphic features include shape features, color features, and text features. Typically, action 1008 may also include generating graphic features for graphic elements through the following steps: extracting shape features based on a normalized edge map of the generated graphic element; extracting color features based on mapping color values ​​in a color space; and extracting text features by analyzing text attributes. In some embodiments, action 1008 further includes extracting the graphic element by utilizing a machine learning model to generate a bounding box including the boundary of the graphic element, a mask region, and a confidence score indicating the probability that the bounding box includes the graphic element.

[0138] In some embodiments, action 1008 includes generating shape features for a graphic element by: generating an edge image using a Canney edge detector; and generating a shape representation indicating a set of pixel coordinates corresponding to the edge image. In some embodiments, generating the edge image further includes utilizing adaptive hysteresis. Furthermore, in one or more embodiments, action 1008 includes generating color features by generating hue and saturation values ​​by mapping pixels from the graphic element to a color space. Additionally, in some embodiments, action 1008 includes generating text features by: identifying text within the graphic element using a text detection machine learning model; and extracting text attributes including at least one of position, style, distortion, and color.

[0139] Furthermore, in some embodiments, action 1008 includes generating a normalized edge map of the graphic element by: generating an edge image using a Canney edge detector and adaptive hysteresis; generating a shape representation indicating a set of pixel coordinates corresponding to the edge image; and normalizing the set of pixel coordinates. Action 1008 may also include extracting text features by: identifying text within the graphic element using a text detection machine learning model; and extracting text attributes including at least one of position, style, distortion, and color.

[0140] Figure 10 The series of actions 1000 shown includes action 1010 of determining a digital image depicting a counterfeit product. Specifically, action 1010 includes determining a digital image depicting a counterfeit product based on a comparison of graphic features with genuine graphic features. In some embodiments, action 1010 includes determining a digital image depicting a counterfeit product by: generating a similarity confidence score using a counterfeit detection model based on analysis of genuine graphic features and graphic features; and determining that at least one of the similarity confidence scores is below a threshold similarity value. In some embodiments, the similarity confidence score includes a shape similarity score, a color similarity score, and a text similarity score.

[0141] As a supplement (or alternative) to the above actions, in some embodiments, the series of actions 1000 includes steps for determining a digital image depicting a counterfeit product based on real graphic features and graphic characteristics. For example, refer to... Figures 7A to 8B The described actions may include corresponding actions (or structures) for performing steps to determine the digital image depiction of a counterfeit product based on real graphic features and graphic characteristics.

[0142] Embodiments of this disclosure may include or utilize a dedicated or general-purpose computer including computer hardware such as one or more processors and system memory, as discussed in more detail below. Embodiments within the scope of this disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Specifically, one or more processes described herein may be implemented at least in part as instructions implemented in a non-transitory computer-readable medium and executable by one or more computing devices (e.g., any media content access device described herein). Typically, a processor (e.g., a microprocessor) receives instructions from a non-transitory computer-readable medium (e.g., memory, etc.) and executes those instructions to perform one or more processes, including one or more processes described herein.

[0143] Computer-readable media can be any available medium accessible by a general-purpose or special-purpose computer system. A computer-readable medium storing computer-executable instructions is a non-transitory computer-readable storage medium (device). A computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of this disclosure may include at least two distinct types of computer-readable media: non-transitory computer-readable storage media (devices) and transmission media.

[0144] Non-transient computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSDs”) (e.g., RAM-based), flash memory, phase-change memory (“PCM”), other types of memory, other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code components in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0145] "Network" is defined as one or more data links that enable the transmission of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer via a network or another communication connection (hardwired, wireless, or a combination of hardwired and wireless), the computer appropriately considers that connection as a transmission medium. Transmission media may include networks and / or data links that can be used to carry desired program code components in the form of computer-executable instructions or data structures, and that are accessible by general-purpose or special-purpose computers. Combinations of the foregoing should also be included within the scope of computer-readable media.

[0146] Furthermore, upon arrival at various computer system components, program code components in the form of computer-executable instructions or data structures can be automatically transferred from the transmission medium to a non-transitory computer-readable storage medium (device) (and vice versa). For example, computer-executable instructions or data structures received via a network or data link can be cached in RAM within a network interface module (e.g., a "NIC") and then ultimately transferred to the computer system RAM and / or a less volatile computer storage medium (device) at the computer system. Therefore, it should be understood that non-transitory computer-readable storage media (devices) can be included in computer system components that also (or even primarily) utilize the transmission medium.

[0147] Computer-executable instructions include, for example, instructions and data that, when executed by a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a particular function or group of functions. In some embodiments, executing the computer-executable instructions on a general-purpose computer transforms the general-purpose computer into a special-purpose computer that implements the elements of this disclosure. The computer-executable instructions may be, for example, binary, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary forms for implementing the claims.

[0148] Those skilled in the art will understand that this disclosure can be implemented in network computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframes, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc. This disclosure can also be implemented in distributed system environments, where both local and remote computer systems, linked via a network (via a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links), perform tasks. In a distributed system environment, program modules can reside on both local and remote memory storage devices.

[0149] Embodiments of this disclosure can also be implemented in a cloud computing environment. In this specification, "cloud computing" is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources. For example, cloud computing can be used in the market to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. The shared pool of configurable computing resources can be rapidly provisioned via virtualization, released with minimal management effort or service provider interaction, and then scaled accordingly.

[0150] Cloud computing models can be composed of various characteristics, such as on-demand self-service, widespread network access, resource pooling, rapid elasticity, and measurable services. Cloud computing models can also exhibit various service models, such as Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“IaaS”). Cloud computing models can also be deployed using different deployment models, such as private clouds, community clouds, public clouds, and hybrid clouds. In this specification and claims, a “cloud computing environment” means an environment in which cloud computing is employed.

[0151] Figure 11 A block diagram of a computing device 1100 is shown, which can be configured to perform one or more of the processes described above. It will be understood that one or more computing devices, such as computing device 1100, can implement the forgery identification system 106 and the online content management system 104. Figure 11 As shown, computing device 1100 may include processor 1102, memory 1104, storage 1106, I / O interface 1108, and communication interface 1110, which can be communicatively coupled via communication infrastructure 1112. In some embodiments, computing device 1100 may include a processor 1102, memory 1104, storage 1106, I / O interface 1108, and communication interface 1110, which can be communicatively coupled via communication infrastructure 1112. Figure 11 The components shown are fewer or more components. A more detailed description will follow. Figure 11 The components of the computing device 1100 shown.

[0152] In one or more embodiments, processor 1102 includes hardware for executing instructions, such as those constituting a computer program. By way of example and not limitation, in order to execute instructions for dynamically modifying a workflow, processor 1102 may retrieve (or fetch) instructions from internal registers, internal cache, memory 1104, or repository 1106, and decode and execute them. Memory 1104 may be volatile or non-volatile memory for storing data, metadata, and programs for execution by the processor(s). Repository 1106 includes storage devices such as hard disk drives, flash drives, or other digital storage devices for storing data or instructions for performing the methods described herein.

[0153] I / O interface 1108 allows a user to provide input to computing device 1100, receive output from computing device 1100, and otherwise transmit and receive data to and from computing device 1100. I / O interface 1108 may include a mouse, keypad or keyboard, touchscreen, camera, optical scanner, network interface, modem, other known I / O devices, or combinations of such I / O interfaces. I / O interface 1108 may include one or more devices for presenting output to a user, including but not limited to a graphics engine, display (e.g., a screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In some embodiments, I / O interface 1108 is configured to provide graphical data to a display for presentation to a user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may serve a particular implementation.

[0154] Communication interface 1110 may include hardware, software, or both. In any case, communication interface 1110 may provide one or more interfaces for communication (e.g., packet-based communication) between computing device 1100 and one or more other computing devices or networks. By way of example and not limitation, communication interface 1110 may include a network interface controller (NIC) or network adapter for communicating with Ethernet or other wired networks, or a wireless NIC (WNIC) or wireless adapter for communicating with wireless networks such as Wi-Fi.

[0155] Furthermore, communication interface 1110 can facilitate communication with various types of wired or wireless networks. Communication interface 1110 can also facilitate communication using various communication protocols. Communication infrastructure 1112 may also include hardware, software, or both, that couple components of computing device 1100 to each other. For example, communication interface 1110 can use one or more networks and / or protocols to enable multiple computing devices connected through a specific infrastructure to communicate with each other to perform one or more aspects of the processes described herein. For illustration, the digital content activity management process can allow multiple devices (e.g., client devices and server devices) to exchange information using various communication networks and protocols for sharing information such as digital messages, user interaction information, engagement metrics, or activity management resources.

[0156] In the foregoing description, this disclosure has been described with reference to specific exemplary embodiments thereof. Various embodiments and aspects of this disclosure have been described with reference to the details discussed herein, and various embodiments are illustrated in the accompanying drawings. The foregoing description and drawings are illustrative of this disclosure and should not be construed as limiting it. Numerous specific details have been described to provide a thorough understanding of various embodiments of this disclosure.

[0157] This disclosure may be implemented in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered illustrative in all respects only and not restrictive. For example, the methods described herein may be performed with fewer or more steps / actions, or the steps / actions may be performed in a different order. Furthermore, the steps / actions described herein may be repeated or performed in parallel with each other, or performed in parallel with different instances of the same or similar steps / actions. Therefore, the scope of this application is indicated by the appended claims, and not by the foregoing description. All modifications within the meaning and equivalent scope of the claims should be included within its scope.

Claims

1. A non-transient computer-readable medium operable to identify counterfeit products by storing instructions thereon, the instructions, when executed by at least one processor, causing a computing device to: In response to a third-party seller uploading one or more digital images of a product to an e-commerce digital environment, access is made from the actual seller to the actual graphic elements, which indicate the acceptable position, text style, text distortion, and text color for the actual graphic elements. The true graphic features for the true graphic elements are determined by using a detector to generate true shape features, using pixel mapping to generate true color features, and using a text detection machine learning model to generate true text features. The hash map is generated by mapping the acceptable positions, text style, text distortion, and text color within the hash map of the real text features; Extract graphic elements from the one or more digital images; Graphic features for the graphic elements are generated by using the detector to generate shape features, using the pixel mapping to generate color features, and using the text detection machine learning model to generate text features. The one or more digital images are analyzed to utilize a trained counterfeit detection model to determine, based on a hash of the true shape features, the true color features, and the true text features, including comparisons with the shape features, the color features, and the text features, to determine whether the one or more digital images uploaded from the third-party seller depict a counterfeit product. as well as Before publicly releasing the one or more digital images of the product into the e-commerce digital environment, the one or more digital images of the product from the third-party seller are marked as counterfeit.

2. The non-transient computer-readable medium of claim 1, further comprising instructions, which, when executed by the at least one processor, cause the computing device to extract the graphic elements from the one or more digital images by: The machine learning model, including an encoder and a decoder, is used to analyze the feature vectors of the one or more digital images and generate graphical element suggestions and suggestion confidence scores indicating bounding boxes around potential objects in the one or more digital images; and Based on the proposed confidence score, the graphic element is extracted from the proposed graphic element.

3. The non-transient computer-readable medium of claim 1, further comprising instructions, which, when executed by the at least one processor, cause the computing device to extract the graphic elements from the one or more digital images by: The system utilizes a machine learning model including an encoder and a decoder to generate feature maps of the one or more digital images, and also generates graphical element proposals and proposal confidence scores indicating bounding boxes around potential objects in the one or more digital images; and Determine whether the suggested confidence score meets the suggested confidence score threshold.

4. The non-transient computer-readable medium of claim 3, further comprising instructions that, when executed by the at least one processor, cause the computing device to: Align the actual graphic elements and the graphic elements in a common orientation plane; The bounding box of the real graphic element is scaled to the bounding box of the graphic element, wherein the real graphic element is in vector form. as well as A first similarity confidence score is generated by scaling the bounding box of the real graphic element to the bounding box of the graphic element and comparing a first list of pixel coordinates corresponding to the edge pixels of the real graphic element with a second list of pixel coordinates corresponding to the edge pixels of the graphic element.

5. The non-transient computer-readable medium of claim 1, further comprising instructions, which, when executed by the at least one processor, cause the computing device to generate a first similarity confidence score by comparing the true shape feature with the shape feature: Determine the shape difference threshold; The sum of shape differences is determined based on the difference between the shape representation and the true shape representation, wherein the true shape representation includes a first list of pixel coordinates corresponding to the edge pixels of the true graphic element, and the shape representation includes a second list of pixel coordinates corresponding to the edge pixels of the graphic element; and The first similarity confidence score is generated based on the sum of shape differences that satisfy the shape difference threshold.

6. The non-transient computer-readable medium of claim 1, further comprising instructions, which, when executed by the at least one processor, cause the computing device to generate a second similarity confidence score from comparing real color features with color features in such a way that: Access the true tonal range, true saturation range, and true luminance values ​​for the true pixel coordinates of the true graphic elements from the true seller; and The color features of the graphic element are compared with the true hue range, the true saturation range, and the true brightness value to generate a second similarity confidence score.

7. The non-transient computer-readable medium according to claim 1, further comprising: Based on the comparison between the hash graph, which includes the real shape features, the real color features, and the real text features, and the shape features, the color features, and the text features, a first similarity confidence score, a second similarity confidence score, and a third similarity confidence score are generated. Establish threshold similarity values ​​that include threshold shape similarity, threshold color similarity, and threshold text similarity. as well as It is determined that at least one of the first similarity confidence score, the second similarity confidence score, or the third similarity confidence score falls below the threshold similarity value.

8. The non-transient computer-readable medium of claim 7, further comprising instructions, which, when executed by the at least one processor, cause the computing device to adjust the threshold shape similarity value, the threshold color similarity value, or the threshold text similarity value by: Determine that the image quality of the graphic element or the real graphic element is below a resolution threshold, and adjust at least one of the threshold shape similarity value, the threshold color similarity value, or the threshold text similarity value; or Receive user input from a client device, and adjust at least one of the threshold shape similarity value, the threshold color similarity value, or the threshold text similarity value based on the user input.

9. The non-transient computer-readable medium of claim 1, further comprising instructions, which, when executed by the at least one processor, cause the computing device to generate the trained forgery detection model by: Generate prediction similarity confidence scores from training real graphical elements and training graphical elements; The prediction's veracity is generated from the prediction similarity confidence score; The accuracy of the predictions is compared with the true accuracy to generate a loss metric; as well as The parameters of the forgery detection model are adjusted based on the loss metric to generate the trained forgery detection model.

10. A method for identifying counterfeit products in a digital image, comprising: In response to a third-party seller uploading one or more digital images of a product to an e-commerce digital environment, access is made from the actual seller to the actual graphic elements, which indicate the acceptable position, text style, text distortion, and text color for the actual graphic elements. Identify authentic graphic features, including authentic shape features, authentic color features, and authentic text features for authentic graphic elements; The hash map is generated by mapping the acceptable positions, text style, text distortion, and text color within the hash map of the real text features; Extract graphic elements from the one or more digital images; Generate graphic features including shape features, color features, and text features for the graphic elements; The true shape features are compared with the shape features to generate a first similarity confidence score; The true color features are compared with the color features to generate a second similarity confidence score; The hash graph is compared with the text features for the graphic elements to generate a third similarity confidence score; Based on the determination that at least one of the first similarity confidence score, the second similarity confidence score, or the third similarity confidence score falls below a threshold similarity value, a trained counterfeit detection model is used to determine that the one or more digital images depict counterfeit products. as well as Before publicly releasing the one or more digital images of the product into the e-commerce digital environment, the one or more digital images of the product from the third-party seller are marked as counterfeit.

11. The method of claim 10, wherein extracting the graphic element from the one or more digital images comprises using a region proposal neural network to locate and identify the graphic element in the one or more digital images by: The encoder of the region suggestion neural network generates feature vectors for the one or more digital images; Using the decoder of the region suggestion neural network, graphical element suggestions and suggestion confidence scores are generated from the feature vector; as well as Based on the suggestion confidence score, the graphic element is extracted from the suggested graphic element.

12. The method of claim 10, further comprising: The machine learning model, which includes an encoder and a decoder, is used to generate feature maps of one or more additional digital images, and also to generate graphical element suggestions and suggestion confidence scores that indicate bounding boxes around potential objects in the one or more digital images. It is determined that the suggested confidence score does not meet the suggested confidence score threshold; as well as The additional one or more digital images are classified as unclassified.

13. The method of claim 10, wherein generating the first similarity confidence score comprises: Determine the shape difference threshold; The sum of shape differences is determined based on the difference between the shape representation and the true shape representation, wherein the true shape representation includes a first list of pixel coordinates corresponding to the edge pixels of the true graphic element, and the shape representation includes a second list of pixel coordinates corresponding to the edge pixels of the graphic element; as well as The first similarity confidence score is generated based on the sum of shape differences that satisfy the shape difference threshold.

14. The method of claim 10, wherein generating the first similarity confidence score comprises: Subtract the shape representation from the actual shape representation; Based on subtracting the shape representation from the true shape representation, the difference between the true graphic element and the shape or edge of the graphic element that satisfies the shape difference threshold is identified; as well as Based on the real graphic elements and the graphic elements satisfying the shape difference threshold, the first similarity confidence score is generated.

Images