Identifying placeholder images

Abstract

To provide a detection system of a more accurate and scalable placeholder image.SOLUTION: Identifying that an input image as a placeholder is accompanied with an image comparison part and a text comparison part based on known placeholder images. A text character group is extracted, and is tokenized to a plurality of input mage words being components of one or more phrases. A placeholder text coincidence score is generated from input image words evaluated on the basis of a word list for a placeholder text related to known placeholder phrases. A flag being a placeholder can be attached to the input image on the basis, at least in part, of the placeholder text coincidence score.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] This disclosure generally relates to the processing of images and text, and more particularly to the identification of placeholder images.

[0002] [Cross - Reference to Related Applications] None

[0003] [Description of Research and Development Funded by the Federal Government] None

Background Art

[0004] Online shopping generally includes consumers purchasing goods and services through the Internet. It continues to gain popularity due to its convenience, wider product options, and price advantages over traditional physical retail stores. By using web browsers and retail - specific applications available on smartphones, tablets, and other mobile devices, purchases can be made almost regardless of time and location. Payments are electronically processed almost instantaneously in various payment methods including credit cards, debit cards, gift cards, and also cryptocurrency. Direct delivery of goods to consumers' homes can also be done through various parcel delivery services.

[0005] Importantly, sellers can provide consumers with detailed information, and such information can better support the purchase decision - making process. Such information includes marketing materials from manufacturing companies or sellers, and is provided in the form of multimedia content such as 3D product models that can be viewed through interfaces for augmented reality, in addition to forms such as text, images, and videos. Furthermore, additional documents such as operation manuals and frequently asked questions from the manufacturing company can also be published. There may also be presented evaluations and reviews from other consumers.

[0006] Market segments are diverse, and the corresponding sellers are equally diverse, catering to the specific needs of each segment. Consequently, there are differences in business processes and online store structures among these sellers. For example, a small retailer may only sell in a specific niche market, but its sales volume may be sufficient to justify a dedicated online store. Other retailers with a larger commercial presence and a wider range of products may also exist. These retailers may manage their store websites in-house, typically allocating sufficient dedicated resources for maintaining and updating the entire online catalog content. This is because there is a direct correlation between sales / revenue and the usability, usefulness, and attractiveness of the website. These sites may utilize existing e-commerce platforms such as Shopify and Squarespace, which provide basic e-commerce functions, including product catalog creation, search, shopping carts, and payment processing.

[0007] Another type of online retailer exists, which, while relatively smaller in scale, can reach a wider customer base by utilizing online marketplaces. Rakuten is one of the well-known marketplaces, alongside Amazon, Walmart, and Target. While the specific names may vary depending on the marketplace, the basic concept is quite similar. Various third-party sellers manage lists of products they want to sell on the marketplace site, and customers can search for products they want to buy. There may be more than one seller for the same product, and once a customer selects a desired product, they can choose one based on price, shipping costs, delivery time, seller ratings, etc. The marketplace sends a purchase transaction to the third-party seller, who then fulfills the order and ships the product to the buyer.

[0008] While product catalogs may resemble those on an online marketplace site due to their adherence to graphic design standards, the content of the information displayed on a catalog page for a particular product may be the responsibility of the third-party seller providing that product. Therefore, third-party retailers may be tasked with providing product descriptions, various product images, and other multimedia content. Nevertheless, third-party sellers may have limited personnel and resources to update descriptions, and high-resolution images showing the product from multiple perspectives may not be available. Taking and editing these product images can be a time-consuming task for such companies without the necessary equipment. High-quality product images are essential for successful online sales, as they often serve as the sole, or at least primary, basis for a customer's purchase decision. Given this importance, the marketplace site's administration interface may require the upload of one or more product images before a catalog listing can be generated. Faced with delays in launching sales due to the availability of high-quality images, third-party sellers may choose to upload placeholder images.

[0009] Placeholder images are understood to include any image that does not accurately represent the product it is associated with. Placeholder images typically include images of one or more objects related to the product in question, such as images of a hammer, screwdriver, and pliers as a specific hand tool. Images may also include a disclaimer indicating that they are stock images not corresponding to the actual product, or that product images are "coming soon." Images may also include text explicitly stating that they are placeholders.

[0010] Regardless of their design and visual harmony with other website design elements, placeholder images detract from the purchasing experience and hinder the decision-making process. While this may lead to short-term sales increases for sellers due to the immediate availability of product listings and the generally wider reach of marketplace sites, in the long term it leads to decreased sales and, consequently, reduced revenue for both marketplace sites and third-party sellers. The negative impact of placeholder images has been examined in the context of third-party sellers uploading them to marketplace sites, but similar problems can also be seen in online stores / websites operated by retailers. Nevertheless, the need to identify and remove placeholder images is particularly pressing on marketplace sites due to the sheer volume and variety of images typically found in their catalogs.

[0011] Therefore, there is a need to detect placeholder images in this field. One known method divides placeholder images into two broad categories: those already identified in the product catalog, e.g., known placeholders, and those not yet identified, e.g., new placeholders. For known placeholder images, one of several known hash algorithms is applied to find other images with the same hash value. For unknown placeholder images, a machine learning model is used to determine whether the input image is a placeholder. This involves three steps: firstly, preparing 10 images for analysis for each product category; secondly, comparing the input image with all 10 images in that product category; and thirdly, determining that the input image is a placeholder if 5 out of the 10 product images are different from the product images.

[0012] Another known method aimed at detecting placeholder images is part of a broader process for selecting the best images for online catalogs. Unlike the placeholder detection methods mentioned above, there is no distinction between known and new placeholders. Instead, a binary image classifier is used to detect placeholder images. This model is understood to be trained on publicly available training datasets and built upon existing models. The model is then iteratively refined using manually prepared product images. [Overview of the Initiative] [Problems that the invention aims to solve]

[0013] Existing methods for detecting placeholder images remain inaccurate, and machine learning-based approaches require complex training procedures and large datasets. Therefore, there is a need in this field for more accurate and scalable placeholder image detection systems. [Means for solving the problem]

[0014] Placeholder images are images that do not contain meaningful content and are intended to be replaced when actual product images become available. It is desirable to flag images as placeholders, as this can improve the customer experience and revenue on e-commerce sites. Furthermore, it is envisioned that this will improve the accuracy of machine learning models based on accurate product images. Accurate and scalable placeholder image detection processes are envisioned according to various embodiments of this disclosure. Generally, placeholder images can be classified as images that are visually similar to known placeholder images, or images that contain some text indicating that they are placeholder images.

[0015] In one embodiment, when the input image is visually similar to a known example, it is envisioned that an image vectorization using a certain type of machine learning model and a vector comparison method for matching with the input image are employed. For images containing text that is likely to represent a placeholder image, optical character recognition (OCR) is used to extract the text from the image, and each word of the extracted text is compared with words in a known set of placeholder phrases / sentences to determine an overall match count or score and match spread. For the phrase that best matches and exceeds a predetermined score threshold, further evaluation of the match spread is performed to ensure that it fits within a predetermined maximum length. Evaluation of the unordered word count is also performed to ensure that it is within a predetermined range. A feedback loop can be implemented to extend the image matching-based capabilities of this process by incorporating newly identified placeholder images into the set of known placeholder images. The image processing workflow may be made more efficient by using a cache layer. Overall, embodiments of the placeholder image detection system can achieve an accuracy of approximately 95%.

[0016] According to one embodiment of the present disclosure, a method for identifying an input image as a placeholder includes evaluating the degree of matching of the input image to a known set of placeholder images. The method may also include extracting a set of text characters from the input image if the input image is evaluated as not matching. The method may also include tokenizing the set of text characters into a plurality of input image words that constitute one or more phrases. The method may include generating a placeholder text match score from the plurality of input image words evaluated based on a word list for placeholder text relating to known placeholder phrases. Each of the placeholder phrases may contain one or more known placeholder words. The method may also include flagging the input image as a placeholder based at least in part on the placeholder text match score. The methods described above can be implemented as a set of machine-readable instructions executed by a computer system, and such instructions are tangibly embodied in a non-temporary program storage medium.

[0017] Another embodiment of this disclosure may be a system for identifying placeholder images in a catalog. The system may include an image comparator that accepts an input image. A placeholder image matching score can be generated by the image comparator from the input image. The system may also include an optical character recognition engine that accepts an input image. The OCR engine can then output a set of text characters from the input image. The set of text characters can be ordered as a plurality of input image words that constitute one or more phrases. There may be a word tokenizer that groups the set of text characters into a plurality of input image words of one or more phrases. The system may further include a word list database for placeholder text having one or more known placeholder phrases. Each of the known placeholder phrases may contain one or more known placeholder words. There may also be a text comparator connected to the word list database for placeholder text. The text comparator may accept a plurality of input image words. A placeholder text matching score can be generated by the text comparator based on an evaluation of the plurality of input image words based on the word list database for placeholder text. Placeholder image recognition can be performed at least partially based on the placeholder text matching score.

[0018] These features and advantages and other features and advantages relating to the various embodiments disclosed herein will be better understood with reference to the following description and drawings. The same numbers in the drawings refer to the same parts throughout the drawings. [Brief explanation of the drawing]

[0019] [Figure 1A] This is an exemplary placeholder image of a company brand that may be displayed on an e-commerce platform site. [Figure 1B] This is an exemplary placeholder image of a company brand that may be displayed on an e-commerce platform site. [Figure 1C]An exemplary placeholder image of a corporate brand that can be displayed on an e-commerce platform site. [Figure 2A] An exemplary damaged image that can be displayed on an e-commerce platform site. [Figure 2B] An exemplary damaged image that can be displayed on an e-commerce platform site. [Figure 3A] An exemplary blank image that can be displayed as a placeholder on an e-commerce platform site. [Figure 3B] An exemplary blank image that can be displayed as a placeholder on an e-commerce platform site. [Figure 4A] An exemplary placeholder image that can be displayed on an e-commerce platform site and does not contain text content. [Figure 4B] An exemplary placeholder image that can be displayed on an e-commerce platform site and does not contain text content. [Figure 4C] An exemplary placeholder image that can be displayed on an e-commerce platform site and does not contain text content. [Figure 4D] An exemplary placeholder image that can be displayed on an e-commerce platform site and does not contain text content. [Figure 5A] An exemplary placeholder image that can be displayed on an e-commerce platform site and contains text content. [Figure 5B] An exemplary placeholder image that can be displayed on an e-commerce platform site and contains text content. [Figure 5C] An exemplary placeholder image that can be displayed on an e-commerce platform site and contains text content. [Figure 5D] An exemplary placeholder image that can be displayed on an e-commerce platform site and contains text content. [Figure 5E]This is an exemplary placeholder image containing text content that may be displayed on an e-commerce platform website. [Figure 5F] This is an exemplary placeholder image containing text content that may be displayed on an e-commerce platform website. [Figure 5G] This is an exemplary placeholder image containing text content that may be displayed on an e-commerce platform website. [Figure 6A] This is an example product image containing text content that could be confused with a placeholder. [Figure 6B] This is an example product image containing text content that could be confused with a placeholder. [Figure 6C] This is an example product image containing text content that could be confused with a placeholder. [Figure 6D] This is an example product image containing text content that could be confused with a placeholder. [Figure 6E] This is an example product image containing text content that could be confused with a placeholder. [Figure 7A] This is an exemplary placeholder image that includes graphic content that may be confused with non-placeholders, but also includes text content that indicates a placeholder. [Figure 7B] This is an exemplary placeholder image that includes graphic content that may be confused with non-placeholders, but also includes text content that indicates a placeholder. [Figure 7C] This is an exemplary placeholder image that includes graphic content that may be confused with non-placeholders, but also includes text content that indicates a placeholder. [Figure 7D] This is an exemplary placeholder image that includes graphic content that may be confused with non-placeholders, but also includes text content that indicates a placeholder. [Figure 7E]This is an exemplary placeholder image that includes graphic content that may be confused with non-placeholders, but also includes text content that indicates a placeholder. [Figure 8] This is a block diagram roughly illustrating the components of an exemplary e-commerce platform in which the envisioned embodiments of the placeholder image recognition system of this disclosure may be used together. [Figure 9] This is a flowchart illustrating an exemplary embodiment of the placeholder image detection process. [Figure 10] This is a block diagram of one embodiment of the placeholder image detection system according to the present disclosure. [Figure 11] This flowchart shows an embodiment of a method for identifying an input image as a placeholder. [Figure 12A] This diagram shows a set of individual Latin alphabet characters recognized by an optical character recognition engine. [Figure 12B] This diagram represents a tokenized word derived from a phrase composed of a series of individual Latin alphabet letters. [Figure 13A] This diagram shows the individual groups of Chinese characters recognized by the optical character recognition engine. [Figure 13B] This diagram represents a tokenized word derived from a phrase composed of a series of individual Chinese characters. [Figure 14] This flowchart shows the cache layer implemented in relation to the placeholder image detection process. [Modes for carrying out the invention]

[0020] This disclosure relates to various embodiments of a method and system for identifying placeholder images. The detailed description below, in relation to the accompanying drawings, is intended to describe several currently conceivable embodiments and is not intended to represent the only form in which such embodiments may be developed or utilized. This description describes the functions and features in relation to the embodiments shown. However, it should be understood that the same or equivalent functions may also be achieved by other embodiments that are similarly intended to be included in the scope of this disclosure. Furthermore, the use of relational terms such as “first,” “second,” etc., is used solely to distinguish one entity from another and is not intended to imply, nor does it necessarily imply, that there is any actual relationship or order between such entities.

[0021] Placeholder images can be displayed or used in a variety of information presentation contexts, but are typically found in online e-commerce platforms or websites that include catalogs of goods and services offered for purchase. Available inventory of cataloged goods on the site is grouped by category or can be viewed via search queries, with further details provided on the corresponding catalog page for the selected product. Catalog entries can be entered along with text and images containing information about the product. As mentioned above, the management of such text and image data may be the responsibility of the e-commerce site owner's staff, third-party vendors of goods using the e-commerce site as a sales platform, manufacturers or distributors, or any other party in the supply chain. In some situations, if a product image is unavailable for any reason, it may be replaced with a placeholder image when presenting a catalog page or its preview to visiting customers.

[0022] Figures 1A-1C, 2A and 2B, 3A and 3B, 4A-4D, and 5A-5G show various examples of placeholder images. In particular, Figures 1A-1C show placeholder images 1a-1c, which are various company logos. Figure 1A shows a placeholder image using the assignee's company logo 1a, "Rakuten." Another logo, shown in Figure 1B, is logo 1b, which can be used to identify the service "kobo." Figure 1C shows a text-free graphical logo 1c for a product or service (e.g., Viber) offered by the same assignee. Figures 2A and 2B show placeholder images 2a and 2b, respectively, with small thumbnails 2a-1 and 2b-1 representing the corrupted images shown in virtual frames 2a-2 and 2b-2. This is necessary because otherwise they would not be visible against the white background. Figures 3A and 3B show blank placeholder images 3a and 3b, respectively. These images are single-color blocks containing no other content. The border surrounding the white block of blank placeholder image 3a is shown solely for the purpose of defining that border. Such borders are not typically displayed in a browser.

[0023] Figures 4A to 4D each show other types of placeholder images 4a to 4d, which clearly indicate and communicate the absence of an image. For example, placeholder image 4a incorporates multiple generic image representations. Placeholder image 4b ​​shows a mountain landscape with diagonal lines to indicate that no image is available. Similarly, placeholder image 4c shows a generic portrait image with diagonal lines to indicate that no image is available. Placeholder image 4d includes a camera icon enclosed in a circle with diagonal lines crossing the circle to indicate that no image (represented by a camera) is available.

[0024] Figures 5A to 5G show variations of different types of placeholder images 5a to 5g, each containing text indicating that the image is a placeholder. Figure 5A shows placeholder image 5a containing Japanese characters 6a that read "GAZOUNASHI", which can be translated as "No image". Figure 5B shows placeholder image 5b containing English characters 6b that make up the word "No Image", followed by Japanese characters 7b that make up the phrase "TADAIMAGAGOSEISAKUCHUDES", which can be translated as "Image is currently being created". The identification name 8b of the website or its operating company can be included in another part of placeholder image 5b. Figure 5C shows placeholder image 5c containing Japanese characters 6c that make up the phrase "GAZOUNBICHU", which can be translated as "Image is being prepared".

[0025] Such phrases indicating that an image is unavailable may be shown in English text only. Figure 5D shows such a placeholder image 5d, which contains the phrase 6d "Sorry Image Not Available," where the word "Sorry" 6d-1 is displayed in a script font and "IMAGE NOT AVAILABLE" 6d-2 is displayed in a sans-serif block font. This example shows that different parts of a phrase can be displayed in different visual styles, such as the word "IMAGE" 6d-2 being displayed in a larger font size than "NOT AVAILABLE" 6d-2. Figure 5E is another placeholder image 5e with the phrase "Image coming soon" as a group of letters 6e on an image background 7e. Figure 5F shows a placeholder image 5f with the words "No Image Currently Available" 6f below a book icon 7f. This example may be used in the context of a product listing of a book or other printed material represented by the icon 7f, but only when neither the actual image of the book cover nor any other image showing the actual appearance of the product is available. Figure 5G shows a placeholder image 5g containing the word 6g "Cover Coming Soon". This example illustrates that the specific wording may change, and the placeholder image does not necessarily need to contain the word "image". Here, the word "cover" is used to indicate that this image is originally a book cover image, and the appropriate image is "coming soon".

[0026] Figures 6A to 6E are exemplary images 9a to 9e, each containing text content that could be confused with placeholder images. Image 9a in Figure 6A contains the text 10a, for example, "new menu coming soon." Image 9b in Figure 6B could be the cover of a book titled "The Book With No Pictures." The phrase "No Pictures" 10b could be confused with typical placeholder text that may also represent "No Picture" or some derivative expression thereof. Similarly, Image 9c in Figure 6C could be the cover of a book titled "He Had No Image," and the word "No Image" 10c could be confused with the same text that may be found in a placeholder image. Image 9d in Figure 6D could be the cover of a book containing the text 11d "COMING SOON: THE FLOOD a novel Zvi Jagendorf," and the phrase "coming soon" 10d contained therein could be confused with the phrase in a placeholder image. Figure 6E shows the product more accurately represented in Image 9e, and the phrase "Coming soon!" may mean that the product will be available soon, not that the product image will be displayed soon.

[0027] Figures 7A to 7E are exemplary images 12a to 12e that may appear to depict a specific product. Image 12a in Figure 7A shows a single component or product that may have varying degrees of relevance to the product being sold. This relevance may be broad, such as being related at a category level, or it may be as specific as a competing product. However, image 12a includes text 13a indicating that the image is “Not actual part.” Image 12b in Figure 7B shows multiple items, including gears, belts, bearings, and gaskets, and it may be clear that this image does not correspond to a specific product. There is also overlay text 13b indicating that it is a “temporary placeholder.” Similarly, image 12c in Figure 7C includes the corporate logo of a seller entity along with images of a brake rotor and spring. In this case, the context of these depictions may suggest that this image does not correspond to a specific product, and it also includes text 13c that reads “Actual Image Unavailable.” Image 12d, shown in Figure 7D, is very similar to Image 12c in that the seller entity's company logo is seen alongside an image of an automobile exhaust system part. Here again, this situation may suggest that the image is not of any specific product, and the text 13d, "Stock Photo - No Direct Image for this part," supports this. However, including related images in these Images 12c and 12d could lead to confusion and a false impression that they represent the product. Image 12e, shown in Figure 7E, is a single item of furniture. Because only one item is shown, this image could be confused with an image of the actual product. That said, this image also includes the text 13e, "Placeholder Only."

[0028] As shown in the block diagram of Figure 8, the exemplary e-commerce platform 20 may include one or more server computer systems that are connected to the internet and communicate with remote client computer systems 22 to exchange data. One such computer system may be a web server 24 that receives various information requests through a browser application running on the client computer system 22. In response, the web server 24 can retrieve the requested information and send it back to the client computer system 22 for rendering and display. In the case of the e-commerce platform 20, this could be product catalog information. This may include thousands to millions of individual records corresponding to products offered for sale on this platform.

[0029] This information can be stored in a product catalog database 26, which consists of one or more product records 28. In the illustrated example, a product record 28 may include an identifier field 28a, a name field 28b, a description field 28c, and one or more image fields 28d. Depending on the embodiment, data files representing (multiple) images can be stored in the product catalog database 26 or in another image database 30. In the former case, the image field 28d may contain image data, or in the latter case, the image field 28d may instead have a reference to a record in the image database 30 that stores the image data.

[0030] The structural relationships shown for the product catalog database 26, the product records 28 contained therein, and the image database 30 represent a significantly simplified representation of a typical e-commerce platform. This is presented as one possible example of structuring various underlying components used in combination to provide online shopping services to customers, and as a way in which the placeholder image recognition system 36 can be adapted to the overall environment of the e-commerce platform 20. Similarly, the specific fields 28a-28d of the product records 28 are also illustrative, and depending on the embodiment, there may be more or fewer fields than those shown. The structure of the record fields may suggest a relational database defined with respect to a table where each record is a row and each field is a column, but this is also presented as an example only. The product catalog database 26 can be implemented in various ways known in the art, and the details shown are not intended to represent the only way in which the e-commerce platform 20 can be constructed.

[0031] Although not specifically indicated, there are numerous other components that are part of the e-commerce platform 20. However, since the implementation of such components is within the scope of the skills of those skilled in the art, it is understood that their additional details are omitted. In line with this idea, the web server 24, the product catalog database 26, and the image database 30 may be implemented in one or more computer systems, each computer system including a general-purpose processor capable of executing pre-programmed instructions, one or more types of memory for storing data and instructions, and a communication method that allows data to be exchanged between computer systems. To provide services to a large number of users and ensure high availability, the e-commerce platform 20 may be implemented using additional bare-metal hardware, load balancers, additional network connectivity, and other redundancies as a cluster. Therefore, the individual representations of the web server 24, the product catalog database 26, and the image database 30 are intended to show each functional element and its general classification, rather than individual hardware and software units.

[0032] The primary purpose of the e-commerce platform 20 is for customers to access it and make purchases, but another aspect is that it is also used by sellers to provide goods for sale. Where used herein, a seller may refer to a third-party seller 32, which is a separate business entity from the business entity operating the e-commerce platform 20, as well as the business entity operating the e-commerce platform 20 or its closely related companies. Therefore, a third-party seller 32 may access an administrator interface to modify product records 28 to list the goods offered for sale. In addition to product records 28, there may also be a system administrator 34 who can modify other functions of the e-commerce platform 20. In the context of this disclosure, if product images are unavailable but are required to complete the product record 28 and begin sales, placeholder images may be uploaded by such third-party sellers 32 and system administrators 34. These placeholder images, which may take the form of the examples described above, may be stored in an image database 30. The placeholder image recognition system 36 is intended to flag such placeholder images in order to delete them or to update them with actual product images.

[0033] The flowchart in Figure 9 illustrates one embodiment of the placeholder image detection process according to the present disclosure. Broadly speaking, this process can be divided into two subparts: one being an image comparison portion 40, and the other being a text evaluation portion 42. The image comparison portion 40 generally involves comparing an input image 44 with an existing set of known placeholder images. Meanwhile, the text evaluation portion 42 involves extracting text from the input image and determining whether such text is likely to indicate that the image from which the text was extracted is a placeholder image. The block diagram in Figure 10 illustrates one possible embodiment of the placeholder image identification system 36, which is similarly divided into an image comparison unit 140 and a text evaluation unit 142. These subparts and components will be described in detail in turn. The present disclosure further envisions a method for identifying an input image as a placeholder, the steps of which are shown in the flowchart in Figure 11. Each step of this method can correspond to a specific aspect of the placeholder image detection process described in relation to the flowchart in Figure 9, as well as to a specific component of the placeholder image recognition system 36. This method can be tangibly embodied in a non-temporary program storage medium product as one or more programs relating to instructions executable on a computing device, for example, a program for the e-commerce platform 20.

[0034] In the placeholder image detection process, the image comparison unit 40 generally compares the input image 44 with a set of known placeholder images 46. This corresponds to step 110 in the method for identifying the input image shown in the flowchart of Figure 11, which evaluates the degree of match between the input image 44 and a set of known placeholder images 46. Using a machine learning model, both the input image 44 and the known placeholder images 46 can be converted into numerical vectors, and then it can be determined whether they match based on a similarity score. The block diagram of Figure 10 shows the image comparison unit 140 having an image comparator 148. The image comparator itself includes a vectorizer 150 and a scorer 152. The image comparator 148 communicates with an image search index 54 that can store the numerical vector values ​​of the known placeholder images 46. Needless to say, the conversion of the known placeholder images 46 to numerical values ​​is performed only once, and such a conversion is performed by components other than the vectorizer 150 shown in Figure 10. According to various embodiments of this disclosure, the vectorizer 150, which is part of the overall placeholder image recognition system 36, operates only on the input image 44.

[0035] In the block diagram, the image search index 54 is shown as being outside the image comparator 148, but this is for illustrative purposes only and is not limiting. In certain embodiments, the image search index 54 may be logically grouped within the image comparator 148. As a general principle applicable to other components and functions of the placeholder image recognition system 36 disclosed herein, the inclusion of a component in another larger component or component class / group is shown only as an example. Those skilled in the art will recognize that certain components may be separated from such a component class or group, or included as part of a different component class or group.

[0036] The known placeholder images 46 are understood to be consistent with the description above, and these images are shown in Figures 1A to 4D. Furthermore, to the extent that more unique placeholder images, such as those shown in Figures 5A to 5G, are commonly found on the e-commerce platform 20, such images may also be identified as known and provided to the machine learning model for image-based comparison. As shown in the flowchart in Figure 9, the known placeholder images 46 are subjected to a vectorization process 50a. Here, the dataset representing the images is converted into a set of known placeholder image vector values ​​51a.

[0037] The image is provided as an array of bitmap pixel values ​​organized according to rows and columns, and the vectorization process is understood as converting such an image into a set of values ​​representing geometric primitives such as points, lines, curves, and polygons. A machine learning module may be used for the vectorization process 50a, one possible embodiment being a convolutional neural network (CNN). However, any other suitable machine learning model may be substituted without departing from the scope of this disclosure. In the context of the placeholder image recognition system 36, this vectorization process may be performed by a vectorizer 150. Known placeholder image vector values ​​51a of known placeholder images 46 are stored in the image search index 54. The image search index 54 may be part of a flexible search module or it may be a commercially available off-the-shelf system such as Facebook AI Similarity Search (FAISS).

[0038] The image comparator 148 receives the input image 44, and the image comparison unit 40 of the placeholder image detection process includes a vectorization step 50b using the input image 44. This step can be performed by the vectorizer 150 of the image comparator 148. As a result, an input image vector value 51b is generated, and this value is queried from the image search index 54. In the context of the method for identifying the input image as a placeholder, as shown in the flowchart of Figure 11, this corresponds to step 109, which converts the input image 44 into an input image vector value 51b.

[0039] A known placeholder image vector value 51a is compared with the input image vector value 51b, and the scorer 152 calculates a placeholder image match score 53 in step 52. In one embodiment, cosine similarity may be used for the placeholder image match score 53. A method for identifying an input image as a placeholder may include the corresponding step 110-2, which generates a placeholder image match score 53 from a query to an image search index 54 using the input image vector value 51b.

[0040] If the calculated placeholder image matching score 53 is 0.94 or greater according to the comparison step 56, the input image 44 is considered to have a positive match to the known placeholder image 46 according to step 58 and is flagged accordingly. The specific value of 0.94 is shown for illustrative purposes only and may be changed depending on the specific content of the embodiment. The threshold matching score 53 may also change with each iteration as a parameter passed to the comparison function. This is to continue searching for a more appropriate image match after the first match identified. This flagging step may include updating a related record field indicating that the corresponding image is a placeholder, or updating any other step for a particular implementation that generally indicates that the target image is a placeholder. There may be a corresponding step 110-3 which flags the input image 44 as a placeholder based on the placeholder image matching score 53. The general concept of using machine learning to identify similar images is known in the art, and therefore the image comparison unit 40 described above may be implemented using any other suitable machine learning-based image comparison system or other image comparison system. One such known conventional image comparison system is the Scale Invariant Feature Transform (SIFT).

[0041] According to one embodiment, after the input image 44 is deemed to be a match according to step 58, there may be a manual verification step 59 where the input image 44 is confirmed as a placeholder. This feedback may be used to further update the image search index 54 for subsequent iterations.

[0042] If the calculated placeholder image matching score 53 is evaluated as less than 0.94 in the comparison step 56, the image comparison unit 40 considers that no result was found. Steps 110-2, which generate the placeholder image matching score 53, and 110-3, which flag the input image as a placeholder, form a loop. In this loop, the vector value of the input image 44 is compared with each of the vector values ​​of known placeholder images in the image search index 54, and this is repeated until one or more placeholder image matching scores 53 exceed a threshold. If there are no known placeholder images 46 that match the input image 44, it is considered that no result was found, and the process proceeds to the text evaluation unit 42. In the context of the placeholder image recognition system 36, the text evaluation component 142 processes the input image 44 using text extraction and text matching processes. This will be described in more detail below.

[0043] One of the text evaluation components 142 is an optical character recognition engine 160 that scans the input image 44 and converts it into machine-encoded text data. Referring to the flowchart in Figure 9, the process proceeds to the OCR preparation step 60. This step may include various OCR preprocessing steps, such as deskewing, binariization, line removal, and character separation / splitting. The OCR engine 160 also performs the following English text extraction step 62a. In this regard, the method of identifying the input image as a placeholder may include extracting a set of text characters from the input image 44 as the corresponding step 112. This is done in response to the evaluation by the image comparison unit 40 described above that the input image 44 does not match. Computational processing for character recognition is well known in the art, and various methods exist. Therefore, the OCR engine 160 may implement any one of these methods, but embodiments of the present disclosure are not limited to any particular method.

[0044] Optical character recognition processing can include extracting sets of characters from an input image. Figure 12A shows a set of individual characters 66a-66v extracted from the exemplary placeholder image 5d "SORRY IMAGE NOT AVAILABLE" shown in Figure 5D. This string of text characters 66 constitutes a phrase 68, which can then be tokenized into a set of input image words 70 that are its constituent elements. Figure 12B shows the tokenization. Here, the first character 66a, the second character 66b, the third character 66c, the fourth character 66d, and the fifth character 66e are tokenized as word 70a "SORRY", the sixth character 66f, the seventh character 66g, the eighth character 66h, the ninth character 66i, and the tenth character 66j are tokenized as word 70b "IMAGE", the eleventh character 66k, the twelfth character 66l, and the thirteenth character 66m are tokenized as word 70c "NOT", and the fourteenth character 66n, the fifteenth character 66o, the sixteenth character 66p, the seventeenth character 66q, the eighteenth character 66r, the nineteenth character 66s, the twenty-tenth character 66t, the twenty-first character 66u, and the twenty-second character 66v are tokenized as word 70d "AVAILABLE".

[0045] The tokenization process 72a may be based on spaces 67 between character groups, and the placeholder image recognition system 36 is understood to include a word tokenizer 172 that performs this process. In the example shown in Figure 12A, the first enlarged space 67a is shown between the fifth character 66e and the sixth character 66f, thereby separating the first word 70a "SORRY" from the second word 70b "IMAGE". Similarly, a second space 67b between the tenth character 66j and the eleventh character 66k separates the second word 70b "IMAGE" from the third word 70c "NOT". A third space 67c between the thirteenth character 66m and the fourteenth character 66n separates the third word 70c "NOT" from the fourth word 70d "AVAILABLE". Once tokenized, stop words may be removed. Because this is a relatively trivial process, it is considered part of the English text extraction process 62a. A method for identifying an input image as a placeholder includes, as a corresponding step 114, tokenizing a set of text characters into multiple input image words that constitute one or more phrases.

[0046] If necessary, there may be an English spelling correction step 74. This is because errors may occur during the text extraction process. Spelling correction may include comparing each of the tokenized words 70a–70d with the English vocabulary list 75 and making appropriate corrections. Similar to optical character recognition, spell checking can be implemented in various ways, but since such differences are considered to be within the scope of the skill of those skilled in the art, additional details regarding this are omitted.

[0047] The detection of placeholder images provided in a foreign language is also assumed, and there may be a foreign language text extraction step 62b in parallel. Such foreign languages may include, in addition to foreign languages that use the Latin alphabet (e.g., Spanish, French, etc.) and foreign languages that use Chinese characters (Standard Chinese, Japanese, etc.), any other character sets (e.g., Hangul / Korean, Arabic, Thai, etc.). For the purpose of illustration, in the following description of the embodiments of the text evaluation unit 42 and the text evaluation component 142, Japanese will be described as an example of a foreign language. Needless to say, it may be replaced with any other foreign language, and those skilled in the art can adapt it to such another foreign language by making appropriate modifications to the text evaluation unit 42 and the text evaluation component 142.

[0048] Optical character recognition processing includes a foreign language segmentation step 72b that divides the detected foreign language characters into a plurality of words, similar to the English segmentation step or tokenization process 72a described above. Generally, it also corresponds to the tokenization step 114 of the method shown in the flowchart of FIG. 11. This step can be performed by a properly configured word tokenizer 172. FIG. 13A shows an example of foreign language characters included in the placeholder image 5c shown in FIG. 5C. These characters form the phrase "画像準備中", which means "preparing an image". The OCR engine 160 recognizes each of the first character "画" 76a, the second character "像" 76b, the third character "準" 76c, the fourth character "備" 76d, and the fifth character "中" 76e individually as a continuous string of Chinese characters 76, but it is understood that it is not necessary to recognize them as words forming a phrase. Referring further to FIG. 13B, the text evaluation unit 42 proceeds to a foreign language or Japanese segmentation process 72b that tokenizes the phrase 78 into its components. The components include the first word "ガゾウ" 80a ("image"), the second word "ジュンビ" 80b ("preparation"), and the third word "チュウ" 80c ("in progress"), which can be combined to translate as "preparing an image" or "image preparation in progress".

[0049] As long as the input image 44 contains both English and Japanese / foreign language, the placeholder image detection process assumes a step 82 that combines an English word 70 with a Japanese / foreign language input image word 80, which could be the placeholder image 5b shown in Figure 5B. Therefore, all possible words suspected to likely suggest that the input image 44 is a placeholder may be evaluated regardless of language. The word tokenizer 172 can perform this step and output a set of input image words 84 consisting of the English word 70 and the Japanese / foreign language input image word 80 extracted from the input image 44. Here again, the ability to process text in multiple languages ​​is shown only as an example, and there may be other embodiments that extract text in only one language. In general, multiple input image words 84 are understood to refer to text data extracted from the input image 44, regardless of their language (may be multiple) and characters.

[0050] Next, the text evaluation unit 42 for placeholder image detection proceeds to step 86, which is performed by the text comparator 186 to obtain a text match score. Generally, the input image word 84 is evaluated based on known placeholder phrases 88 that are usually included in placeholder images. For example, these phrases include phrases such as "Sorry, image not available" in placeholder image 5d, "image coming soon" in placeholder image 5e, and "no image currently available" in placeholder image 5f, as well as foreign language or Chinese character phrases such as "No image" shown in placeholder image 5a, "Image currently being created" shown in placeholder image 5b, and "Image being prepared" in placeholder image 5c. These known placeholder phrases consist of individual placeholder words such as "no," "image," "unavailable," "coming," and "available," and are divided into such multiple words in the splitting step 90 before being stored in the placeholder text word list database 92.

[0051] The input image 44, specifically the multiple input image words 84 contained within it, are matched with these known placeholder phrases 88, and a placeholder text matching score 194 is generated, which quantifies the degree of matching. This generally corresponds to step 116 in the method for identifying input images as placeholders, as shown in the flowchart of Figure 11, where the degree of matching of input image phrases is evaluated based on the placeholder text word list. As a further example illustrating the step of obtaining a text matching score, the extracted input image word may be "sorry this image is not yet available Rakuten shopping". The first phrase among the known placeholder phrases 88 may be "product image not available", and the second phrase among the known placeholder phrases 88 may be "image is unavailable".

[0052] For a specific phrase or input image word 84, the matching process begins with the step of determining a match count value 194 and a match phrase spread 196. These values ​​can be generated by a text comparator 186, and this process can correspond to step 116-1, which generates the match count value. The match count value can be generated for one or more phrases. In one embodiment, this can be based on the number of input image words contained in a particular one of several phrases that match known placeholder words in the placeholder text word list database 92. There may also be a correspondence between step 116-3, which generates a match spread value between the first input image word that matches one of the known placeholder image words and the last input image word that matches another known placeholder word, both belonging to the same known placeholder phrase (or known placeholder word list). In this specification, the match count is the number of input image words 84 found in and common to a single placeholder text word list stored in the database 92. The match phrase spread is understood to be the word length between the first and last match word. Using scores and spreads is assumed to be a further rule applied to the input text to determine a match with one of the known placeholder phrases 88, which goes beyond simple text matching. Continuing with the previous example where the first known placeholder phrase is "product image not available," a match count value of 3 is obtained because the words "image," "not," and "available" are considered to match between the first known placeholder phrase and the input image word 84. The match spread 196 is understood to be 5, for example, in "image is not yet available," because there are a total of 5 words between the first match words "image" and "available."

[0053] The known placeholder phrase 88 with the highest match count, for example, the first phrase, is selected, and a placeholder text match score 194 is calculated. In one embodiment, the placeholder text match score 194 is the match count divided by the length of one of the selected known placeholder phrases 88, and can similarly be generated by the text comparator 186. In general, the placeholder text match score 194 should be understood as being derived as a whole from one of the selected input image words 84 and the phrase defined by it. This generally corresponds to step 116-2 in the method for identifying an input image as a placeholder, as shown in the flowchart of Figure 11, in obtaining the placeholder text match score 194. Using the example above again, the match count is 3. Meanwhile, the length of one of the selected known placeholder phrases 88 is 4, i.e., product(1), image(2), not(3), available(4). The placeholder text match score 194 is 3 / 4, i.e., 0.75. According to one embodiment, the threshold score for determining a match may be 0.75 or higher. However, this is merely an example. By reason, a placeholder text match score of 194 less than 0.75 is considered not to match. Any other suitable threshold value may be substituted without departing from the scope of this disclosure. This evaluation based on a given threshold may be performed in determination block 96.

[0054] In step 86, which obtains the match score, there may be an additional step 100 in which the calculated placeholder text match score 194 is evaluated based on a threshold, and then the match spread is determined. This generally corresponds to step 116-3 in the method of identifying input images as placeholders, as shown in the flowchart of Figure 11, which generates the match spread value. Specifically, the match spread is checked based on the length of one selected from the known placeholder phrases 88, and if the match spread is more than twice the length of the known placeholder phrase, these input image words 84 are evaluated as not matching. Continuing with the example above, the match spread is 5, for example, image(1), is(2), not(3), yet(4), available(5). Also, the length of one selected from the known placeholder phrases 88 is 4, namely product(1), image(2), not(3), available(4). Twice the length of one of the 88 known placeholder phrases selected is 8, and 5 (match spread) is less than 8. Therefore, this match is considered valid.

[0055] The aforementioned evaluation or check, based on the length of one selected from the known placeholder phrases 88, may be performed after comparing the placeholder text match score 194 to a provisional threshold, or at least after the placeholder text match score 194 has been generated. In either case, the specific action taken in response to a failed check may be to flag the input image 44 as non-placeholder, according to step 118 of the method shown in the flowchart of Figure 11. The order of operations is not intended to be limiting in any case.

[0056] There may be another evaluation step 102 that checks the out-of-order rate of multiple input image words 84 that match one selected from the known placeholder phrases 88. This generally corresponds to step 116-4, which generates the out-of-order rate in the method of identifying input images as placeholders, as shown in the flowchart of Figure 11. In a pair of two phrases, for example, one has one selected from the known placeholder phrases 88 and the other has multiple input image words 84, the number of words that are out of order is defined as the out-of-order count. As one example, for the phrases [image not available] and [image available not], the out-of-order count is 1. This is because the word "image" is in the same position in both phrases, but the order of the following two word pairs, "available" and "not," is not the same. As another example, for [image not available] and [available image not], the out-of-order count is 2. This is because the pairs of words, "image" and "not," are not in the same order, and the pairs of words, "available" and "not," are also not in the same order. Next, the disorder ratio is determined by dividing the disorder count by the length of each phrase being compared. If the disorder ratio is 0.5 or greater, the phrases are not considered to be identical. In the first example above, comparing [image not available] and [image available not], the disorder count is 1 and the length is 3, so the disorder ratio is 0.333. In other words, the two phrases are considered to be identical. However, in the second example above, comparing [image not available] and [available image not], the disorder count is 2 and the length is 3, so the disorder ratio is 0.6667. In other words, the two phrases are considered to be identical.

[0057] The evaluation or check of the order inconsistency rate described above may also be performed after comparing the placeholder text match score 194 to a provisional threshold, after an evaluation based on the match spread value 196, or at least after the placeholder text match score 194 has been generated. In any case, the specific action taken in response to a failed check may be to flag the input image 44 as not being a placeholder, according to step 118 of the method shown in the flowchart of Figure 11. The order of operations is not intended to be limiting in any case.

[0058] A positive text match result 104 is obtained if the placeholder text match score 194 is rated 0.75 or higher, the match spread is rated to be 2 times or less the length of the known placeholder phrase according to step 100, and the orderlessness rate is 0.5 or less. Otherwise, a negative text match result 106 is obtained for one specific of the known placeholder images 88. It should be understood that as long as there is at least one positive match result, the input image 44 is considered a placeholder. The text comparator 186 is understood to generate a placeholder text match score 194 from the evaluation of multiple input image words 84 based on the word list database 92 for placeholder text, and placeholder image recognition is performed at least in part on the placeholder text match score 194. The placeholder image recognition step can correspond to step 116-5, which flags the input image in a way that identifies it as a placeholder. According to one embodiment, scoring and verification can be performed by the text comparator 186. Referring again to the block diagram in Figure 10, various scores and indicators regarding the matching of multiple input image words 84 with known placeholder phrases 88 can be evaluated by the text comparator 186, and the placeholder image is identified 108 as either a positive text match result 104 or a negative text match result 106. However, instead, these evaluations and the final flagging of the input image 44 as a placeholder may be performed by the score evaluation and verification unit 124.Step 116 evaluates matching of input image phrases based on a word list 88 for placeholder text, including constituent steps such as step 116-5 for flagging input images as placeholders, in addition to step 116-1 for generating a match count value, step 116-2 for obtaining a placeholder text match score, step 116-3 for generating a match spread value, and step 116-4 for generating an order inconsistency rate. Step 118 for flagging input images as non-placeholders based on the match spread value and order inconsistency rate loops to compare multiple input image words 84 of a single input image 44 with each of the known placeholder phrases 88. The flagging steps 118, 116-5 are understood to be applicable to only one specific known placeholder phrase 88 in question and do not affect matches already identified. This loop can be terminated once a match has been identified. In general, the text evaluation unit 42 is intended to identify one good match and identify the target input image 44 as a placeholder.

[0059] According to one embodiment, a positive text match result 104 may be confirmed in a manual verification step 126, and the input image 44 may be added to the image search index 54 as a known placeholder image 46 after vectorization. Additional duplicate identification processing may be performed to prevent multiple instances of the same placeholder image from being stored in the image search index 54. Alternatively, or in addition to that, multiple input image words 84 may be added to the word list database 92 for placeholder text as one of the known placeholder phrases 88 after a splitting step 90 that follows the same manual verification step 126.

[0060] Referring again to the block diagram in Figure 8, the e-commerce platform 20 can continuously utilize the placeholder image recognition system 36, which flags images linked to pages provided by the web server 24. A cache layer 130 may exist to reduce the number of unnecessary calls to the placeholder image recognition system 36. In addition to using the URL (uniform resource locator) of a specific image included in the output file corresponding to the provided web page and generated by the web application server, the image itself is also used to create a hash code, which then serves as the cache key. A typical record of the placeholder image recognition 108 may include a key value such as "000019690543e1222adbc6cdfe46ef9f" along with related values ​​such as the following.

number

[0061] Thus, it is understood that the cache entry for a particular image record includes not only a placeholder text match score of 194, but also a boolean flag indicating whether the image in question is a placeholder.

[0062] Referring further to the flowchart in Figure 14, one possible embodiment of the caching process includes a step 200 of parsing the input file 132 output from the upstream pipeline. Next, it is determined whether the URL of the image referenced in the input file 132 exists by performing a lookup in the placeholder cache 134. If a record is evaluated in the determination block 202, the corresponding placeholder image identifier 108 for that image is retrieved from the placeholder cache 134 and passed to the next step in the data pipeline 136. If no entry is found in the placeholder cache 134, the process proceeds to step 204 of downloading the image referenced in the input file 132. A lookup of the hash code of the downloaded image is performed again against the placeholder cache 134, and if the existence of an image record is confirmed in the determination block 202, the process proceeds to the next step in the data pipeline 136.

[0063] If the image hash value is also not found in the placeholder cache 134, the process proceeds to step 206, which calls the placeholder image recognition system 36. The image URL and the image hash code value, along with the resulting placeholder text match score 194 and placeholder image recognition 108, are added to the placeholder cache 134 according to step 208. The process then proceeds to the next step in the data pipeline 136.

[0064] The placeholder image recognition system 36 of this disclosure achieves an effective level of performance and is evaluated for precision, recall, and F-1 score. As recognized, precision evaluates the accuracy of predicted placeholders, and recall evaluates the proportion of all detected placeholder images. This value is understood to be the complement of the number of missed images. The F-1 score is understood to represent the combination of precision and recall. The following table lists the number of images processed, the number of predicted placeholder images, the number of correctly predicted placeholder images, and the number of incorrectly predicted placeholder images for several instances. Furthermore, the precision based on such values ​​is also shown.

[0065] [Table 1]

[0066] The full picture of recall performance cannot be used because it would require further manual verification, but a limited dataset is shown in Table 2 below with recall data.

[0067] [Table 2]

[0068] The details provided herein are intended to illustrate, as examples, each embodiment of a placeholder image detection system, a placeholder image detection process, and a method for identifying an input image as a placeholder, and are provided to offer what is considered to be the most useful and easily understandable explanation of the principles and conceptual aspects. In this regard, there is no intention to provide any more specific details than necessary, and the description, in conjunction with the drawings, will make it clear to those skilled in the art how the various forms of this disclosure can be actually implemented.

Claims

1. A method for identifying an input image as a placeholder, The steps include evaluating the degree of agreement between the input image and a known set of placeholder images, In response to the evaluation that the input image does not match, the steps include extracting a group of text characters from the input image, The steps include tokenizing the aforementioned text character set into multiple input image words that constitute one or more phrases, The steps include generating a placeholder text match score from a plurality of input image words evaluated against a word list for placeholder text of a group of known placeholder phrases, each containing one or more known placeholder words, The steps include flagging the input image as a placeholder based at least partially on the placeholder text matching score, and A method that includes this.

2. The step of generating the placeholder text matching score is, The steps include generating multiple match count values ​​for one or more phrases based on the number of input image words included in one of the phrases that match known placeholder words in the placeholder text word list, The steps of generating a plurality of matching spread values ​​with respect to one or more phrases between the first word of the input image word included in one of the given phrases that matches one of the known placeholder words and the last word of the input image word included in one of the given phrases that matches another of the known placeholder words, A step of obtaining the placeholder text match score for one of the one or more phrases selected, in accordance with one of the plurality of match count values ​​and one of the plurality of match spread values. including, The method according to claim 1.

3. The method according to claim 2, further comprising the step of flagging the input image as a non-placeholder, in accordance with the fact that the match spread value for the selected one of the one or more phrases has been evaluated based on a predetermined match spread threshold.

4. The method according to claim 2, further comprising the step of flagging an input image as not being a placeholder, based on an evaluation of the order inconsistency rate of the plurality of input image words based on a predetermined word order rate threshold.

5. The step of evaluating the degree of agreement between the input image and the known group of placeholder images is: The steps include converting the aforementioned input image into input image vector values, A step of generating a placeholder image matching score from a query to a search index using the input image vector values, wherein the search index is created from known placeholder image vector values ​​generated from the known group of placeholder images. The steps include: flagging the input image as a placeholder based on the placeholder image matching score; including, The method according to claim 1.

6. The conversion of the input image into the input image vector value is performed by a machine learning system. The known placeholder image vector values ​​are generated by the machine learning system. The method according to claim 5.

7. The method according to claim 6, wherein the machine learning system is a convolutional neural network.

8. The method according to claim 2, further comprising the step of receiving input that the degree of agreement between the input image and the known group of placeholder images has been evaluated and confirmed.

9. The method according to claim 1, wherein the step of extracting the text character set from the input image is performed by an optical character recognition system.

10. The method according to claim 1, wherein the text character set includes characters from one or more languages.

11. The method according to claim 1, further comprising the step of removing a stop word from one or more of the aforementioned phrases.

12. The method according to claim 1, further comprising the step of performing spell correction on the plurality of input image words in the one or more phrases.

13. A system for identifying placeholder images in a catalog, An image comparator that receives an input image and generates a placeholder image matching score from the input image, An optical character recognition engine that receives an input image and outputs a set of text characters from the input image, wherein the set of text characters is ordered as a plurality of input image words that are components of one or more phrases, A word tokenizer that groups the aforementioned text character set into the aforementioned multiple input image words of one or more phrases, A word list database for placeholder text, each containing one or more known placeholder phrases that include one or more known placeholder words, A text comparator connected to the placeholder text word list database, which accepts the plurality of input image words and generates a placeholder text match score from an evaluation of the plurality of input image words based on the placeholder text word list database, wherein the placeholder image is identified at least partially based on the placeholder text match score. A system equipped with these features.

14. The text comparator generates multiple match count values ​​for one or more phrases based on the number of input image words contained in one of the phrases that match known placeholder words. The text comparator generates a plurality of match spread values ​​for one or more phrases, one of which is based on the first word of the input image words included in the given one of the phrases that matches one of the known placeholder words, and the last word of the input image words included in the given one of the phrases that matches another of the known placeholder words. The placeholder text match score is obtained from one of the one or more selected phrases, according to one of the multiple match count values ​​and one of the multiple match spread values. The system according to claim 13.

15. The system according to claim 14, wherein the identification of the placeholder image is performed at least in part on an evaluation of a match spread value relating to one of the selected phrases based on a predetermined match spread threshold.

16. The system according to claim 14, wherein the identification of the placeholder image is performed at least in part on an evaluation of the order discrepancy rate of the plurality of input image words based on a predetermined word order rate threshold.

17. The system according to claim 13, wherein the optical character recognition engine outputs the text character set in one or more languages, and the word tokenizer associates the text character set as specific to one of the given languages.

18. The aforementioned image comparator, A vectorizer that generates input image vector values ​​from the aforementioned input image, A placeholder image index containing one or more known placeholder image vector values ​​generated from known placeholder images by the vectorizer, A scorer that generates a placeholder image matching score from a query to the placeholder image index using the input image vector value. Equipped with, The system according to claim 13.

19. A product comprising a non-temporary program storage medium readable by a computing device, wherein the medium tangibly embodies one or more programs of instructions executable by the computing device to perform a method of identifying an input image as a placeholder, The aforementioned method, The steps include evaluating the degree of agreement between the input image and a known set of placeholder images, In response to the evaluation that the input image does not match, the steps include extracting a group of text characters from the input image, The steps include tokenizing the aforementioned text character set into multiple input image words that constitute one or more phrases, The steps include generating a placeholder text match score from a plurality of input image words evaluated against a word list for placeholder text of a group of known placeholder phrases, each containing one or more known placeholder words, The steps include: flagging the input image as a placeholder based on the placeholder text matching score; including, product.

20. The step of generating the placeholder text match score, which is embodied as one or more programs of the instruction, The steps include generating multiple match count values ​​for one or more phrases based on the number of input image words included in one of the phrases that match known placeholder words in the placeholder text word list, The steps of generating a plurality of matching spread values ​​with respect to one or more phrases between the first word of the input image word included in one of the given phrases that matches one of the known placeholder words and the last word of the input image word included in one of the given phrases that matches another of the known placeholder words, A step of obtaining the placeholder text match score for one of the one or more phrases selected, in accordance with one of the plurality of match count values ​​and one of the plurality of match spread values. including, The product according to claim 19.

Images