Identification of placeholder image

Abstract

To provide a detection system of a more accurate and scalable placeholder image.SOLUTION: Identifying that an input image as a placeholder is accompanied with an image comparison part and a text comparison part based on known placeholder images. A text character group is extracted, and is tokenized to a plurality of input mage words being components of one or more phrases. A placeholder text coincidence score is generated from input image words evaluated on the basis of a word list for a placeholder text related to known placeholder phrases. A flag being a placeholder can be attached to the input image on the basis, at least in part, of the placeholder text coincidence score.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure generally relates to the processing of images and text, and more specifically to the identification of placeholder images.

[0002] [Cross - Reference to Related Applications] None applicable

[0003] [Description of Research and Development Funded by the Federal Government] None applicable

Background Art

[0004] Online shopping is generally understood to include consumers purchasing goods and services through the Internet, and it continues to gain popularity due to its convenience, wider product options, and price advantages over traditional physical retail stores. By using web browsers and retail - specific applications available on smartphones, tablets, and other mobile devices, purchases can be made almost regardless of time and location. Payments are electronically processed almost instantaneously in various payment methods, including credit cards, debit cards, gift cards, and in addition, cryptocurrency. Direct delivery of goods to consumers' homes can also be carried out by various parcel delivery services.

[0005] Importantly, the seller can provide detailed information to consumers, and such information can better assist in the purchase decision - making process. Such information includes marketing materials from manufacturing companies or retailers, and is provided in the form of multimedia content such as 3D product models viewable through an interface for augmented reality, in addition to text, images, and videos. Furthermore, additional documents such as user manuals and frequently asked questions from the manufacturing company can also be posted. Evaluations and reviews from other consumers may also be presented.

[0006] The market segments are diverse, and the corresponding sellers also vary widely to meet the requirements of each such segment. Therefore, differences exist in the business processes and the mechanisms of online stores among these sellers. For example, small retail stores sell products only in certain niche markets, but may have sufficient sales volume to justify a dedicated online store. There are also other retail stores with a greater commercial presence that offer a wider range of products. These retail stores may handle the management of their store sites in-house and usually allocate sufficient dedicated resources for the maintenance and update of all the content of the online catalog. This is because there is a direct relationship between sales revenue and the usability, usefulness, and attractiveness of the website. These sites may utilize existing e-commerce platforms such as Shopify and Squarespace, which provide basic e-commerce functions, especially product catalog creation, search, shopping cart, and payment processing, etc.

[0007] Another type of online retail store also exists. Although its business scale is relatively small, it can reach a wider customer base by using an online marketplace. Rakuten is one of the well-known marketplaces, alongside Amazon, Walmart, Target, etc. The specific name may vary depending on the marketplace, but the basic concept is quite similar. Various third-party sellers manage the list of products they want to sell on the marketplace site, and customers can search for the products they want to purchase. There may be one or more sellers for the same product, and once the customer selects the desired product, they can choose one based on factors such as price, shipping cost, delivery time, and the seller's evaluation. On the marketplace, the purchase transaction is sent to the third-party seller, who fulfills the order and ships the product to the buyer.

[0008] Since the product catalog conforms to the standards of its graphic design, it appears to be that of the online marketplace site. However, the content of the information published on the catalog page for a specific product may be the responsibility of the third-party seller offering that product. Therefore, third-party retailers may be tasked with providing product descriptions, various product images, and other multimedia content. Nevertheless, third-party sellers may have limited personnel and resources for updating the description content, and there may be a possibility that high-resolution images showing the product from multiple perspectives are not available. Photographing and editing these product images can be a time-consuming task for such companies without the proper facilities. Showing high-quality product images is an essential element in achieving successful online sales. This is because such images often become the sole or at least the main basis when customers make purchasing decisions. Considering this importance, the management interface of the marketplace site may require the upload of one or more product images before a catalog list can be generated. When faced with a delay in starting sales until high-quality images are available, third-party sellers may choose to upload placeholder images.

[0009] A placeholder image is understood to include any image that does not accurately reflect the product associated with it. Placeholder images generally may include images of one or more objects related to the product in question, for example, images of a hammer, driver, and pliers as specific hand tools. The image may also include a stock image that does not correspond to the actual product or a notice indicating that the product image will be "released soon". The image may sometimes include text clearly stating that the image is a placeholder.

[0010] Regardless of the design harmony and visual harmony with other design elements of the website, the placeholder images are impairing the purchasing experience and hindering the process of making a purchase decision. This leads to a short-term increase in sales for the seller as the product list is immediately available and the reach of the marketplace site is generally wide, but in the long term, it leads to a decrease in sales and thus a decrease in revenue for both the marketplace site and third-party sellers. The negative impact of placeholder images has been considered in the context of third-party sellers uploading them to the marketplace site, but similar problems may also be seen in online stores / websites operated by retailers. Nevertheless, on the marketplace site, due to the large quantity and variety of items typically found in its catalog, the need to identify and remove placeholder images is particularly urgent.

[0011] Therefore, there is a need to detect placeholder images in the art. In one known approach, placeholder images are divided into two broad categories: those already identified in the product catalog, such as known placeholders, and those not identified so far, such as new placeholders. One of a plurality of known hash algorithms is applied to known placeholder images to find other images with the same hash value. For unknown placeholder images, a machine learning model is used to determine whether the input image is a placeholder. This includes three steps. First, the step of preparing 10 images for analysis for each product category, second, the step of comparing the input image with all 10 images of that product category, and third, the step of determining that the input image is a placeholder if 5 out of the 10 product images are different from the product images.

[0012] Another known approach for detecting placeholder images is part of an extensive process for selecting optimal images for an online catalog. Unlike the placeholder detection methods described above, there is no distinction between known and new placeholders. Instead, a binary image classifier is utilized to detect placeholder images. This model is understood to be trained on publicly available training datasets and constructed from existing models. The model is subsequently fine-tuned repeatedly using manually prepared product images.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0013] Existing methods for detecting placeholder images are still inaccurate, and machine learning-based approaches require complex training procedures and large datasets. Therefore, there is a need in the art for a more accurate and scalable placeholder image detection system.

MEANS FOR SOLVING THE PROBLEMS

[0014] A placeholder image is an image that has no meaningful content and is intended to be replaced when an actual product image becomes available. It is desirable to flag an image as a placeholder, as this can improve the customer experience and revenue on an e-commerce site. Additionally, an improvement in the accuracy of a machine learning model based on accurate product images is envisioned. An accurate and scalable placeholder image detection process is envisioned in various embodiments of the present disclosure. Generally, placeholder images can be classified as images that are visually similar to known placeholder images or images that contain some text indicating that they are placeholder images.

[0015] In one embodiment, it is assumed that when the input image is visually similar to a known example, a vectorization of the image using a certain type of machine learning model and a vector comparison method for matching with the input image are used. In an image containing text that is likely to indicate a placeholder image, text is extracted from the image using optical character recognition (OCR), and each word of the extracted text is compared with words included in a set of known placeholder phrases / sentences to obtain an overall match count or score and the spread of the matches. For the phrase that best matches and exceeds a predetermined score threshold, a further evaluation regarding the spread of the matches is performed, and it is confirmed that it fits within a predetermined maximum length. The evaluation of the word count regardless of order is also an evaluation target, and it is confirmed that it is within a predetermined value range. A feedback loop can be implemented to incorporate the newly identified placeholder image into the set of known placeholder images and expand the capabilities of the image matching aspect of this process. The image processing workflow may be made more efficient by using a cache layer. Overall, embodiments of the placeholder image detection system may achieve an accuracy of about 95%.

[0016] According to one embodiment of the present disclosure, a method for identifying an input image as a placeholder includes evaluating a degree of match of the input image against a group of known placeholder images. The method may include extracting a group of text characters from the input image in response to an evaluation that the input image does not match. There may be a step of tokenizing the group of text characters into a plurality of input image words that are components of one or more phrases. The method can include generating a placeholder text match score from the plurality of input image words evaluated based on a word list for placeholder text regarding known placeholder phrases. Each of the placeholder phrases may include one or more known placeholder words. There may be a step of flagging the input image as a placeholder based at least in part on the placeholder text match score. The method described above can be realized as a set of machine-readable instructions executed by a computer system, and such instructions are tangibly embodied in a non-transitory program storage medium.

[0017] Another embodiment of the present disclosure can be a system for identifying placeholder images in a catalog. The system can include an image comparator that receives an input image. A placeholder image match score can be generated by the image comparator from the input image. The system may also include an optical character recognition engine that receives the input image. The OCR engine can then output a group of text characters from the input image. The group of text characters can be ordered as a plurality of input image words that are components of one or more phrases. There may be a word tokenizer that groups the group of text characters into a plurality of input image words of one or more phrases. The system can further include a word list database for placeholder text having one or more known placeholder phrases. Each of the known placeholder phrases can include one or more known placeholder words. There may be a text comparator connected to the word list database for placeholder text. The text comparator may receive a plurality of input image words. A placeholder text match score can be generated by the text comparator based on an evaluation of the plurality of input image words based on the word list database for placeholder text. Placeholder image identification can be performed at least partially based on the placeholder text match score.

[0018] These and other features and advantages regarding the various embodiments disclosed herein will be more fully understood in connection with the following description and drawings. The same numbers in the drawings refer to the same parts throughout the drawings.

Brief Description of the Drawings

[0019]

Figure 1A

Figure 1B

Figure 1C

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 5E

Figure 5F

Figure 5G

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 6E

Figure 7A

Figure 7B

Figure 7C

Figure 7D

Figure 7E

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13A

Figure 13B

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0020] The present disclosure relates to various embodiments of a method and system for identifying placeholder images. The detailed description set forth below in connection with the accompanying drawings is intended as a description of the presently contemplated embodiments and is not intended to represent the only form in which such embodiments may be developed or utilized. In this description, functions and features are described in relation to the embodiments shown. However, it should be understood that the same or equivalent functions may be achieved by other embodiments that are also intended to be within the scope of the present disclosure. Further, the use of relational terms such as "first" and "second" is used only to distinguish one entity from another and is not intended to imply any actual relationship or order between such entities, nor is it necessarily suggestive of such a relationship or order.

[0021] Placeholder images can be displayed or used in various information presentation situations, but are typically found in the context of an online e-commerce platform or website that includes catalogs of products and services offered for purchase. The available inventory of products cataloged on the site is grouped by category or browsable by search query, and more details are provided on the corresponding catalog page for the selected product. Catalog entries can be input along with text and images containing information about the product. As described above, the management of such text and image data can be performed by the staff of the e-commerce site owner, a third-party seller of products using the e-commerce site as a sales platform, a manufacturing company or sales agency, or any other party in the supply chain. Depending on the situation, for some reason, the image of the product may not be available, and when presenting the catalog page or its preview to visiting customers, it can be replaced with a placeholder image.

[0022] Figures 1A - 1C, 2A and 2B, 3A and 3B, 4A - 4D, and 5A - 5G show various examples of placeholder images. In particular, Figures 1A - 1C show placeholder images 1a - 1c which are various company logos. Figure 1A shows a placeholder image using the "Rakuten" company logo 1a of the present assignee. As another logo, the logo 1b shown in Figure 1B can identify the service "kobo". Figure 1C shows a graphical logo 1c without text of a product or service provided by the same assignee (for example, Viber). Figures 2A and 2B show placeholder images 2a and 2b respectively, and small thumbnails 2a - 1, 2b - 1 of this image representing a damaged image are shown against virtual frames 2a - 2, 2b - 2. This is because otherwise it would not be visible against a white background. Figures 3A and 3B show blank placeholder images 3a, 3b respectively, and these images are single - color blocks and contain no other content. The boundary surrounding the white block of the blank placeholder image 3a is shown only for the purpose of defining that boundary. Such a boundary is usually not displayed in a browser.

[0023] Figures 4A - 4D show yet another type of placeholder images 4a - 4d respectively, through which it is clearly shown and conveyed that there is no image. For example, placeholder image 4a incorporates the representation of a plurality of general - purpose images. On the other hand, in placeholder image 4b, a mountain landscape is shown with diagonal lines, indicating that the image is not available. Similarly, in placeholder image 4c, a general - purpose portrait image is drawn with diagonal lines, indicating that the image is not available. Placeholder image 4d contains a camera icon enclosed in a circle with diagonal lines crossing the circle, indicating that the image (represented by the camera) is not available.

[0024] Figures 5A to 5G each show variations of different types of placeholder images 5a to 5g, which include text indicating that the image is a placeholder. Figure 5A shows a placeholder image 5a that includes Japanese characters 6a that can be read as "gazou nashi", which can be translated as "no image". Figure 5B shows a placeholder image 5b that includes English characters 6b that form the word "No Image", followed by Japanese characters 7b that form the phrase "tadaima gazou seisaku chuu desu", which can be translated as "currently image is being produced". The identification name 8b of the website or its operating company can be included in another part of the placeholder image 5b. Figure 5C shows a placeholder image 5c that includes Japanese characters 6c that form the phrase "gazou junbi chuu", which can be translated as "image is being prepared".

[0025] Such phrases indicating that an image is not available may be shown using only English characters. Such a placeholder image 5d is shown in FIG. 5D, which includes a phrase 6d of "Sorry Image Not Available", where the word "Sorry" 6d-1 is displayed in a script font and "IMAGE NOT AVAILABLE" 6d-2 is displayed in a sans-serif block font. This example shows that depending on the part of the phrase, it can be shown in different visual styles, as the word "IMAGE" 6d-2 is shown in a larger font size than "NOT AVAILABLE" 6d-2. FIG. 5E is another placeholder image 5e in which a phrase of "Image coming soon" is shown as a group of characters 6e on an image background 7e. FIG. 5F shows a placeholder image 5f with the text 6f of "No Image Currently Available" written under the icon 7f of a book. This example may be used in the context of a product list of a book or other printed matter represented by the icon 7f, but only when the actual image of the book cover or other images showing the actual appearance of the product are not available. FIG. 5G shows a placeholder image 5g including the word 6g of "Cover Coming Soon". This example shows that the specific wording may vary, and the placeholder image does not necessarily need to include the word "image". Here, the word "cover" is used to indicate that this image would originally be an image of the book cover and the appropriate image is "coming soon".

[0026] Figures 6A to 6E are exemplary images 9a to 9e respectively, each containing text content that may be confused with placeholder images. For example, the image 9a shown in Figure 6A contains text 10a such as "new menu coming soon". The image 9b in Figure 6B may be the cover of a book titled "The Book With No Pictures". The phrase 10b "No Pictures" may be confused with typical placeholder text that may indicate "No Picture" or some derivative expression thereof. Similarly, the image 9c in Figure 6C may be the cover of a book titled "He Had No Image", and the word 10c "No Image" may be confused with the same text that may be seen in placeholder images. The image 9d in Figure 6D may be the cover of a book containing text 11d "COMING SOON: THE FLOOD a novel Zvi Jagendorf", and the phrase 10d "coming soon" contained therein may be confused with the phrase of a placeholder image. Figure 6E is the product accurately represented by the image 9e, and the word "Coming soon!" means that the product will be available soon, and it is not necessarily the case that the product image will be displayed soon.

[0027] Figures 7A to 7E are respectively exemplary images 12a to 12e of images that appear to be of a specific product. Image 12a in Figure 7A shows a single component or product that may have a different degree of relevance to the product being sold. The relevance may be broad, such as generally related at the category level, or specific to the same extent as competing products. However, image 12a includes text 13a that indicates "Not actual part". Image 12b in Figure 7B shows a plurality of items including gears, belts, bearings, and gaskets, etc., and it may be clear that this image does not correspond to a specific product. There is also overlay text 13b indicating that it is a "temporary placeholder". Similarly, image 12c shown in Figure 7C includes the company logo of the seller entity along with images of brake rotors and springs. In this case, from the context of these depictions, it may suggest that this image does not correspond to a specific product, and it also includes text 13c that says "Actual Image Unavailable". Image 12d shown in Figure 7D is very similar to image 12c in that the company logo of the seller entity can be seen along with an image of automotive exhaust system parts. Here too, from the context, it may suggest that it is not an image corresponding to any specific product, and text 13d that says "Stock Photo - No Direct Image for this part" supports this. However, including relevant images in these images 12c and 12d may cause confusion and a false impression that they correspond to the product. Image 12e shown in Figure 7E is a single item of furniture. Since only one item is shown in this image, it may be confused with an image of the actual product. Nevertheless, this image also includes text 13e that says "Placeholder Only".

[0028] As shown in the block diagram of FIG. 8, an exemplary e-commerce platform 20 can include one or more server computer systems that are connected to the Internet and communicate with a remote client computer system 22 to communicate data. One such computer system may be a web server 24 that receives various information requests through a browser application operating on the client computer system 22. The web server 24 can, in response, obtain the requested information and send it back to the client computer system 22 for rendering and display. In the case of the e-commerce platform 20, this can be product catalog information. Among them, there may be thousands to millions of individual records corresponding to the products offered for sale on this platform.

[0029] This information can be stored in a product catalog database 26 composed of one or more product records 28. In the illustrated example, the product record 28 can include an identifier field 28a, a name field 28b, a description field 28c, and one or more image fields 28d. Depending on the embodiment, it is possible to store a data file representing the (multiple) images in the product catalog database 26 or another image database 30. In the former case, the image field 28d can accommodate the image data, or in the latter case, instead, the image field 28d can have a reference to a record in the image database 30 that stores the image data.

[0030] The structural interrelationships shown for the product catalog database 26, the product records 28 it contains, and the image database 30 represent a greatly simplified representation of an embodiment of a typical e-commerce platform. This is shown as one possible example of structuring the various underlying components used in combination to provide an online shopping service to customers, and also as a way in which the placeholder image identification system 36 can be adapted to the overall environment of the e-commerce platform 20. Similarly, the specific fields 28a - 28d of the product records 28 are merely illustrative, and depending on the embodiment, the number of fields may be increased or decreased compared to what is shown. The structure of the record fields may suggest a relational database defined with each record as a row and each field as a column, but this is also shown as only an example. The product catalog database 26 can be implemented in various ways known in the art, and the details shown are not intended to be the only way to configure the e-commerce platform 20.

[0031] Although not specifically shown, there are also a number of other components that are part of the e-commerce trading platform 20. However, since the implementation of such components is within the skill of those skilled in the art, it is understood that the additional details thereof are omitted. In line with this concept, the web server 24, the product catalog database 26, and the image database 30 may be implemented in one or more computer systems, and each computer system includes a general-purpose processor capable of executing pre-programmed instructions, one or more types of memory for storing data and instructions, and a communication method capable of exchanging data between computer systems. In order to provide services to a large number of users and ensure high availability, the e-commerce trading platform 20 may be implemented by using additional bare-metal hardware, load balancers, additional network connections, and other redundancies as a cluster. Therefore, representing the web server 24, the product catalog database 26, and the image database 30 individually is for showing each functional element and its general classification rather than individual hardware and software units.

[0032] The main purpose of the e-commerce trading platform 20 is for customers to access it for shopping. However, as another aspect, it is also for sellers to provide products for sale. When referred to in this specification, the seller may refer to a third-party seller 32, which is a business entity different from the business entity operating the e-commerce trading platform 20, or may refer to the business entity operating the e-commerce trading platform 20 or its closely related companies. For this reason, the third-party seller 32 may access the administrator interface to change the product record 28 to list the products provided for sale. In addition to the product record 28, there may also be a system administrator 34 who can change other functions of the e-commerce trading platform 20. In the context of the present disclosure, when the product image is not available but the image is required to complete the product record 28 to start selling, such placeholder images may be uploaded by such third-party sellers 32 and system administrators 34. These placeholder images, which may take the form of the examples described above, may be stored in the image database 30. The placeholder image identification system 36 is assumed to flag such placeholder images for deletion or for updating to the actual product image.

[0033] The flowchart of FIG. 9 shows one embodiment of the placeholder image detection process according to the present disclosure. Generally, this process can be divided into two sub-parts. One is the image comparison part 40, and the other is the text evaluation part 42. The image comparison part 40 generally includes comparing the input image 44 with a set of existing known placeholder images. On the other hand, the text evaluation part 42 includes extracting text from the input image and determining whether the text so extracted indicates a high likelihood that the image from which such text was extracted is a placeholder image. The block diagram of FIG. 10 shows one possible embodiment of the placeholder image identification system 36, which is also similarly divided into an image comparison unit 140 and a text evaluation unit 142. These sub-parts and components will be described in detail in turn. The present disclosure further contemplates a method of identifying an input image as a placeholder, the steps of which are shown in the flowchart of FIG. 11. Each step of this method can also correspond to a specific component of the placeholder image identification system 36 in addition to the specific aspects of the placeholder image detection process described in relation to the flowchart of FIG. 9. This method can be tangibly embodied as a product of a non-transitory program storage medium as one or more programs executable by a computing device, for example, a program of the e-commerce platform 20.

[0034] In the placeholder image detection process, the image comparison unit 40 generally collates the input image 44 with a set of known placeholder images 46. This can correspond to step 110 of evaluating the degree of match between the input image 44 and a set of known placeholder images 46 in the method of identifying the input image shown in the flowchart of FIG. 11. Using a machine learning model, both the input image 44 and the known placeholder image 46 can be converted into numerical vectors, and then it can be determined whether they match based on a similarity score. The block diagram of FIG. 10 shows an image comparison unit 140 having an image comparator 148. The image comparator itself includes a vectorizer 150 and a scorer 152. The image comparator 148 communicates with an image search index 54 that can store the values of the numerical vectors of the known placeholder images 46. Needless to say, the conversion of the known placeholder image 46 into numerical values is performed only once, and such conversion is performed by components other than the vectorizer 150 shown in FIG. 10. According to various embodiments of the present disclosure, the vectorizer 150, which is part of the overall placeholder image identification system 36, operates only on the input image 44.

[0035] Although the block diagram shows the image search index 54 as being outside the image comparator 148, this is merely for illustrative purposes and is not limiting. This is because in certain embodiments, the image search index 54 may be logically grouped within the image comparator 148. As a general matter applicable to other components and functions of the placeholder image identification system 36 disclosed herein, including one component within another larger component or component class / group is shown only by way of example. One skilled in the art will recognize that in some cases, a particular component may be separated from such a component class or group, or included as part of a different component class or group.

[0036] The known placeholder images 46 are understood to be in line with the foregoing description content, and these images are shown in FIGS. 1A to 4D. Further, to the extent that more unique placeholder images such as those shown in FIGS. 5A to 5G are commonly seen in the e-commerce trading platform 20, such images may be identified as known and provided to the machine learning model for comparison based on the images. As shown in the flowchart of FIG. 9, the known placeholder images 46 are the subject of the vectorization process 50a. Here, the dataset representing the images is converted into a set of known placeholder image vector values 51a.

[0037] The images are provided as an array of bitmap pixel values organized according to rows and columns, and the vectorization process is understood to convert such images into a set of values representing geometric primitives such as points, lines, curves, and polygons. The machine learning module may be utilized in the vectorization process 50a, and one possible implementation is a convolutional neural network (CNN). However, without departing from the scope of the present disclosure, it can be substituted with any other suitable machine learning model. In the context of the placeholder image identification system 36, this vectorization process may be executed by the vectorizer 150. The known placeholder image vector values 51a of the known placeholder images 46 are stored in the image search index 54. The image search index 54 can also be part of a flexible search module or a commercially available off-the-shelf system such as Facebook AI Similarity Search (FAISS).

[0038] The image comparator 148 receives the input image 44, and the image comparison unit 40 of the placeholder image detection process includes a vectorization step 50b using the input image 44. This step can be performed by the vectorizer 150 of the image comparator 148. As a result, the input image vector value 51b is generated, and this value is queried from the image search index 54. In the context of the method of identifying the input image as a placeholder as shown in the flowchart of FIG. 11, this corresponds to step 109 of converting the input image 44 into the input image vector value 51b.

[0039] The known placeholder image vector value 51a is compared with the input image vector value 51b, and the scorer 152 calculates the placeholder image match score 53 in step 52. In one embodiment, cosine similarity may be used for the placeholder image match score 53. The method of identifying the input image as a placeholder can include the corresponding step 110-2 of generating the placeholder image match score 53 from the query to the image search index 54 using the input image vector value 51b.

[0040] If the calculated placeholder image match score 53 is 0.94 or higher according to the comparison step 56, the input image 44 is considered to have a positive match with the known placeholder image 46 according to step 58, and a flag to that effect is set. The specific value of 0.94 is shown only for illustrative purposes and may be changed according to the specific details of the embodiment. The threshold match score 53, as a parameter passed to the comparison function, may also change with each iteration. It is to continue to search for a more appropriate image match after the first identified match. This flagging step may include updating the relevant record field indicating that the corresponding image is a placeholder, or updating any other step specific to the implementation that generally indicates that the target image is a placeholder. There may be a corresponding step 110-3 of flagging the input image 44 as a placeholder based on the placeholder image match score 53. The general concept of using machine learning to identify similar images is known in the art, and thus the aforementioned image comparison unit 40 may be implemented using any other suitable machine learning-based image comparison system or other image comparison system. One such known conventional image comparison system is Scale Invariant Feature Transform (SIFT).

[0041] According to one embodiment, after the input image 44 is considered to match according to step 58, there may be a manual verification step 59, where the input image 44 is confirmed as a placeholder. This feedback may be used to further update the image search index 54 for subsequent iterations.

[0042] If the calculated placeholder image matching score 53 is evaluated to be less than 0.94 in the comparison step 56, the image comparison unit 40 considers that no result is obtained. The steps 110-2 of generating the placeholder image matching score 53 and 110-3 of attaching a flag indicating that it is a placeholder to the input image form a loop. In this loop, the vector value of the input image 44 in the image search index 54 is compared with each of the vector values of the known placeholder images, and the comparison is repeated until one or more placeholder image matching scores 53 exceed the threshold. If there is no known placeholder image 46 that matches the input image 44, it is considered that no result is obtained, and the process proceeds to the text evaluation unit 42. In the context of the placeholder image identification system 36, the text evaluation component 142 processes the input image 44 using text extraction and text matching processes. This will be described in more detail below.

[0043] One of the text evaluation components 142 is an optical character recognition engine 160 that scans the input image 44 and converts it into text data encoded by a machine. Referring to the flowchart of FIG. 9, the process proceeds to the OCR preparation step 60. This step can include various OCR preprocessing steps, such as deskewing, binarization, line removal, and character separation / splitting. The OCR engine 160 also performs the next English text extraction step 62a. In this context, the method of identifying the input image as a placeholder can include, as the corresponding step 112, extracting a group of text characters from the input image 44. This is performed in response to the evaluation that the input image 44 does not match in the image comparison unit 40 described above. Computational processing for character recognition is well known in the art, and there are various methods. Therefore, the OCR engine 160 can implement any one of these methods, but the embodiments of the present disclosure are not limited to any specific method.

[0044] Optical character recognition processing can include extracting a group of characters from an input image. FIG. 12A shows a set of individual characters 66a to 66v extracted from the exemplary placeholder image 5d "SORRY IMAGE NOT AVAILABLE" shown in FIG. 5D. This string of text characters 66 constitutes a phrase 68, which can then be tokenized into a plurality of input image words 70 that are its components. FIG. 12B shows the tokenization. Here, the 1st character 66a, the 2nd character 66b, the 3rd character 66c, the 4th character 66d, and the 5th character 66e are tokenized as the word 70a "SORRY", the 6th character 66f, the 7th character 66g, the 8th character 66h, the 9th character 66i, and the 10th character 66j are tokenized as the word 70b "IMAGE", the 11th character 66k, the 12th character 66l, and the 13th character 66m are tokenized as the word 70c "NOT", and the 14th character 66n, the 15th character 66o, the 16th character 66p, the 17th character 66q, the 18th character 66r, the 19th character 66s, the 20th character 66t, the 21st character 66u, and the 22nd character 66v are tokenized as the word 70d "AVAILABLE".

[0045] The tokenization process 72a may be based on the whitespace 67 between groups of characters, and the placeholder image identification system 36 is understood to include a word tokenizer 172 that performs this process. In the example shown in FIG. 12A, a first enlarged whitespace 67a is shown between the fifth character 66e and the sixth character 66f, thereby separating the first word 70a "SORRY" from the second word 70b "IMAGE". Similarly, a second whitespace 67b between the tenth character 66j and the eleventh character 66k separates the second word 70b "IMAGE" from the third word 70c "NOT". A third whitespace 67c between the thirteenth character 66m and the fourteenth character 66n separates the third word 70c "NOT" from the fourth word 70d "AVAILABLE". Once tokenized, stop words may be removed. Due to the relatively obvious process, this is considered part of the English text extraction process 62a. The method of identifying the input image as a placeholder includes, as a corresponding step 114, tokenizing the text character group into a plurality of input image words that are components of one or more phrases.

[0046] Optionally, there may be an English spelling correction step 74. This is because errors may occur in the text extraction process. Spelling correction may include comparing each of the tokenized words 70a - 70d with an English vocabulary list 75 and making appropriate corrections. Similar to optical character recognition, spell checking can be implemented in various ways, but since such differences are considered within the skill level of those skilled in the art, additional details regarding this are omitted.

[0047] Detection of placeholder images provided in foreign languages ​​is also contemplated, and there may be a parallel foreign language text extraction step 62b. Such foreign languages ​​may include languages ​​that use the Latin alphabet (e.g., Spanish, French, etc.), foreign languages ​​that use Chinese characters (e.g., Mandarin Chinese, Japanese, etc.), as well as any other character set (e.g., Hangul / Korean, Arabic, Thai, etc.). For illustrative purposes, the following description of the embodiments of the text evaluator 42 and the text evaluation component 142 will be described using Japanese as an example of the foreign language. Of course, any other foreign language may be substituted, and one skilled in the art can adapt the text evaluator 42 and the text evaluation component 142 to such other foreign languages ​​by making appropriate modifications.

[0048] The optical character recognition process includes a foreign language segmentation step 72b for segmenting the detected foreign language characters into words in a manner similar to the English segmentation or tokenization process 72a described above, and thus generally corresponds to the tokenization step 114 of the method shown in the flow chart of FIG. 11. This step can be performed by a suitably configured word tokenizer 172. FIG. 13A shows an example of foreign language characters contained in the placeholder image 5c shown in FIG. 5C. The characters form the phrase "Gazoujunbichuu", or "Image Preparation". It will be understood that the OCR engine 160 recognizes each of the first character "图" 76a, the second character "画像" 76b, the third character "準" 76c, the fourth character "便" 76d, and the fifth character "中" 76e individually as consecutive Chinese character strings 76, but not necessarily as words constituting a phrase. 13B, the text evaluator 42 proceeds to a foreign or Japanese segmentation process 72b which tokenizes the phrase 78 into its constituent parts, including a first word "gazou" 80a ("image"), a second word "junbi" 80b ("preparation"), and a third word "chuu" 80c ("in the middle"), which, when combined, can be translated as "image is being prepared" or "image is being prepared."

[0049] As long as the input image 44 contains both English and Japanese / foreign languages, the placeholder image detection process assumes a step 82 of combining the English word 70 and the Japanese / foreign language input image word 80, and the placeholder image 5b shown in FIG. 5B may be applicable. Therefore, all possible words that are highly suspected of suggesting that the input image 44 is a placeholder may be evaluated regardless of the language. The word tokenizer 172 can execute this step to output a set of input image words 84 composed of the English word 70 and the Japanese / foreign language input image word 80 extracted from the input image 44. Here too, the ability to process text in multiple languages is only shown as an example, and there may be another embodiment that extracts only the text of one language. Generally, the plurality of input image words 84 are understood to refer to the text data extracted from the input image 44 regardless of their language (or languages in some cases) and characters.

[0050] Next, the text evaluation unit 42 for placeholder image detection proceeds to step 86 of obtaining a text match score, which can be executed by the text comparator 186. Generally, the input image word 84 is evaluated based on known placeholder phrases 88 that may normally be included in the placeholder image. For example, these phrases include "Sorry, image not available" in the placeholder image 5d, "image coming soon" placed in the placeholder image 5e, and "no image currently available" in the placeholder image 5f, as well as phrases in foreign languages or Chinese characters such as "no image" shown in the placeholder image 5a, "image is currently being produced" shown in the placeholder image 5b, and "image is being prepared" in the placeholder image 5c. These known placeholder phrases are composed of individual placeholder words such as "no", "image", "unavailable", "coming", and "available", etc. After being split into such multiple words in the splitting step 90, they are stored in the placeholder text word list database 92.

[0051] The input image 44, specifically a plurality of input image words 84 included therein, is compared with these known placeholder phrases 88, and a placeholder text match score 194 is generated, which quantifies and represents the degree of match. This generally corresponds to step 116 of evaluating the degree of match of the input image phrase based on the word list for placeholder text in the method of identifying the input image as a placeholder shown in the flowchart of FIG. 11. As an example further showing the step of obtaining the text match score, the extracted input image word may be "sorry this image is not yet available Rakuten shopping". The first phrase among the known placeholder phrases 88 may be "product image not available", and the second phrase among the known placeholder phrases 88 may be "image is unavailable".

[0052] For a particular phrase or input image word 84, the matching process begins with determining a match count value 194 and a spread of the match phrase 196. These values ​​may be generated by the text comparator 186, and this process may correspond to step 116-1 of generating a match count value. A match count value may be generated for one or more phrases. In one embodiment, this may be based on the number of input image words in a particular one of the phrases that match known placeholder words in the placeholder text word list database 92. There may also be a correspondence with step 116-3 of generating a match spread value for a first input image word that matches one of the known placeholder image words and a last input image word that matches another known placeholder word, both belonging to the same known placeholder phrase (or known placeholder word list). As used herein, a match count is the number of input image words 84 that are common to and found in one placeholder text word list stored in the database 92. The spread of a match phrase is understood to be the word length between the first and last match word. The use of scores and spreads is envisioned as an additional rule applied to the input text to determine a match with one of the known placeholder phrases 88, going beyond simple text matching. Continuing with the previous example where the first known placeholder phrase is "product image not available", the words "image", "not" and "available" are considered to match between the first known placeholder phrase and the input image word 84, resulting in a match count value 194 of 3. The match spread 196 is understood to be 5, for example, since in "image is not yet available" there are a total of 5 words between the first match words "image" and "available", including the first match words.

[0053] The known placeholder phrase 88 with the largest number of matches, for example, the first phrase, is selected, and a placeholder text match score 194 is calculated. In one embodiment, the placeholder text match score 194 is the number of matches divided by the length of one of the selected known placeholder phrases 88, and can be generated by the text comparator 186 as well. Generally, the placeholder text match score 194 should be understood as being derived as a whole from one of the selected input image words 84 and the phrase defined thereby. This generally corresponds to step 116-2 of obtaining the placeholder text match score 194 in the method of identifying the input image as a placeholder shown in the flowchart of FIG. 11. Using the above example again, the number of matches is 3. On the other hand, the length of one of the selected known placeholder phrases 88 is 4, that is, product(1), image(2), not(3), available(4). The placeholder text match score 194 is 3 / 4, that is, 0.75. According to one embodiment, the threshold score for determining a match may be 0.75 or more. However, this is only shown as an example. From the inference, a placeholder text match score 194 less than 0.75 is considered not to match. Any other appropriate threshold numerical value may be substituted without departing from the scope of the present disclosure. This evaluation based on a predetermined threshold may be performed in the determination block 96.

[0054] In step 86 of obtaining a consensus score, after evaluating the calculated placeholder text match score 194 based on a threshold value, there may be an additional step 100 of determining a match spread. This generally corresponds to step 116-3 of generating a match spread value in the method of identifying an input image as a placeholder as shown in the flowchart of FIG. 11. Specifically, the match spread is checked based on the length of one of the selected known placeholder phrases 88, and if the match spread is more than twice the length of the known placeholder phrase, these input image words 84 are evaluated as not matching. Continuing with the above example, the match spread is 5, for example, image(1), is(2), not(3), yet(4), available(5). Also, the length of one of the selected known placeholder phrases 88 is 4, that is, product(1), image(2), not(3), available(4). Twice the length of one of the selected known placeholder phrases 88 is 8, and 5 (match spread) is less than 8. Therefore, this match is considered valid.

[0055] The above-mentioned evaluation or check based on the length of one of the selected known placeholder phrases 88 may be performed after comparing the placeholder text match score 194 with a provisional threshold value or at least after the placeholder text match score 194 is generated. In either case, the specific action taken in response to a failed check may be to flag the input image 44 as non-placeholder according to step 118 of the method shown in the flowchart of FIG. 11. The order of operations is not intended to be limiting either way.

[0056] There may be another evaluation step 102 that checks the unordered rate of a plurality of input image words 84 that match one of the selected known placeholder phrases 88. This generally corresponds to step 116-4 of generating the unordered rate in the method of identifying an input image as a placeholder, as shown in the flowchart of FIG. 11. In a pair of two phrases, for example, when one has one of the selected known placeholder phrases 88 and the other has a plurality of input image words 84, the number of words with different orders is defined as the unordered count. As an example, in the case of the phrases [image not available] and [image available not], the unordered count is 1. This is because the word "image" is in the same position in both phrases, but the order of the next pair of two words "available" and "not" is not the same. As another example, in the case of [image not available] and [available image not], the unordered count is 2. This is because the pair of two words "image" and "not" is not in the same order, and the pair of two words "available" and "not" is also not in the same order. Next, the unordered rate is defined as the unordered count divided by the length of each phrase being compared. If the unordered rate is 0.5 or more, the phrases are not considered to match. In the first example above comparing [image not available] and [image available not], the unordered count is 1 and the length is 3, so an unordered rate of 0.333 is obtained. That is, the two phrases are considered to match. However, in the second example above comparing [image not available] and [available image not], the unordered count is 2 and the length is 3, so an unordered rate of 0.6667 is obtained. That is, the two phrases are not considered to match.

[0057] Similarly, the evaluation or check of the above-mentioned order-independent rate may be performed after comparing the placeholder text matching score 194 with a provisional threshold value, or after the evaluation based on the match spread value 196, or at least after the placeholder text matching score 194 is generated. In any case, the specific action taken in response to a failed check may be to flag the input image 44 as non-placeholder according to step 118 of the method shown in the flowchart of FIG. 11. The order of operations is not intended to be limiting either way.

[0058] When the placeholder text match score 194 is evaluated to be 0.75 or more, the spread of the match is evaluated to be less than or equal to twice the length of the known placeholder phrase according to step 100, and the unordered rate is 0.5 or less, a positive text match result 104 is obtained. Otherwise, for a specific one of the known placeholder images 88, a negative text match result 106 is obtained. It should be understood that as long as there is even one positive match result, the input image 44 is regarded as a placeholder. The text comparator 186 is understood to generate the placeholder text match score 194 from the evaluation of a plurality of input image words 84 based on the placeholder text word list database 92, and the placeholder image identification is performed at least partially based on the placeholder text match score 194. The placeholder image identification step can correspond to the flagging step 116-5 in the method of identifying the input image as a placeholder. According to one embodiment, scoring and verification can be performed by the text comparator 186. Referring again to the block diagram of FIG. 10, various scores and metrics regarding the match of the plurality of input image words 84 and the known placeholder phrase 88 can be evaluated by the text comparator 186, and the identification 108 of the placeholder image is performed as either a positive text match result 104 or a negative text match result 106. However, alternatively, these evaluations and finally flagging the input image 44 as a placeholder may be performed by the score evaluation / verification unit 124.Step 116-1 of generating a consistent count value, step 116-2 of obtaining a placeholder text matching score, step 116-3 of generating a match spread value, step 116-4 of generating an order-insensitive rate, and in addition to these, step 116 of evaluating the match of an input image phrase based on the word list 88 for placeholder text, including configuration steps such as step 116-5 of flagging the input image as a placeholder, and step 118 of flagging the input image as non-placeholder based on the match spread value and the order-insensitive rate, a loop is formed to compare the multiple input image words 84 of one input image 44 with each of the known placeholder phrases 88. The flagging steps 118, 116-5 are understood to be applicable only to a specific one of the corresponding known placeholder phrases 88 and do not affect the already identified matches. This loop can end once a match is identified. Generally, in the text evaluation unit 42, it is intended to identify one good match and identify that the target input image 44 is a placeholder.

[0059] According to one embodiment, the positive text match result 104 may be confirmed in the manual verification step 126, and the input image 44 may be added to the image search index 54 as a known placeholder image 46 after vectorization. There may be additional duplicate identification processing to prevent multiple instances of the same placeholder image from being stored in the image search index 54. Instead of or in addition to this, the multiple input image words 84 may be added to the word list database 92 for placeholder text as one of the known ones among the placeholder phrases 88 after the splitting step 90 following the same manual verification step 126.

[0060] Referring again to the block diagram of FIG. 8, the e-commerce platform 20 can continuously utilize a placeholder image identification system 36 that flags images linked to pages provided by the web server 24. A cache layer 130 may exist to reduce the number of unnecessary calls to the placeholder image identification system 36. In addition to the URL (Uniform Resource Locator) of a specific image included in the output file corresponding to the provided web page and generated by the web application server, the image itself is also used to create a hash code, which then serves as the cache key. A typical record of placeholder image identification 108 may contain a key value such as "000019690543e1222adbc6cdfe46ef9f" along with related values as follows.

Number

[0061] Thus, it is understood that the cache entry for a specific image record includes, in addition to the placeholder text match score 194, a boolean flag indicating the situation as to whether the image corresponding to the image record is a placeholder.

[0062] Referring further to the flowchart of FIG. 14, one possible embodiment of the cache processing includes step 200 of parsing an input file 132 output from an upstream pipeline. Next, it is determined by performing a lookup in the placeholder cache 134 whether there is a URL of an image referenced in the input file 132. If there is a record evaluated in decision block 202, the corresponding placeholder image identifier 108 of that image is retrieved from the placeholder cache 134 and passed to the next step of the data pipeline 136. If no entry is found in the placeholder cache 134, the process proceeds to step 204 of downloading the image referenced in the input file 132. A lookup of the hash code of the downloaded image is performed again against the placeholder cache 134, and if the existence of the image record is confirmed in decision block 202, the process proceeds to the next step of the data pipeline 136.

[0063] If the hash value of the image is also not found in the placeholder cache 134, the process proceeds to step 206 of calling the placeholder image identification system 36. The URL of the image and the hash code value of the image are added to the placeholder cache 134 according to step 208, together with the resulting placeholder text match score 194 and placeholder image identifier 108. Thereafter, the process proceeds to the next step of the data pipeline 136.

[0064] The placeholder image identification system 36 of the present disclosure achieves an effective performance level and is evaluated in terms of precision, recall, and F-1 score. As is recognized, precision evaluates the accuracy rate of predicted placeholders, and recall evaluates the proportion of all detected placeholder images. This numerical value is understood to be the complement of the number of missed ones. The F-1 score is understood to represent a combination of precision and recall. The following table lists, for a plurality of instances, the number of processed images, the number of predicted placeholder images, the number of correctly predicted placeholder images, and the number of incorrectly predicted placeholder images. Further, the precision based on such values is also shown.

[0065]

Table 1

[0066] The full picture of the recall performance cannot be utilized because manual verification processing is also required otherwise, but a limited dataset with recall data is shown in Table 2 below.

[0067]

Table 2

[0068] The details shown in this specification are for illustratively explaining each embodiment regarding a placeholder image detection system, a placeholder image detection process, and a method for identifying an input image as a placeholder, and are shown to provide what is considered to be the most beneficial and easily understandable explanation regarding the principle and conceptual aspects. In this context, there is no intention to show details more specifically than necessary, and it will be apparent to those skilled in the art how the various forms of the present disclosure can be actually implemented by the description in combination with the drawings.

Claims

1. 1. A method for identifying an input image as a placeholder, comprising: evaluating the match of the input image to a set of known placeholder images; extracting text characters from the input image in response to determining that there is no match for the input image; tokenizing the text characters into a plurality of input image words that are components of one or more phrases; generating a placeholder text match score from the plurality of input image words evaluated against a placeholder text word list of known placeholder phrases, each of the known placeholder phrases including one or more known placeholder words; flagging the input image as a placeholder based at least in part on the placeholder text match score; The method includes:

2. generating the placeholder text match score, generating a plurality of match count values ​​for the one or more phrases based on the number of the input image words in a given one of the phrases that match known placeholder words in the placeholder text word list; generating a plurality of match spread values ​​for the one or more phrases between a first word of the input image word in the given one of the one or more phrases that matches one of the known placeholder words and a last word of the input image word in the given one of the one or more phrases that matches another one of the known placeholder words; obtaining the placeholder text match score for the selected one of the one or more phrases as a function of a corresponding one of the plurality of match count values ​​and a corresponding one of the plurality of match spread values; Including, The method of claim 1.

3. 3. The method of claim 2, further comprising flagging the input image as a non-placeholder in response to a match spread value for the selected one of the one or more phrases being evaluated against a predetermined match spread threshold.

4. 3. The method of claim 2, further comprising flagging the input image as a non-placeholder in response to an out-of-order rate of the plurality of input image words being evaluated against a predetermined word order rate threshold.

5. evaluating a match of the input image to the set of known placeholder images, converting the input image into input image vector values; generating a placeholder image match score from a query to a search index using the input image vector values, the search index being generated from known placeholder image vector values ​​generated from the set of known placeholder images; flagging the input image as a placeholder based on the placeholder image match score; Including, The method of claim 1.

6. the transformation of the input image into the input image vector values ​​is performed by a machine learning system; the known placeholder image vector values ​​are generated by the machine learning system; The method according to claim 5.

7. The method of claim 6 , wherein the machine learning system is a convolutional neural network.

8. The method of claim 2 , further comprising the step of accepting input that the input image has been evaluated and confirmed as a match to the set of known placeholder images.

9. The method of claim 1 , wherein the step of extracting the text characters from the input image is performed by an optical character recognition system.

10. The method of claim 1 , wherein the set of text characters includes characters from one or more languages.

11. The method of claim 1 , further comprising removing stop words from the one or more phrases.

12. The method of claim 1 , further comprising the step of: providing spelling corrections to the plurality of words in the one or more phrases.

13. 1. A system for identifying a placeholder image in a catalog, comprising: an image comparator that accepts an input image and generates a placeholder image match score from the input image; an optical character recognition engine that accepts the input image and outputs text characters from the input image, the text characters being ordered as a plurality of input image words that are components of one or more phrases; a word tokenizer that groups the text characters into the input image words of the one or more phrases; a placeholder text word list database having one or more known placeholder phrases, each of which includes one or more known placeholder words; a text comparator coupled to the placeholder text word list database for receiving the plurality of input image words and generating a placeholder text match score from an evaluation of the plurality of input image words against the placeholder text word list database, wherein an identification of a placeholder image is based at least in part on the placeholder text match score; A system comprising:

14. the text comparator generating a plurality of match count values ​​for the one or more phrases based on a number of input image words in a given one of the phrases that match known placeholder words; the text comparator generates a plurality of match spread values ​​for the one or more phrases, a given one of the plurality of match spread values ​​being based on a first word of the input image word in the given one of the phrases that matches one of the known placeholder words and a last word of the input image word in the given one of the phrases that matches another one of the known placeholder words; the placeholder text match score is derived from a selected one of the one or more phrases as a function of a corresponding one of the plurality of match count values ​​and a corresponding one of the plurality of match spread values. The system of claim 13.

15. 15. The system of claim 14, wherein the identification of the placeholder image is based at least in part on an evaluation of a match spread value for the selected one of the one or more phrases based on a predetermined match spread threshold.

16. The system of claim 14 , wherein the identification of the placeholder image is based at least in part on an assessment of an out-of-order rate for the plurality of input image words based on a predefined word order rate threshold.

17. 14. The system of claim 13, wherein the optical character recognition engine outputs the text characters in one or more languages, and the word tokenizer associates the text characters as being unique to a given one of the languages.

18. The image comparator, a vectorizer for generating input image vector values ​​from the input image; a placeholder image index including one or more known placeholder image vector values ​​generated from a known placeholder image by the vectorizer; a scorer that generates the placeholder image match score from a query to the placeholder image index using the input image vector values; Equipped with The system of claim 13.

19. 1. An article of manufacture comprising a non-transitory program storage medium readable by a computing device, the medium tangibly embodying one or more programs of instructions executable by the computing device to perform a method of identifying an input image as a placeholder; The method comprises: evaluating the match of the input image to a set of known placeholder images; extracting text characters from the input image in response to determining that there is no match for the input image; tokenizing the text characters into a plurality of input image words that are components of one or more phrases; generating a placeholder text match score from the plurality of input image words evaluated against a placeholder text word list of known placeholder phrases, each of the known placeholder phrases including one or more known placeholder words; flagging the input image as a placeholder based on the placeholder text match score; Including, product.

20. embodied in one or more programs of instructions, generating the placeholder text match score comprises: generating a plurality of match count values ​​for the one or more phrases based on the number of the input image words in a given one of the phrases that match known placeholder words in the placeholder text word list; generating a plurality of match spread values ​​for the one or more phrases between a first word of the input image word in the given one of the one or more phrases that matches one of the known placeholder words and a last word of the input image word in the given one of the one or more phrases that matches another one of the known placeholder words; obtaining the placeholder text match score for the selected one of the one or more phrases as a function of a corresponding one of the plurality of match count values ​​and a corresponding one of the plurality of match spread values; Including, 20. The article of manufacture of claim 19.

Images