System for product component identification based on artificial intelligence (AI)

Abstract

An artificial intelligence (AI)-based system for product ingredient identification is described. The system can receive an image associated with a product listed for sale on an online marketplace. The product can belong to a health and dietary supplement category (e.g., a vitamin supplement) or a food category (e.g., a beverage). The system can generate a prompt for a large language model (LLM) based on the image, and the LLM can identify ingredients (e.g., components) of the product from the prompt. That is, the LLM can identify text from the image that lists the ingredients and their respective amounts (e.g., weights or percentages). The system can generate an output that indicates the identified ingredients and a determination of whether the product listing is approved or rejected based on whether the ingredients are allowed or disallowed.

Description

Background Technology

[0001] Online marketplaces support and therefore experience numerous different activities that facilitate transactions. Some of these activities might involve users (e.g., sellers) posting listings of products for sale, such as health and dietary supplements or food. Listings may include a product description and one or more images. Some ingredients or components of the product may be prohibited in certain regions or countries (e.g., by the U.S. Food and Drug Administration (FDA)). Therefore, online marketplace administrators can use their systems to identify text indicating the product's ingredients from the images in the listing and allow or remove the listing based on whether those ingredients are approved according to a set of predetermined rules. Summary of the Invention

[0002] Artificial intelligence (AI)-based systems for identifying the component compounds of products are used in conjunction with online marketplaces. In one or more implementations, users of the online marketplace (e.g., sellers) can post products for sale, such as health and dietary supplements or foods. The listing may include one or more images showing the product's ingredients or components. The system can use the images to generate prompts for an AI model (e.g., a Large Language Model (LLM)), and the LLM can perform actions based on these prompts to identify the components (e.g., ingredients, chemical compounds) in the product. For example, if the image shows a bottle of dietary supplement with a list of components and their corresponding amounts, the LLM can identify the components based on the prompt. The LLM can output a list of components identified from the image, and the system can use the identified components to determine whether the product listing is approved or rejected. For example, the system can compare the identified components to a database of component regulations (e.g., the FDA's list of prohibited substances) and determine whether the product is approved or rejected based on the database. If rejected, the product listing may be removed from the online marketplace.

[0003] The present invention provides a simplified summary of a series of concepts further described in the following detailed description. Therefore, the present invention is not intended to identify the essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. Attached Figure Description

[0004] The specific implementation method is described with reference to the accompanying drawings.

[0005] Figure 1 This is a diagram illustrating an environment operable in the example implementation that employs the techniques described herein.

[0006] Figure 2 Examples of AI-based systems for product component identification, based on various aspects of this disclosure, are described.

[0007] Figure 3 Examples of images depicting various aspects of a product according to this disclosure.

[0008] Figure 4 Examples of user interfaces depicting various aspects of this disclosure are provided.

[0009] Figure 5 The process in an example implementation of an AI-based system for product component identification, based on various aspects of this disclosure, is described.

[0010] Figure 6 An example of a system including an example computing device is shown, which represents one or more computing systems and / or devices that can implement the various technologies described herein. Detailed Implementation

[0011] Overview

[0012] An AI-based system for product component identification is described. According to the described technology, items can be offered for sale (e.g., listed) on an online marketplace. In one or more implementations, the online marketplace can be accessed by a decentralized computing device corresponding to a "customer" of the online marketplace (e.g., a user with an account on the online marketplace). Users of the online marketplace have some control over which items they list or purchase from that marketplace. For example, users can determine when to list or purchase items, and can do so from different locations via a website or mobile application used for the online marketplace. When a seller posts a product listing for sale, the seller can include information about the product, such as a title, description, and one or more images, as well as any other relevant information.

[0013] In some implementations, users can buy and sell products in the health and dietary supplements or food categories on online marketplaces. For example, sellers might list vitamin supplements and energy drinks. Product listings for such products may include one or more images of the product. In some cases, at least one image in the product listing may depict the active ingredients or components that make up the product, along with a list of each component's corresponding weight (e.g., in milligrams (mg)) or percentage composition. Some components may be permitted in some geographic regions but prohibited in others (e.g., absolutely prohibited or prohibited if exceeding a certain weight or percentage composition). For example, some components may be banned in the United States and therefore listed on the FDA's list of prohibited ingredients. Therefore, online marketplace administrators can employ systems to detect prohibited product ingredients in health and dietary supplements and foods listed for sale and remove such product listings from the online marketplace.

[0014] In one or more implementations, the system can extract text (e.g., an ingredient list) from an image of a product list. For example, the system can use optical character recognition (OCR) to convert the text in the image into a machine-readable text format. The extracted text can be provided to a rule-based decision system, which may include a predefined set of regular expression (Regex) rules. These predefined rules can determine whether a particular product or any individual component or ingredient thereof is permitted or prohibited in a specific geographic region (e.g., absolutely prohibited or prohibited if exceeding a certain amount). For example, a rule could stipulate that component X is prohibited if it exceeds a certain percentage of component Z within country Y.

[0015] If all components identified from the image meet the set of rules (e.g., are all permitted), the system can approve the product list for sale. Alternatively, if any component fails to meet the rules (e.g., is absolutely prohibited or is prohibited because its weight exceeds the permitted weight), the system can remove the product list, at least for the relevant geographic region.

[0016] If the set of predefined rules is insufficient to determine whether a particular product is permitted or prohibited (e.g., if no rule takes into account a particular component), the product list can be directed to a customer support agent or compliance officer for manual review of the products and their components. For example, a customer support agent could manually verify each individual component against a list of prohibited components for each applicable geographic region.

[0017] However, in existing systems, each of these predefined rules can be manually developed by the online marketplace administrator (e.g., a compliance officer). Therefore, existing systems and methods can be inefficient at scale, especially when more ingredient, weight, or percentage composition thresholds and geographic regions are added. As the number of sellers increases, it may be impossible to review every product listed on the online marketplace, potentially leading to prohibited products being listed and potentially purchased. Furthermore, rewriting predefined rules every time a new health and dietary supplement or food product is published is impractical. Additionally, the system may be inefficient by invoking each predefined rule individually for every ingredient identified from an image. Sellers may also publish prohibited products by including poor-quality images (e.g., blurry, dark), not including any images at all, or including false information in the product description, making it impossible to identify the product's ingredients.

[0018] To address these limitations, an AI-based system for product component identification is described. In one or more implementations, the described technology relates to an AI-based system that includes LLM and is implemented as part of an online marketplace. The AI-based system can be able to identify product components from a product listing image and perform LLM based on a single prompt to determine whether the product listing should be approved or rejected based on the components.

[0019] Specifically, users (e.g., sellers) can post product listings on an online marketplace. These listings can be for products belonging to the health and dietary supplement category (e.g., vitamins) or the food category (e.g., soda drinks). A product listing can include one or more images of the product. For example, such an image could depict a vitamin bottle showing a list of each component or ingredient in the vitamins. The system can use the images to generate cues for LLM or some other multimodal AI model. Alternatively, predefined cues can be applied to the relevant images.

[0020] Based on this prompt, an LLM can perform actions to identify one or more components of a product from an image. The LLM can be trained using data related to component regulations in different regions, countries, and other geographic areas (e.g., an FDA database of prohibited ingredients). In some examples, the LLM can be a multimodal model capable of receiving (as input) and outputting data in different formats, including images, text, audio, etc. The LLM can output a list of components identified from the image, and the system can use the identified components to determine whether the product list is approved or rejected. For example, if any component of a product is prohibited or banned in a given geographic area (e.g., at a specific percentage or weight), the product list can be rejected. If the product list is rejected, it can be removed from online marketplaces. In some examples, different AI models (e.g., multimodal models) can apply prompts to an image to identify text within it. The identified text can then be fed to the LLM to identify component compounds and determine whether the list is approved or rejected.

[0021] In some implementations, product lists are approved or rejected based on comparisons of identified components with a dataset, database, or set of rules (e.g., a lookup database) indicating which health and dietary supplements, foods, ingredients, or other components are absolutely prohibited or prohibited in quantities exceeding a threshold. In at least one variant, the AI-based system may send the LLM output to a user (e.g., an administrator of the AI-based system) for human review, whereby the user can determine whether to approve or reject the list based on comparisons of each of the identified components with a dataset.

[0022] The described technology can lead to several improvements, including faster processing, improved data storage and retrieval efficiency, enhanced graphical user interface (GUI) features, and others. For example, configuring the LLM to identify each component or ingredient of a product based on a single prompt can improve system efficiency. Furthermore, training the LLM on a database of component regulations can result in faster processing than existing systems. Specifically, existing systems rely on individually invoking predefined rules corresponding to each applicable component regulation, rather than utilizing a trained LLM, which often leads to errors in component identification or failure to identify certain components altogether. Moreover, after identifying prohibited components, existing systems cannot recommend removing products from the product list or restrict certain products from being listed at all. Therefore, the described system can process more product lists in less time and more accurately to ensure that prohibited products are not traded on online marketplaces.

[0023] The described technology can also improve data storage and retrieval efficiency by centralizing the processes for collecting product images, analyzing images (using LLM), and enforcing component regulations within the same system. For example, automatically removing non-compliant listings from online marketplaces based on LLM output reduces manual work (e.g., by compliance officers) and the system's reliance on regular expression rules. As new regulations evolve, the LLM can also be retrained to output more accurate results over time, further improving the overall performance of the LLM. Furthermore, datasets or databases indicating which health and dietary supplements, foods, ingredients, or other components are absolutely prohibited or prohibited in quantities exceeding a threshold can be organized for structured retrieval. For example, the database can be organized based on weighted component quantities within various products, and the LLM can also be trained based on these weighted component quantities. Additionally, the database can be organized, and the LLM can be trained to identify specific target terms (e.g., common components) or to identify one or more countries and related database information associated with those countries.

[0024] In some aspects, the technology described herein relates to a computer-implemented method comprising: receiving one or more images associated with a product listing in an online marketplace, wherein the product listing is for products belonging to at least one of the health and dietary supplements category or the food category; generating a cue for a large language model (LLM) based on the one or more images; performing the LLM based on the cue to identify one or more components of the product from the one or more images; and generating an output based on the performance of the LLM, wherein the output indicates the one or more components identified from the one or more images and a determination of whether the product listing is approved or rejected.

[0025] In some respects, the techniques described herein relate to a computer-implemented method that also includes comparing one or more components with a dataset that includes component regulations for a geographic region; and determining whether a product list is approved or rejected based on a dataset that indicates whether one or more components are permitted or prohibited in the geographic region.

[0026] In some respects, the techniques described herein relate to a computer-implemented method that further includes: comparing one or more components to a set of rules indicating whether the components are permitted in a geographic area; and determining whether a product list is approved or rejected based on whether the components are permitted in the geographic area according to the set of rules.

[0027] In some respects, the technology described herein relates to a computer-implemented method that also includes sending output to a user for human review, wherein the determination of whether a product list is approved or rejected is based on human review.

[0028] In some respects, the techniques described herein relate to a computer-implemented method that also includes displaying a product list to a user via a user interface based on an indication that the output has been rejected.

[0029] In some respects, the techniques described herein relate to computer-implemented methods in which each component corresponds to the weight or percentage composition of a product.

[0030] In some respects, the techniques described herein relate to computer-implemented methods in which identifying one or more components of a product includes identifying text in one or more images corresponding to one or more components.

[0031] In some respects, the techniques described herein relate to computer-implemented methods in which the output indicates that one or more images do not depict one or more components.

[0032] In some respects, the techniques described herein relate to a computer-implemented method in which a product list is rejected based on at least one of one or more components having an amount greater than a threshold.

[0033] In some respects, the techniques described herein relate to a computer-implemented method that also includes removing product listings from an online marketplace based on the product listing being rejected.

[0034] In some respects, the techniques described in this article involve computer-implemented methods in which a product list is rejected based on the fact that the product itself is prohibited in a geographic region.

[0035] In some aspects, the technology described herein relates to a system comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to: receive one or more images associated with a product listing in an online marketplace, wherein the product listing is for products belonging to at least one of the health and dietary supplements category or the food category; generate a prompt for a large language model (LLM) based on the one or more images; perform the LLM based on the prompt to identify one or more components of the product from one or more images; and generate an output based on the performance of the LLM, wherein the output indicates the one or more components identified from one or more images and a determination of whether the product listing is approved or rejected.

[0036] In some respects, the technology described herein relates to a system in which instructions further cause the system to compare one or more components with a dataset including component regulations for a geographic region; and to determine whether a product list is approved or rejected based on a dataset indicating whether one or more components are permitted or prohibited in a geographic region.

[0037] In some respects, the technology described herein relates to a system in which instructions further cause the system to compare one or more components with a set of rules indicating whether the components are permitted in a geographic area; and to determine whether a product list is approved or rejected based on whether the components are permitted in the geographic area according to the set of rules.

[0038] In some respects, the technology described herein relates to a system in which instructions also cause the system to send output to a user for human review, wherein the determination of whether a product list is approved or rejected is based on human review.

[0039] In some respects, the technology described herein relates to a system in which instructions also cause the system to display a product list to a user via a user interface based on an indication that the output has been rejected.

[0040] In some respects, the technology described herein relates to a system in which each of one or more components corresponds to the weight or percentage composition of a product.

[0041] In some respects, the technology described herein relates to a system in which each component corresponds to the weight or percentage composition of a product.

[0042] In some respects, the technology described herein relates to a system in which, in order to identify one or more components of a product, instructions also cause the system to identify text in one or more images corresponding to one or more components.

[0043] In some respects, the techniques described herein relate to a system in which the output indicates that one or more images do not depict one or more components.

[0044] In some respects, the techniques described herein relate to a system in which a product list is rejected based on the fact that at least one of one or more components has an amount greater than a threshold.

[0045] In some respects, the technology described herein relates to a system in which the instructions further enable the system to remove a product listing from an online marketplace based on the product listing being rejected.

[0046] In some respects, the technology described in this article relates to a system in which a product list is rejected based on the fact that the product itself is prohibited in a geographic region.

[0047] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause one or more processors to perform operations including: receiving one or more images associated with a product listing in an online marketplace, wherein the product listing is for products belonging to at least one of the categories of health and dietary supplements or food; generating a prompt for a large language model (LLM) based on the one or more images; performing the LLM based on the prompt to identify one or more components of the product from one or more images; and generating an output based on the performance of the LLM, wherein the output indicates the one or more components identified from one or more images and a determination of whether the product listing is approved or rejected.

[0048] In the following discussion, exemplary environments in which the techniques described herein can be employed are first described. Then, details and examples of implementations and processes that can be executed in the exemplary and other environments are described. The execution of the exemplary processes is not limited to the exemplary environment, and the exemplary environment is not limited to the execution of the exemplary processes.

[0049] Figure 1 This is an illustration of an environment 100 operable in an example implementation using the techniques described herein. Environment 100 includes a computing device 102, a service provider system 104, and a component identification module 106. In one or more implementations, the computing device 102, service provider system 104, and component identification module 106 are communicatively coupled to each other via a network 108. An example of network 108 is the Internet, but in various implementations (e.g., the cloud), one or more of the computing device 102, service provider system 104, and component identification module 106 may use one or more different connections or different networks to be communicatively coupled.

[0050] Although computing device 102 is depicted in environment 100 as separate from service provider system 104 and component identification module 106, in one or more implementations, all or parts of component identification module 106 are implemented at computing device 102 and / or service provider system 104, or are implemented by computing device 102 and / or service provider system 104. In at least one implementation, for example, at least a portion of component identification module 106 is implemented by application 110 of computing device 102 and / or using various resources of computing device 102, such as hardware resources, operating system, firmware, etc. Additionally or alternatively, at least a portion of component identification module 106 is implemented by resources of service provider system 104 (e.g., server-based storage, processing, etc.). Additionally or alternatively, at least a portion of component identification module 106 is implemented using third-party services (e.g., providing one or more hardware and / or other computing resources to support a web service platform provided by a web service provider).

[0051] The computing device implementing Environment 100 can be configured in a variety of ways. For example, the computing device can be configured as a desktop computer, laptop computer, mobile device (e.g., a handheld configuration such as a tablet or mobile phone), Internet of Things (IoT) device, wearable device (e.g., a smartwatch, ring, or smart glasses), augmented reality (AR) / virtual reality (VR) device (e.g., smart glasses), server, etc. Therefore, the range of computing devices extends from fully-resourced devices with abundant memory and processor resources to low-resource devices with limited memory and / or processing resources. Furthermore, although the computing device is referred to in the singular form in the examples discussed below, it also refers to multiple different devices, such as multiple servers in a server cluster used to perform operations "in the cloud," as in the context of… Figure 6 Further description.

[0052] In at least one implementation, application 110 supports data communication across network 108, such as data communication between computing device 102 and service provider system 104 and / or between computing device 102 and component identification module 106. By supporting such data communication, application 110 provides access to online marketplace 112 to the corresponding users of computing device 102 (and users of other computing devices). For example, computing device 102 receives data from service provider system 104. Based on the received data, application 110 causes various systems of computing device 102 to output the user interface of online marketplace 112, for example, by displaying the user interface via a display device or by creating an accessible voice-based user interface.

[0053] Through user interaction with computing device 102, application 110 receives user input via one or more user interfaces of online marketplace 112. Examples of such input include, but are not limited to: receiving touch input associated with a portion of the displayed user interface, receiving one or more voice commands, receiving typed input (e.g., via a physical or virtual (“soft” keyboard), receiving mouse or stylus input, etc. An example of application 110 is a browser operable to: navigate to a website of online marketplace 112, display pages of the website, and facilitate user interaction with web pages of the website of online marketplace 112. Another example of application 110 is a web-based computer application of online marketplace 112, such as a mobile application or a desktop application. Without departing from the spirit or scope of the techniques described herein, application 110 may be configured in various ways that enable users to interact with their computing devices and perform actions on online marketplace 112 through extensions.

[0054] In one or more implementations, a user registers with service provider system 104 to obtain a corresponding user account for online marketplace 112. Such registration may include, for example, providing an email address and establishing a username and password combination. After registering with service provider system 104, a computing device (e.g., computing device 102) may facilitate logging into or otherwise authenticating to the user account in various ways, such as by receiving a username and matching password, receiving biometric information appropriately matched to stored biometric information associated with the user account (e.g., at least one captured image of the face or captured information of another body part such as a thumb or finger), etc. However, in at least some cases, the user account through which the user accesses online marketplace 112 may be a guest account, which does not require the user to log in or otherwise authenticate to an established account before interacting with online marketplace 112.

[0055] In a broader sense, online marketplace 112 is configured to generate lists 114 of items 116 (e.g., also referred to herein as item lists and product lists) and expose these lists 114 across network 108 to (e.g., publish the lists 114) one or more computing devices, including computing device 102. For example, online marketplace 112 may generate lists 114 of items 116 for sale and expose these lists to computing devices, allowing users of the computing devices to interact with the lists via a user interface to initiate transactions related to the respective items 116 or items 116 in the list 114 (e.g., buy, sell, add to a desired list, share, etc.). According to the described technology, the online marketplace 112 is configured to generate a list 114 for one or more types of physical goods or property (e.g., clothing and / or clothing accessories, collectibles, furniture, decorative items, textiles, luxury goods, electronics, health and dietary supplements, food, real estate, physical computer-readable storage devices with one or more video games stored thereon, etc.), services (e.g., childcare, dog walking, house cleaning, etc.), digital items that can be downloaded via the network 108 (e.g., digital images, digital music, digital videos), and blockchain-backed assets (e.g., non-fungible tokens (NFTs)), to name just a few.

[0056] In the illustrated environment 100, the online marketplace 112 may include a storage device 118 depicted as maintaining real-time list data 120. Real-time list data 120 includes a set of lists 114 of the online marketplace 112. Storage device 118 may represent one or more databases and / or other types of storage capable of storing the real-time list data 120. Examples of storage device 118 include, but are not limited to, mass storage and virtual storage. For example, in one or more implementations, storage device 118 may be virtualized across multiple data centers and / or cloud-based storage devices. Service provider system 104 may implement the online marketplace 112 using servers that execute storage instructions to deploy various services of service provider system 104, enabling these services to perform substantial computations that effectively provide the functionality described above and below. It should be understood that the online marketplace 112 may include more, fewer, or different components without departing from the spirit or scope described herein.

[0057] In one or more implementations, the online marketplace 112 may be accessed by a decentralized computing device corresponding to a “customer” of the online marketplace 112 (e.g., a user with an account on the online marketplace 112 and / or a user accessing the online marketplace as a “guest” who is not signed to such an account or tracked as a user with an account). In at least some cases, apart from the provision of account and system guardrails implemented by various aspects of the online marketplace 112 (e.g., the user interface of application 110), the online marketplace 112 generally does not control the actions of users using the functions of the online marketplace 112 to list items on it. For example, multiple (e.g., a majority) users of the online marketplace 112 may not be employed by or otherwise similarly controlled by a company associated with the online marketplace 112. In this way, users of the online marketplace 112 can exert more control over the items 116 listed using the online marketplace 112 (e.g., those items 116 that users decide to list through the online marketplace 112) than the company associated with the online marketplace 112 (or its employees or agents).

[0058] Users who list item 116 on online marketplace 112 can be referred to as “sellers,” while users who purchase or otherwise acquire item 116 listed on online marketplace 112 through their listings can be referred to as “buyers.” Both sellers and buyers interact with the user interface of online marketplace 112 (e.g., via application 110) to perform desired functions. Additionally, individual users of online marketplace 112 can interact via an interface to become both sellers and buyers on online marketplace 112, for example, by interacting with the user interface to list one or more items 116 on online marketplace 112, and by interacting with the user interface to purchase one or more items 116 from the list on online marketplace 112.

[0059] For example, a seller user can interact with one or more user interfaces on the online marketplace 112 (e.g., via application 110) to provide information about one or more items 116 that the user has listed on the online marketplace 112. Such a user interface may include prompts that instruct or guide the seller user to provide information 122 about the listed items 116. Examples of such interface prompts for seller information and the information provided by these users include, but are not limited to: title, description (of item 116), one or more prices (e.g., buy item 116 now and / or minimum starting price for item 116), brand information, size, year, color, shipping information (e.g., fees and / or available types), delivery information, return information, payment information, image 124, video, model, authenticity information, item history (e.g., chain of custody), and condition (of item 116), to name just a few.

[0060] One or more parts of such information may be referred to as attributes of list 114 in this document. For example, the title of list 114 may be an attribute of list 114, the description of the listed item 116 may be an attribute of list 114, uploading or selecting one or more images for list 114 may be one or more attributes of list 114, the color of item 116 may be an attribute of list 114, one or more categories of item 116 may be an attribute of list 114, and so on.

[0061] In one or more implementations, the categories of online marketplace 112 include a category hierarchy (e.g., a tree structure), where more specific subcategories (e.g., smartphones or vitamins) belong to more general parent categories (e.g., electronics or health and dietary supplements, respectively). Alternatively or additionally, the categories of online marketplace 112 include multiple industry categories (e.g., sports), each industry category including one or more top-level or root categories (e.g., sports memorabilia, sporting goods). Given this, the categories associated with list 114 of item 116 may include categories in the category hierarchy to which item 116 belongs (e.g., from root category to lowest level or leaf category), and one or more industry categories to which item 116 belongs.

[0062] In one or more implementations, the online marketplace 112 stores and maintains information 122 of list 114 in storage device 118 in fields of a data structure or data record for populating list 114, wherein a given field and the information 122 populated and maintained for that field correspond to a specific attribute of list 114. For example, the “title” field of such a data structure or data record may be populated with information 122 (e.g., text) entered by the seller of list 114 into the user interface. The title field and the information 122 entered by the user as the title of list 114 correspond to an attribute of list 114, such as a title attribute. In one or more implementations, the online marketplace 112 may derive one or more attributes of list 114 and then populate them, for example, by processing one or more portions of information entered by the user to populate one or more corresponding attributes of list 114.

[0063] The component identification module 106 may support an LLM 126 and a customer support module 128. The LLM 126 may be a multimodal LLM capable of handling multiple types of input, such as text, images 124, and video. For example, a multimodal LLM may receive an image (e.g., as input) and output text. In some implementations, the LLM 126 may be trained on a variety of data including component regulations for different geographic regions. For example, the LLM 126 may be trained on a set of rules or a database indicating which health and dietary supplements, foods, ingredients, or other components are absolutely prohibited or prohibited in a particular country in amounts exceeding a specific threshold. In the United States, this may include a database of FDA-unapproved ingredients, a database of FDA-banned ingredients, a database of prohibited and restricted items from U.S. Customs and Border Protection (CBP), and a list of controlled substances from the Drug Enforcement Administration (DEA). Alternatively or concurrently, the database may include the European Medicines Agency (EMA) database, Operation Pangoal (managed by Interpol), and other examples. Customer support module 128 can provide a platform for one or more customer support agents (e.g., including compliance officers or other management staff in the online marketplace) to review list 114.

[0064] According to the described technology, a seller may list one or more items 116 or products for sale on an online marketplace 112. Item 116 may include dietary supplements, health supplements, and food. For example, item 116 may be listed under the Health and Dietary Supplements category or the Food category. In the corresponding list 114, the seller may include information 122 related to item 116, which includes a title (e.g., the name of item 116), a description (e.g., including information such as the manufacturer of item 116, its uses, etc.), and one or more images 124. At least one of the images 124 may depict item 116 in its packaged state (e.g., a bottle or other container or encapsulation), which may display some text. For example, the front of the package may depict the name, manufacturer, and other information of item 116, while the back of the package may depict an ingredient panel listing each component included in item 116 and the weight (e.g., in mg) or percentage of each component.

[0065] Once the seller publishes a list 114 of item 116, the component identification module 106 can receive the image 124 included in the list 114. Alternatively, the component identification module 106 can obtain the image 124 for text recognition before publishing the list 114 (e.g., when the seller uploads the image 114 during the process of entering list information). In such a case, if item 116 includes a prohibited component or is prohibited itself, the list 114 may not be published.

[0066] The component identification module 106 can generate prompts for the LLM 126 to analyze the components of item 116 in list 114 to identify whether any of the components is prohibited in a specific geographic region (e.g., country). Alternatively, the prompts can be predefined for the LLM 126. For example, the prompt could be “If the amount of an ingredient is greater than a certain number, what is the ingredient and its amount?”, where that number can be predefined. In this way, the component identification module 106 can prompt the LLM 126 to identify the components or ingredients that make up item 116 from image 124. For example, if image 124 depicts the packaging of item 116, the LLM 126 can recognize text on the packaging and identify the components or ingredients of item 116.

[0067] Based on prompting for LLM 126, component identification module 106 can output a list (e.g., in text format) of components identified from image 124. In some examples, component identification module 106 can also output the corresponding weight or percentage composition of each component. In some implementations, component identification module 106 can identify a target name identifier from a generic description (e.g., brand name or non-scientific name), which enables the disclosed technology to identify regulated substances in a list of items or images uploaded by a seller to compare the target component with a relevant database. In some cases, the standard, internationally recognized name of the active ingredient may be a name that is regulated, monitored, and listed in a database. For example, methylphenidate may be regulated in the United States and listed as a Schedule II controlled substance, and is used to treat ADHD and narcolepsy, where methylphenidate is the name of the substance used in regulatory documents, customs forms, and medical records. Brand names of the drug may include Ritalin (Novartis), Concerta (Janssen), Daytrana (patch version), Equasym (Europe), and Metadate CD (USA). However, these names may not be directly labeled unless the base ingredient (such as methylphenidate) is matched.

[0068] Additionally, the component identification module 106 can output list 114 based on a determination that a component should be approved or rejected. For example, if all components are permitted in a specified geographic area, the component identification module 106 can instruct item 116 to be approved for sale on the online marketplace, and list 114 will be permitted. Alternatively, if any component is prohibited in a specified geographic area, the component identification module 106 can instruct item 116 to be rejected, and list 114 will be removed from the online marketplace 112.

[0069] The system can monitor where the seller is located and from where they ship item 116, as well as where the seller attempts to sell item 116 and ship it. Therefore, the described technology can support product component identification when the seller is located in a first country (e.g., the United States) and lists item 116 for sale in a second country (e.g., France). In some examples, a component may be allowed to be listed for sale in the first country but not in the second. Alternatively, the component may be allowed in both countries, but only below a certain threshold in the second country. In such cases, LLM 126 can be trained to detect these types of differences, and component identification module 105 can allow the same item 116 to be listed for sale in the first country but not in the second country.

[0070] Having already considered examples of environments, we will now discuss some examples of techniques for component identification using AI-based systems, based on one or more implementations.

[0071] Figure 2 Examples of an AI-based system 200 for product component identification, according to various aspects of this disclosure, are depicted. The AI-based system 200 can be used as described in reference... Figure 1 The computing device 102, service provider system 104, and component identification module 106 described herein are implemented in or otherwise supported by the computing device 102, service provider system 104, and component identification module 106. For example, the AI-based system 200 may include image storage 204 (which may be an example of storage device 118), LLM 206 (which may be an example of LLM 126), and online marketplace 216 (which may be an example of online marketplace 112).

[0072] In the AI-based system 200, a seller can list one or more items or products for sale on an online marketplace 112. For example, a seller can list products for sale (e.g., edible products) under the health and dietary supplements category (e.g., vitamins) or the food category (e.g., beverages). In the corresponding list, the seller can include relevant information about the product, including a title (e.g., the product name), a description (e.g., information such as the product's manufacturer, its uses, etc.), and one or more images 202, which can be references. Figure 1 Examples of images 124 are described. In some examples, the seller may have taken the images using a computing device (e.g., a smartphone, tablet, or laptop camera), or the images may have been uploaded from an image database accessible to the seller.

[0073] At least one of the images in 202 may depict a product in its packaged state, which may display some text. For example, image 202 may depict a vitamin bottle (e.g., listed under the Health & Dietary Supplements category), where the front and / or back of the bottle may depict information about the vitamins. For example, the bottle may display the name (e.g., MaxPerformance Gummies), manufacturer, ingredient information (e.g., a "Supplement Ingredients" panel), and other information. The ingredient information may list serving sizes and the number of servings per container, a list of components or ingredients included in the product, the amount of each component (e.g., by weight or percentage composition), etc. (See reference) Figure 3 An example describing image 202.

[0074] Image 202 can be uploaded to image storage 204, which may represent one or more databases and / or other types of storage devices capable of storing real-time image data. Image storage 204 can store image data of any number of product lists in different categories; however, only images related to health and dietary supplements and food will be provided to LLM 206.

[0075] LLM 206 can be a multimodal LLM capable of handling multiple types of input, such as text, images 202, and video. In some implementations, LLM 206 can be trained on a variety of data including ingredient regulations from different geographic regions. For example, LLM 206 can be trained on a set of rules (e.g., a database lookup) that indicates which health and dietary supplements, foods, ingredients, or other components are absolutely prohibited or prohibited in excess of a threshold amount in a particular country. This data can include various modalities or types of data, including textual data (e.g., names, ingredients, etc.) and image data (e.g., images of dietary supplements). In some examples, the data can be manipulated to be used more effectively by LLM 206. In some implementations, LLM 206 can also be trained on health and dietary supplements and foods first listed on an online marketplace 216, enabling LLM 206 to subsequently identify new relevant products or components and more accurately identify components. Alternatively or additionally, LLM 206 can be trained to identify target identifiers from generic names or brand names. For example, LLM 206 can be trained to identify the compound methylphenidate from brand names such as Ritalin (Novartis), Concerta (Janssen), Daytrana (patch version), Equasym (Europe), and Metadate CD (USA).

[0076] Image 202 can be used to generate a prompt for LLM 206. This prompt can be an input in the form of a question, instruction, or other text format (e.g., JavaScript Object Notation (JSON)) that guides LLM 206 to generate a specific response or output. For example, see reference... Figure 4 As described, the prompt could include text such as, “If the amount of an ingredient is > 0, what is the ingredient and its quantity? Format the output as a JSON object.” Such a prompt could be generated based on the system recognizing that image 202 contains some text (e.g., a list of ingredients). In some implementations, the prompt can be generated based on image 202 by a model other than LLM 206 (e.g., another LLM or other multimodal model) – a model trained to generate prompts instructing the LLM to identify product components. In some examples, the prompt can differ depending on whether the product is a health and dietary supplement or a food. For example, for health and dietary supplements, the prompt might use specific language to instruct LLM 206 to look for “active ingredient” or “compound,” but for food, it might look for “ingredients,” as the text in image 202 is more likely to be labeled that way (e.g., on product packaging).

[0077] The prompt may instruct the LLM 206 to identify the ingredients or components of the product and their corresponding quantities based on image 202, and provide the output in JSON format. The prompt instructs the LLM 206 to include the component and its quantity (e.g., by weight or percentage composition) if the quantity of the component is greater than zero (e.g., 0 mg or 0%). Additionally, the prompt may include instructions to trigger the identification of two types of components: components that are fully restricted (e.g., at any percentage or weight); and components that are restricted only if they include quantities exceeding a threshold. Additionally or alternatively, the prompt may include specific language or instructions based on the seller's geographic location (e.g., where the seller will ship the product) or the potential buyer's geographic location (e.g., where the seller will ship the product to).

[0078] The prompt can be modified to obtain different results from LLM 206. For example, a user can change the prompt to look for compounds above a certain percentage instead of above a certain weight, or a user can change the specified percentage or weight. In some examples, the prompt can be predefined for LLM 206. For example, the prompt could default to "If the amount of an ingredient is > 0, what is the ingredient and its amount? Format the output as a JSON object." Additionally or alternatively, the prompt can ask LLM 206 to identify specific components or products that are known to be prohibited.

[0079] The system can prompt LLM 206 to identify product components from image 202. For example, if image 202 depicts a "supplement ingredient" or other ingredient information, LLM 206 can recognize text in the image and identify the component and its corresponding quantity from that text. Additionally, LLM 206 can recognize text in languages ​​other than English from images, allowing it to identify components used in many languages ​​across many different geographic regions. In some examples, LLM 206 can identify only the components and avoid recognizing any other text on the packaging (such as serving size, product name, and any other text that does not include the components). In this way, LLM 206 can identify which images in image 202 include text related to the components and which do not.

[0080] Based on the prompts, LLM 206 can output ingredient text 208, which includes text recognized from image 202. For example, if image 202 shows the ingredients of a vitamin supplement, ingredient text 208 can include serving size (e.g., 2 gummies) and servings per container (e.g., 30 gummies). Additionally, the ingredient text 208 may include a list of ingredients (e.g., components) per serving, which may include the following information: calories (e.g., 10), total carbohydrates (3 grams (g) or 1%), total sugars (2g, including 2g of added sugar or 4% of the daily intake (DV)), vitamin B6 (e.g., as pyridoxine hydrochloride, 1 mg (29.5%)), folic acid (as 240 mcg folic acid, 400 mcg DFE (50%)), vitamin B12 (as cyanocobalamin, 2.4 mcg (50%)), iodine (as potassium iodide, 100 mcg (33.5%)), apple cider vinegar (e.g., 1000 mg (5% total acid)), pomegranate juice powder (80 mg), and beetroot juice powder (80 mg). Additionally, ingredient text 208 may list other ingredients (with or without amounts), such as glucose, sucrose, water, glucose, pectin, natural flavoring agents (e.g., apple), sodium citrate, citric acid, varnishes (e.g., carnauba wax and vegetable oil), unfiltered liquid apple cider vinegar, purple carrot concentrate, and beta-carotene (1%).

[0081] The ingredient text 208 can be compared with information in the ingredient lookup service 210. The ingredient lookup service 210 can support datasets of ingredient regulations for geographic regions. For example, regarding products listed for sale in the United States, the ingredient lookup service 210 can support a database 212 of FDA-unapproved ingredients (e.g., ingredients or compounds not yet processed by the FDA) and a database 214 of FDA-restricted ingredients. If a product's compounds or the product itself are listed in such databases, the product is prohibited from being sold on the online marketplace 216. Therefore, the system can determine whether a product is approved or rejected for listing on the online marketplace 216 based on whether any of its components are included in the FDA-unapproved ingredients database 212 or the FDA-restricted ingredients database 214.

[0082] The databases 212 for FDA-unapproved ingredients and 214 for FDA-banned ingredients can be managed by FDA administrators and updated regularly as new ingredients are developed or used in products in the health and dietary supplement or food categories. For example, a database managed by another country or state could send a notification to update the US database to add new ingredients, even if the database is managed by the US federal government. In such cases, the system can automatically update and retrain the LLM 206 using the updated input.

[0083] In some implementations, the system can determine product approval or rejection based on a database of target components with specific percentages, weights, or percentage or weight ranges (e.g., a database 212 of FDA-unapproved ingredients, a database 214 of FDA-banned ingredients, etc.). For example, LLM 206 can determine that the target component is below a threshold and is therefore permitted. In another example, LLM 206 can determine that the target component is above a threshold and is therefore rejected. In this way, in addition to identifying a one-to-one mapping between the text identified from image 202 and the database, LLM 206 is also able to identify the amount and range of components and perform a more granular analysis of the product.

[0084] Different regions or countries can utilize different databases. In addition to the US FDA-unapproved ingredients database 212 or FDA-banned ingredients database 214, the AI-based system 200 can support the US DEA Controlled Substances Schedule and the US CBP's database of prohibited and restricted items. Furthermore, the AI-based system 200 can support databases from the European Medicines Agency (EMA), Operation Pangoal (managed by Interpol), Health Canada's Drug Database (DPD), Australia's Therapeutic Goods Administration (TGA), Japan's Pharmaceuticals and Medical Devices Agency (PMDA), and South Korea's Ministry of Food and Drug Safety (MFDS), among others.

[0085] Additionally or alternatively, ingredient text 208 can be compared with a set of rules indicating whether a component is permitted in a geographic region. The system can determine whether a product is approved or rejected for listing on the online marketplace 216 based on whether any of its components fails to meet a rule. For example, a rule could stipulate that component X is prohibited in country Y if it exceeds the percentage of composition Z. This set of rules can be managed by the administrator of the online marketplace 216.

[0086] In some examples, if a seller is listing a product for sale in the United States, LLM 206 can be trained to identify ingredients or components based on which specific state the seller is selling to. For instance, if the seller's product includes cannabidiol (CBD) or a similar substance, LLM 206 can be trained to detect that CBD is permitted in some states (e.g., Colorado) but prohibited in others. In another example, LLM 206 can be trained on different state databases of alcohol sales rules (e.g., the Florida Department of Agriculture and Consumer Services (FDACS) database) and can be used to allow or prohibit certain products from being listed and sold in certain states.

[0087] In addition to the ingredient text 208, the system can also output a list based on the determination that a component should be allowed or rejected. In some examples, the output can also indicate the threshold amount if a component is allowed but only in amounts below a threshold. For example, the output could indicate that, according to the FDA Unapproved Ingredients Database 212 or the FDA Banned Ingredients Database 214, a component is allowed when its percentage composition is below 50%, but prohibited when it exceeds 50%.

[0088] In some implementations, a product can be automatically rejected and its corresponding listing removed from the online marketplace 216, based on a comparison of the components in ingredient text 208 with the ingredient lookup service 210. A product can also be automatically rejected if any component in ingredient text 208 is included in the FDA-unapproved ingredients database 212 or the FDA-banned ingredients database 214. In some examples, the product itself may be listed as a banned product in the FDA database. In such cases, the product can also be automatically rejected and removed from the online marketplace (if already published), or it may not be permitted for publication in the first place.

[0089] Alternatively, a product may be approved based on a comparison of the components in ingredient text 208 with ingredient lookup service 210. In such a case, the product may be listed for sale on online marketplace 216. For example, if none of the product's components are included in FDA-unapproved ingredients database 212 or FDA-banned ingredients database 214, the product may be approved.

[0090] In some implementations, the results of comparisons between ingredient text 208 and / or a database supported by ingredient lookup service 210 can be sent to customer support agent 218 for manual review. Customer support agent 218 can review the provided information and determine whether the product list is approved or rejected. Such manual review may be utilized, for example, if the product is relatively new or if some components are excluded from the FDA database. The system may forward product lists including new products or components that LLM 206 has not yet encountered or been trained on. Additionally or alternatively, customer support agent 218 may review the list on a random basis to check if the system has correctly approved or rejected the list. Customer support agent 218 may verify each individual component in ingredient text 208 against a database 212 of FDA-unapproved ingredients or a database 214 of FDA-banned ingredients and approve or reject the list accordingly. In some examples, feedback from customer support agent 218 can be used to continuously retrain LLM 206 and improve the model's ability to identify new components in health and dietary supplements or foods.

[0091] Figure 3 An example of image 300 depicting a product listed on an online marketplace according to various aspects of this disclosure. Image 300 may be as shown in the reference. Figure 2 Example of the described image 202. In this example, image 300 may depict an advertisement or packaging for a health and dietary supplement, although the product may be any item under the health and dietary supplement or food category.

[0092] Image 300 may depict details about the composition of the dietary supplement under headings such as “Supplement Ingredients,” “Active Ingredients,” or some other title. In some examples, these details may include serving size (e.g., 2 capsules) and number of servings per container (e.g., 30 servings). Furthermore, these details may include a list of ingredients or components 302 that make up the dietary supplement in what quantities. In the example of image 300, the amount of each component 302 may be given in terms of amount per serving (e.g., weight in mg or mcg) and percentage DV; however, this amount may additionally or alternatively be provided in terms of percentage composition. For example, component 302 may include 90 mg of vitamin C (which is 100% DV), 25 mcg of vitamin D3 (which is 125% DV), 100 mg of turmeric (no DV, as indicated by an asterisk), and 350 mg of a multimineral mixture (no DV). In some cases, component 302 (e.g., a multimineral mixture) may be a mixture of ingredients that are also listed in the dietary supplement details. For example, a multi-mineral mixture may include apple cider vinegar, dandelion root extract, and other ingredients or compounds.

[0093] Image 300 may also include a depiction of the dietary supplement in bottle 304 and any number of badges or symbols (such as badge 306, natural ingredient symbol 308, “Made in the USA” symbol 310, Good Manufacturing Practice (GMP) certification symbol 312, and formula symbol 314).

[0094] LLM can recognize text from image 300 and from that text identify the corresponding dietary supplement component 302 or ingredient, and then display them as ingredient text (e.g., Figure 2 The component text (208) is output for comparison with a database of disabled components. Figure 3 In the example, the LLM can identify component 302, including any mixture of ingredients included in the supplement ingredients. That is, the LLM can identify Vitamin C, Vitamin D3, Turmeric, and a multi-mineral blend (including additional ingredients such as apple cider vinegar, dandelion root extract, and the multi-mineral blend) as component 302. The LLM can also identify the corresponding amount of each of these components 302. In this way, the LLM can distinguish the text of component 302 in image 300 from any other text, such as the text "Ultimate All-in-One Supplement Blend" on badge 306, symbols 308, 310, 312, and 314, the text "Supplement Blend" (e.g., title) and "Fortified" on bottle 304, and text in the supplement ingredients related to serving size.

[0095] Figure 4 An example of a user interface 400 for an AI-based system for product component identification, according to various aspects of this disclosure, is depicted.

[0096] Administrators of AI-based systems can utilize user interface 400 to generate prompts for LLM (Limited Management Model) to identify one or more components of health and dietary supplements or foods listed for sale in online marketplaces, as described herein. User interface 400 may include a tool input window 402, which includes a text field 404 and an image URL field 406. Administrators can type prompts in text field 404, such as “If the amount of an ingredient is > 0, what is the ingredient and its amount? Format the output as a JSON object.” That is, the prompt may instruct the LLM to identify the components or ingredients of the relevant product based on an image of the product, along with the corresponding amount of each component, and provide the output in JSON format. The prompt instructs the LLM to include the component and its amount (e.g., by weight or percentage composition) if the amount of the component is greater than zero (e.g., 0 mg or 0%). The prompt can be modified to obtain different results from the LLM, and in some examples, the prompt may be predefined (e.g., not typed into text field 404 at runtime). Administrators can type a URL corresponding to an image of the product in image URL field 406. For example, the URL may point to a list of the product in an online marketplace. Administrators can select the "Execute" button 408 to prompt the LLM (e.g., execute the LLM based on the prompt).

[0097] As prompted, the LLM can output an image preview 410 based on the information entered into the text field 404 and the image URL field 406. The image preview 410 can show an image 412 of the product (e.g., the same image uploaded by the user in a corresponding list that lists the product's components).

[0098] The AI ​​system can display output 414 via user interface 400. Output 414 can be generated partly by the LLM and partly by the AI ​​system itself. In an example of user interface 400, output 414 may include a list of components identified by the LLM and their corresponding amounts, in text format (e.g., JSON). For example, the list of components may include “Vitamin C” in an amount of “90 mg”. The AI ​​system can then compare the output components with a database of prohibited components (e.g., a database of FDA-unapproved ingredients and a database of FDA-banned ingredients) and / or a set of rules (e.g., a lookup database) that indicates which health and dietary supplements, foods, ingredients, or other components are absolutely prohibited or prohibited in a particular country if they exceed a threshold amount.

[0099] The AI ​​system can include in output 414 a “permitted” status for each component, indicating whether the component is permitted at that level (e.g., “true”), and a “listed status” indicating overall permitted (assuming all other components identified by the LLM are also permitted). If the AI ​​system determines a component is prohibited based on rules, output 414 can indicate a “permitted” status as “false”, and the “listed status” will be “blocked”. In some examples, output 414 can indicate that the product itself is prohibited. For example, the LLM can determine that the product itself is prohibited before analyzing individual components. Additionally, output 414 can indicate that the LLM determines why a product is permitted or prohibited. For example, for a prohibited item, the reasoning process could state, “The identified component ‘Biphentin’ is in the provided image and is the Canadian brand name for the active ingredient methylphenidate, which is a Schedule II controlled substance in many countries, including the United States.” For a permitted item, the reasoning process could state, “The identified component vitamin C is in the provided image and is permitted in the United States.”

[0100] Exemplary details of AI-based systems for product component identification have already been discussed; now, some examples of the process are considered to illustrate additional aspects of the technology.

[0101] Example process

[0102] This section describes examples of processes for an AI-based system for product component identification. Aspects of the process can be implemented in hardware, firmware, or software, or a combination thereof. The process is shown as a set of blocks specifying operations to be performed by one or more devices, and is not necessarily limited to the order in which the operations are performed by the respective blocks as shown.

[0103] Figure 5 Process 500 is described in an example implementation of an AI-based system for product component identification.

[0104] Receive one or more images associated with a product listing on an online marketplace, wherein the product listing pertains to products in at least one of the categories of health and dietary supplements or food (block 502). By way of example, a user (e.g., a seller) may post a listing 114 of products for sale on online marketplace 112. The products may be health and dietary supplements (e.g., vitamins) or food (e.g., beverages). Listing 114 may include one or more images 124 of the products.

[0105] Generate a prompt for LLM based on one or more images (block 504). By way of example, the prompt may include text instructions for LLM to identify one or more components (e.g., ingredients) from image 124.

[0106] LLM performs based on prompts to identify one or more components of a product from one or more images (block 506). By way of example, LLM can recognize text in image 124 to identify components. The text may include a list of ingredients or “supplement ingredients” on the product packaging shown in image 124. In some examples, LLM can also identify the amount of each component in the product (e.g., weight or percentage composition).

[0107] The system generates output based on the execution of LLM, where the output indicates whether one or more components identified from one or more images and the product list are approved or rejected (block 508). By way of example, the system can determine whether a product list is approved or rejected based on comparing the components identified by the LLM with a set of rules indicating which components (and in some cases, which products themselves) are permitted in different geographic regions at what quantities.

[0108] Examples of processes according to one or more implementations have been described; now, examples of systems and devices that can be used to implement the various technologies described herein are considered.

[0109] Figure 6 An example of a system 600 is shown, which typically includes a computing device 602, representing one or more computing systems and / or devices that can implement the various technologies described herein. This is illustrated by including an application 110 and a component identification module 106. The computing device 602 can be, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system-on-a-chip, and / or any other suitable computing device or computing system.

[0110] The example computing device 602 shown includes a processing system 604 communicatively coupled to each other, one or more computer-readable media 606, and one or more I / O interfaces 608. Although not shown, the computing device 602 may also include a system bus or other data and command transfer system that couples various components to each other. The system bus may include different bus architectures such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or any or a combination of a processor or local bus utilizing any of these various bus architectures. Various other examples, such as control lines and data lines, are also contemplated.

[0111] Processing system 604 represents the functionality of performing one or more operations using hardware. Therefore, processing system 604 is shown as including hardware elements 610 that can be configured as processors, function blocks, etc. This can include a hardware implementation as an application-specific integrated circuit or other logic device formed using one or more semiconductors. Hardware elements 610 are not limited by the materials forming them or the processing mechanisms employed therein. For example, a processor can include semiconductors and / or transistors (e.g., integrated circuits (ICs)). In such a case, processor-executable instructions can be electronically executable instructions.

[0112] Computer-readable medium 606 is shown as including memory / storage device 612. Memory / storage device 612 represents a memory / storage capacity associated with one or more computer-readable media. Memory / storage device 612 may include volatile media (e.g., random access memory (RAM)) and / or non-volatile media (e.g., read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). Memory / storage device 612 may include fixed media (e.g., RAM, ROM, fixed hard disk drive, etc.) and removable media (e.g., flash memory, removable hard disk drive, optical disk, etc.). Computer-readable medium 606 may be configured in various other ways, as further described below.

[0113] Input / output interface 608 represents a function that enables a user to input commands and information into computing device 602 and also enables the presentation of information to the user and / or other components or devices using various input / output devices. Examples of input devices include keyboards, cursor control devices (e.g., mice), microphones, scanners, touch functionality (e.g., capacitive sensors or other sensors configured to detect physical touch), camera devices (e.g., which may employ visible or invisible wavelengths, such as infrared frequencies, to recognize movement as gestures not involving touch), etc. Examples of output devices include display devices (e.g., monitors or projectors), speakers, printers, network interface cards, haptic-responsive devices, etc. Therefore, computing device 602 can be configured in various ways to support user interaction, as further described below.

[0114] This document describes various technologies in the general context of software, hardware components, or program modules. Typically, such modules include routines, programs, objects, components, parts, data structures, etc., that perform specific tasks or implement specific abstract data types. The terms “module,” “function,” and “part” as used herein generally refer to software, firmware, hardware, or a combination thereof. The technologies described herein are characterized as platform-independent, meaning that they can be implemented on a variety of commercial computing platforms with various processors.

[0115] Implementations of the described modules and technologies may be stored on or transmitted across some form of computer-readable medium. The computer-readable medium may include a variety of media accessible by the computing device 602. By way of example, and not limitation, the computer-readable medium may include both "computer-readable storage media" and "computer-readable signal media".

[0116] "Computer-readable storage medium" can refer to a medium and / or device that enables the persistent and / or non-transitory storage of information, compared to mere signal transmission, carrier waves, or signals themselves. Therefore, a computer-readable storage medium refers to a non-signal-bearing medium. Computer-readable storage media include hardware, such as volatile and non-volatile, removable and non-removable media, and / or storage devices implemented with methods or techniques suitable for storing information such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to: RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, hard disks, magnetic tape cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or other storage devices, tangible media, or articles of art suitable for storing desired information and accessible by a computer.

[0117] "Computer-readable signal medium" can refer to a signal-bearing medium configured to send instructions to the hardware of computing device 602, for example, via a network. Signal media can typically embody computer-readable instructions, data structures, program modules, or other data as modulated data signals, such as carrier waves, data signals, or other transmission mechanisms. Signal media also includes any information delivery medium. The term "modulated data signal" refers to a signal whose characteristics are set or altered in a manner that encodes information in the signal. By way of example, and not limitation, communication media include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0118] As previously described, hardware element 610 and computer-readable medium 606 represent modules, programmable device logic, and / or fixed device logic implemented in hardware that can be employed in some embodiments to implement at least some aspects of the techniques described herein (e.g., executing one or more instructions). The hardware may include components of integrated circuits or systems-on-a-chip, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or other hardware. In this context, the hardware may operate as a processing device to perform program tasks defined by instructions and / or logic embodied in the hardware, as well as hardware for storing instructions for execution (e.g., the previously described computer-readable storage medium).

[0119] The various techniques described herein can also be implemented using the combinations described above. Therefore, software, hardware, or executable modules can be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage medium, and / or implemented by one or more hardware elements 610. Computing device 602 can be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Therefore, the implementation of a module executable as software by computing device 602 can be at least partially implemented in hardware, for example, by using a computer-readable storage medium and / or hardware elements 610 of processing system 604. Instructions and / or functions can be executed / operated by one or more articles of art (e.g., one or more computing devices 602 and / or processing systems 604) to implement the techniques, modules, and examples described herein.

[0120] The techniques described herein can be supported by various configurations of computing device 602, and are not limited to specific examples of the techniques described herein. This functionality can also be implemented, in whole or in part, using a distributed system, such as via platform 616 on “cloud” 614 as described below.

[0121] Cloud 614 includes and / or represents platform 616 for resource 618. Platform 616 abstracts the underlying functionality of the hardware (e.g., server) and software resources of cloud 614. Resource 618 may include applications and / or data that can be utilized when performing computer processing on a server remote from computing device 602. Resource 618 may also include services provided via the Internet and / or via subscriber networks such as cellular networks or Wi-Fi networks.

[0122] Platform 616 can abstract resources and functions to connect computing device 602 to other computing devices. Platform 616 can also be used to abstract resource scaling to provide corresponding levels of scaling to meet the demands encountered by resource 618 implemented via platform 616. Therefore, in the interconnect device implementation, the implementation of the functions described herein can be distributed throughout system 600. For example, functions can be implemented partly on computing device 602 and partly via platform 616, which abstracts the functions of cloud 614.

[0123] in conclusion

[0124] Although the systems and techniques have been described in language specific to structural features and / or methodological actions, it should be understood that the systems and techniques defined in the appended claims are not necessarily limited to the specific features or actions described. Rather, the specific features and actions are disclosed as exemplary forms for implementing the claimed subject matter.

Claims

1. A computer-implemented method, comprising: Receive one or more images associated with a product list in an online marketplace, wherein the product list refers to products in at least one of the health and dietary supplements category or the food category; Based on the one or more images, generate prompts for a large language model (LLM); Based on the prompt, the LLM is executed to identify one or more components of the product from the one or more images; and Output is generated based on the execution of the LLM, wherein the output indicates the one or more components identified from the one or more images and a determination of whether the product list is approved or rejected.

2. The computer-implemented method according to claim 1 further includes: Compare the one or more components with a dataset that includes component regulations for geographic regions; as well as The approval or rejection of the product list is determined based on the dataset indicating whether one or more components are permitted or prohibited in the geographic region.

3. The computer-implemented method according to claim 1 further includes: The one or more components are compared with a set of rules indicating whether the component is permitted in a geographic area; as well as The approval or rejection of the product list is determined based on whether the component is permitted in the geographic region according to the set of rules.

4. The computer-implemented method according to claim 1 further includes: The output is sent to the user for manual review, wherein the determination of whether the product list is approved or rejected is based on the manual review.

5. The computer-implemented method according to claim 1, further comprising: The product list is displayed to the user via a user interface based on an indication that the output has been rejected.

6. The computer-implemented method according to claim 1, wherein, Each component corresponds to the weight or percentage composition of the product.

7. The computer-implemented method according to claim 1, wherein, Identifying one or more components of the product includes identifying text in one or more images corresponding to the one or more components.

8. The computer-implemented method according to claim 1, wherein, The output indicates that none of the one or more images depict the one or more components.

9. The computer-implemented method according to claim 1, wherein, The product list is rejected based on the fact that at least one of the one or more components has a quantity greater than a threshold.

10. The computer-implemented method according to claim 1, further comprising: The product listing is removed from the online marketplace because it was rejected.

11. The computer-implemented method according to claim 1, wherein, The product list was rejected because the product itself was prohibited in the geographic region.

12. A system comprising: One or more processors; as well as The memory stores instructions that, when executed by the one or more processors, cause the system to: Receive one or more images associated with a product list in an online marketplace, wherein the product list refers to products in at least one of the health and dietary supplements category or the food category; Based on the one or more images, generate prompts for a large language model (LLM); Based on the prompt, the LLM is executed to identify one or more components of the product from the one or more images; and Output is generated based on the execution of the LLM, wherein the output indicates the one or more components identified from the one or more images and a determination of whether the product list is approved or rejected.

13. The system according to claim 12, wherein, The instructions also cause the system to: Compare the one or more components with a dataset that includes component regulations for geographic regions; and The approval or rejection of the product list is determined based on the dataset indicating whether one or more components are permitted or prohibited in the geographic region.

14. The system according to claim 12, wherein, The instructions also cause the system to: The one or more components are compared with a set of rules indicating whether the component is permitted in a geographic area; as well as The approval or rejection of the product list is determined based on whether the component is permitted in the geographic region according to the set of rules.

15. The system according to claim 12, wherein, The instructions also cause the system to: The output is sent to the user for manual review, wherein the determination of whether the product list is approved or rejected is based on the manual review.

16. The system according to claim 12, wherein, The instruction also causes the system to display the product list to the user via a user interface based on an indication that the output has been rejected.

17. The system according to claim 12, wherein, Each component corresponds to the weight or percentage composition of the product.

18. The system according to claim 12, wherein, In order to identify one or more components of the product, the instructions also cause the system to identify text in one or more images corresponding to the one or more components.

19. The system according to claim 12, wherein, The output indicates that none of the one or more images depict the one or more components.

20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations including: Receive one or more images associated with a product list in an online marketplace, wherein the product list refers to products in at least one of the health and dietary supplements category or the food category; Based on the one or more images, generate prompts for a large language model (LLM); Based on the prompt, the LLM is executed to identify one or more components of the product from the one or more images; and Output is generated based on the execution of the LLM, wherein the output indicates the one or more components identified from the one or more images and a determination of whether the product list is approved or rejected.

Images