Demand detection based on query image

Abstract

Embodiments of the present disclosure relate to demand detection based on query images. Systems and methods for detecting object demand based on query images are provided. An image annotation module generates multiple image tags for a query image. An image content analyzer module analyzes multiple query image tags based on a knowledge graph associated with an online platform to create query feature data. A topic identification module identifies one or more query topics based on aggregated query image feature data. A demand analysis module generates demand data indicating user demand for an object corresponding to the query topic by comparing the query topic with catalog data of the online platform.

Description

Technical Field

[0001] The present disclosure generally relates to generative artificial intelligence (AI). More particularly, but not by way of limitation, the present disclosure relates to demand detection based on a query image. Background Art

[0002] Digital platforms, such as search engines and various websites, provide users with search functionality to search for content of interest. Most commonly, users formulate search queries by typing keywords that describe the content of interest. Some platforms provide the ability to search by image. For example, a user may generate and optimize an image using generative image technology and then use the generated image as a search query. The digital platform may search its database and / or other Internet sources to identify content that is responsive to the query. Some digital platforms may track and report search trends by analyzing user search queries over a period of time. Search trends may provide intelligence about user attention at a high level and guide the platform or stakeholders to make informed decisions. Summary of the invention

[0003] Certain embodiments relate to detecting user demand for items based on a query image. Generative image technology is used to generate a query image based on a prompt input by a user (e.g., a prompt from a user computing device). The query image is provided to or generated by a demand detection server. The demand detection server processes the query image using image recognition technology and generates an image tag that describes the content of the query image. The demand detection server analyzes multiple image tags based on a knowledge graph associated with an online platform to obtain query image feature data, and the query image feature data includes inferred semantics of the image tags. The demand detection server identifies one or more query topics based on the aggregated query image feature data. The demand detection server detects user demand for at least one item by comparing one or more query topics with the catalog data of the online platform.

[0004] These illustrative embodiments are mentioned not to limit or define the present disclosure, but to provide examples to aid in understanding the present disclosure.Additional embodiments are discussed in the Detailed Description, and further description is provided therein. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The features, embodiments, and advantages of the present disclosure may be better understood when the following detailed description is read with reference to the accompanying drawings.

[0006] Figure 1 An example of a computing environment is depicted in which a need detection server detects user needs on an online platform based on a query image, according to certain embodiments of the present disclosure.

[0007] Figure 2 An example of a process for detecting user needs based on a query image in accordance with certain embodiments of the present disclosure is depicted.

[0008] Figure 3 is an example workflow for detecting user needs on an online platform according to certain embodiments of the present disclosure.

[0009] Figure 4 Depicted is an example of a GUI (graphical user interface) of an online platform for inputting user queries according to certain embodiments of the present disclosure.

[0010] Figure 5 Depicted is an example of a GUI for generating a query image in accordance with certain embodiments of the present disclosure.

[0011] Figure 6 Depicted is an example of a GUI for providing feedback regarding a generated query image, in accordance with certain embodiments of the present disclosure.

[0012] Figure 7 An example of a GUI illustrating a situation where the online platform 306 does not offer products that match or are sufficiently similar to a generated or regenerated query image is depicted.

[0013] Figure 8 An example of a computing system for implementing certain embodiments of the present disclosure is depicted. DETAILED DESCRIPTION

[0014] The present disclosure provides techniques for analyzing query images to detect unmet demand for items provided via online platforms. Online platform operators continuously improve their catalogs and related inventories to keep up with popular trends and changing user needs, thereby maintaining and expanding user groups. Curating online catalogs is often labor-intensive and also requires expertise, insight, and understanding of user behavior. In addition, identifying items that users are searching for but cannot find on the platform is critical to informing what items are added to the operator's catalog. This process typically requires manual sifting through scattered data (including keywords entered by users) to formulate their searches. There are limitations to the analysis of keywords because the keywords selected by users may not best describe the items in the user's mind.

[0015] In an embodiment of the present disclosure, a demand detection server may detect user demand based on a query image submitted to an online platform. The online platform queries its catalog (e.g., inventory or database) to determine whether it includes items that match the object shown in the query image, and displays the search results (or lack of search results) to the user. The online platform also provides the query image to the demand detection server for processing and analysis. The query image is processed and annotated by the demand detection server to indicate the object depicted in the image. Image tags are analyzed based on a knowledge database (e.g., a knowledge graph) associated with the catalog of the online platform to generate query image feature data including an inferred semantic description of the image tags. The semantic description identifies the context and attributes of the image tags. The resulting query image feature data is stored in a data store for further analysis by the demand detection server.

[0016] In one example, the online platform is a market platform that provides certain products. When the demand detection server processes and analyzes the query image, the query image feature data is clustered to identify common themes among the query images. Each query theme corresponds to a pattern or trend in shopper demand. The query theme is compared with the catalog data from the online platform to detect demand. For example, as a result of a large number of query images depicting cyan clothes for women, the query theme may be "cyan women's clothing". As a result of multiple query images showing women wearing short-sleeved shirts, another query theme obtained from the same query image set may be "short-sleeved women's shirts". If the catalog data from the online platform does not include cyan women's clothing, particularly cyan short-sleeved women's shirts, or enough short-sleeved women's shirts of various colors, the demand detection server will determine that there is a demand for such clothes. As the demand detection server processes more query images, more query image feature data can be obtained and further aggregated to determine whether the demand continues to be met or not and / or whether there are additional needs.

[0017] The demand data may be sent or displayed to the operator of the online platform. The operator of the online platform may decide to add items to the online platform catalog in response to the demand data indicating that the level of unmet product demand is sufficiently high. When an item is added to the online platform catalog, the online platform may generate a notification message and send it to users who have previously searched for items similar to the newly added item. Following the example above, after receiving the demand data, the operator of the online platform adds some long-sleeved and short-sleeved cyan women's tops to its catalog, as well as short-sleeved women's shirts in other colors in addition to the previously provided colors. The online platform then sends notification messages to users who have submitted query images including cyan women's tops within a specific time period (such as at least the past two weeks) to notify them of the new products offered by the online market platform.

[0018] Certain embodiments of the present disclosure overcome the shortcomings of the prior art by detecting demand for items provided on an online platform based on a query image submitted to the online platform. The process adopted automatically processes the query image to identify the objects described therein and annotate them. In some embodiments, the image tags are analyzed in the context of the knowledge graph to infer the semantics of the image tags representing the features of the objects depicted in the query image. Clustering of query image feature data is used to identify trends in user needs that have not yet been met by the online platform. Operators of online platforms do not need to screen data from different sources (including keyword history) to manually identify products that are in demand but not provided on the online platform, and determine existing or new product categories to be added to the online platform catalog. At the same time, when products are added to the online platform catalog, the demand detection server enables the online platform to notify users who have searched for certain products. In general, the process adopted by the demand detection server identifies user needs more accurately and quickly, and improves user engagement and conversion rates of online platforms.

[0019] As used herein, the term "query image" is used to refer to an image or picture featuring an object of interest to a user. For example, a query image may be a composite image featuring at least one object corresponding to a user's prompt input to an image-generating AI model. The query image is generated by or submitted to an online platform and used by the online platform as the basis for a database search to identify items that are the same or similar to the object shown in the query image.

[0020] As used herein, the term "image tag(s)" is used to refer to metadata or other data associated with an image that identifies an object category, object name, or object attribute associated with an object identified in the image. For example, a rubber duck is identified in an image, and the image tags for the image may include "rubber duck," "yellow," "duck," etc.

[0021] As used herein, the term "catalog data" is used to refer to information related to items offered by an online platform, such as an inventory of products offered by an online merchant. Catalog data may include images and descriptions of the items offered.

[0022] As used herein, the term "knowledge graph" or "knowledge database" refers to a specific type of graph-based data model that includes interlinked descriptions of real-world entities (e.g., objects, events, etc.). A knowledge graph can encode the semantics of the terms used in the knowledge graph to enable search, recommendation, and analysis. The knowledge graph can be built by a third party for a domain or market sector associated with an online platform. Alternatively, the knowledge graph can be built by the online platform itself. For example, a market platform may include the functionality of building a product knowledge graph for products provided or capable of being provided by the online platform based on product data extracted from various product manufacturers, industry organizations, and any suitable entity.

[0023] As used herein, the term "query image feature data" is used to refer to a semantic description of the image tags associated with the query image, such as whether the image tags refer to a specific product or indicate an attribute (such as size or color), whether the image tags indicate that the object in the query image is a component of a larger item, etc. For example, from the image tags "rubber duck" and "yellow", in addition to "rubber duck" and "yellow", the query image feature data "toy", "bathtub toy", "rubber ducky" can also be inferred.

[0024] As used herein, the term "query subject" refers to one or more object categories reflected in a collection of query images. Query subjects are identified by clustering query image feature data to identify commonalities, such as the same or similar object types or attributes. Query subjects that appear when query images associated with a particular object type or attribute are received from different users are above a certain threshold percentage of total queries. For example, if approximately 10% of all query images submitted to an online platform during a particular time frame are associated with cyan women's blouses, then "cyan women's blouses" is a query subject and the query cluster associated with that query subject is 10% of all query images directed to the online platform during such a time frame.

[0025] As used herein, the term "demand data" is used to refer to data representing a user's demand for items (such as products, services, or other objects) provided or potentially provided on an online platform. The demand data is generated by a demand analysis module of a demand detection server. The demand data includes data indicating whether the online platform meets or does not meet the user's demand. For example, the demand detection server determines the user's demand for products on the online platform based on a query image and a query result. If the query result shows a mismatch with the query image, the demand data indicates that the user's demand is not met, and a notification is displayed or provided to the platform operator. In some embodiments, the notification is in the form of a recommendation by certain suppliers of certain products that match the detected user demand. If the query result shows matching products provided on the online platform and a good user conversion rate, the demand data indicates that the user's demand is met, and it is recommended that the online platform retain inventory of the product.

[0026] As used herein, the terms "generative artificial intelligence," "generative AI," or "generative AI model" are used to refer to an AI system that can generate text, images, or other media in response to a prompt. Examples of image or art generative AI systems include Stable Diffusion, Midjourney, DALL-E, and similar machine learning models.

[0027] Now referring to the accompanying drawings, Figure 1 An example of a computing environment 100 is depicted in accordance with certain embodiments of the present disclosure, wherein a demand detection server 102 detects user demand for items provided or not provided by an online platform 136 based on a query image 125. In various embodiments, the computing environment 100 includes a demand detection server 102 connected to an online platform 136 and user computing devices 146A, 146B, and 146C (which may be individually referred to as user computing devices 146 or collectively referred to as user computing devices 146 herein) via a network 148. The computing environment may optionally include an image generation server 104 accessible by the user computing device 146 or the platform server 138. The user computing device 146 may be a desktop computer, a laptop computer, a tablet computer, a smart phone, or any suitable computing device. The network 148 may be a local-area network ("LAN"), a wide-area network ("WAN"), the Internet, or any other network topology known in the art to connect the user computing device 146 to the online platform 136.

[0028] The online platform 136 includes a platform server 138 and a database 140. Examples of the online platform 136 may include a market platform that provides certain products or services, or any other suitable online platform that provides items to users. The database 140 stores catalog data 144. The catalog data 144 may include images and descriptions of items provided on the online platform. The platform server 138 includes a query image search module 139, which receives the query image 125 and performs a search of the catalog data 144 to identify any products, services, or other items in the catalog of the online platform provider that match, substantially match, or are otherwise similar to the object depicted in the query image 125. In the context of the present invention, providing an item on the online platform 136 may mean offering for sale, rental, trade, gift, etc.

[0029] In some embodiments, the platform server 138 executes a query image generation module 150 that generates one or more query images 125 in response to a prompt input by a user. The prompt may be free-form text entered by the user, or may be a predefined description selected by the user (e.g., via a drop-down menu or other interactive graphical user interface item). The user computing device 146 may upload a locally generated or captured query image 125 to the online platform 136. Alternatively or additionally, the user computing device 146 may communicate with the image generation server 104 executing the query image generation module 150 to generate the query image 125 in response to the prompt submitted via the user computing device 146. The query image generation module 150 may be or may execute a pre-trained image generation AI model configured to generate images based on the prompt. The pre-trained image generation AI model may be retrained or fine-tuned based on the specific requirements of the online platform 136 and / or the demand detection server 102.

[0030] In some embodiments, the query image(s) 125 may be displayed to the user, for example, in parallel with the query search results. The user may provide feedback on the query image 125 or certain aspects of the query image 125. The feedback may be in the form of a comment on the query image 125, or an indication of a "favorite," "unmatch," "like," or "dislike" of the query image 125. In some embodiments, the feedback may be in the form of an edit to the query image 125, which results in a new prompt being submitted to the query image generation module 150 to generate a new query image 125. For example, the original query image 125 may show a short-sleeved blouse, and the user may indicate a desire to change the sleeve length. As a result, the query image generation module 150 receives a new prompt that guides the generation of a new query image 125 showing a long-sleeved blouse. In another example, the user may edit or change the prompt used to generate the original query image 125.

[0031] Data may also be collected based on a user's interaction with displayed search results (e.g., images of any related items provided by the online platform 136). For example, a user clicks on an image to add the associated item to a shopping cart and / or is redirected to a web page for purchasing the item. The user may also interact with the online platform 136 in other ways (such as by browsing a web page, navigating away from a web page, closing a web page, searching for new items, etc.). The collected user feedback and other data about the query image and data about the user's interaction with the online platform may be stored as metadata or other data structures associated with the query image 125. The query image 125 and any associated data (e.g., user feedback data, user interaction data, query results) are transmitted to the demand detection server 102 as query image data 126.

[0032] In some examples, database 140 stores a knowledge graph 142 associated with online platform 136. Knowledge graph 142 includes information about items provided on online platform 136. Knowledge graph 142 can be generated by online platform 136 or a third party (not shown). In some examples, knowledge graph 142 is stored on a third-party platform and can be accessed by platform server 138 and / or demand detection server 102 via an API, regardless of whether the knowledge graph is created by online platform 136. Knowledge graph 142 is used by query image search module 139 to perform a search of catalog data 144 based on query image 125.

[0033] The query image 125 is received (as part of the query image data 126) and processed by the demand detection server 102, which includes the image annotation module 106, the image content analyzer module 108, the subject identification module 120, the demand analysis module 122, and the data store 124. The image annotation module 106 is configured to identify (multiple) objects in the query image 125 and generate query image tags 128 based on the object recognition. The query image tags 128 may include one or more object categories, one or more object names, and one or more object attributes. The image annotation module 106 may implement a machine learning (ML) algorithm to identify objects and generate tags (or labels) that describe the objects. The query image tags 128 associated with the generated query image 125 may be stored in the data store 124.

[0034] The image content analyzer module 108 is configured to analyze the plurality of query image tags 128 based on the knowledge graph 142 associated with the online platform 136 to obtain the query image feature data 130. In some examples, the image content analyzer module 108 maps the query image tags 128 associated with the generated query image to the knowledge graph 142, and determines the semantic context of the query image tags 128 to obtain the inferred semantic descriptions of the plurality of image tags. The image content analyzer module 108 extracts the inferred semantic descriptions (e.g., inferred object categories, inferred object names, and inferred object attributes) from the knowledge graph 142 to create the query image feature data 130. In some embodiments, the image content analyzer module 108 accesses the database 140 of the online platform 136 to retrieve or query the knowledge graph 142. In other embodiments, the knowledge graph 142 is transmitted to the demand detection server for use by the image content analyzer module 108. The query image feature data 130 associated with different query images 125 generated for different user queries can be aggregated and stored in the data store 124.

[0035] The topic identification module 120 is configured to identify query topics 132 from query image feature data 130 aggregated from query images 125 generated for user queries during a period of time. The topic identification module 120 can implement a clustering model to cluster the query image feature data 130 corresponding to different query images 125 into one or more query clusters. The query topics 132 can be extracted from the query image feature data 130 corresponding to the query clusters. The query topics 132 can be associated with object categories, object names, or object attributes. The query topics 132 can be stored in the data store 124.

[0036] The demand analysis module 122 is configured to detect user needs by comparing one or more query topics 132 with the catalog data 144 of the online platform 136 to generate demand data 134. In some examples, the demand analysis module 122 uses a similarity model to determine the similarity between the identified query topic 132 and the catalog data 144 of the online platform by calculating a similarity score. If the similarity is above a threshold, it is determined that the online platform 136 meets the needs related to the identified query topic 132. If the similarity is below a threshold, it indicates that the needs of users from the online platform 136 are not met. In some embodiments, the demand data 134 may include a composite image of any item that represents a user of the online platform 136, whether it is a satisfied or unsatisfied need. In some embodiments, the demand data 134 may also include a recommendation on whether to take action to meet the unsatisfied need or to continue to meet the need. The demand data 134 may be stored in the data storage 124.

[0037] In some embodiments, the demand analysis module 122 transmits the demand data 134 to the online platform 136. In other embodiments, the demand data 134 is displayed for viewing by the operator of the online platform 136 via a graphical user interface of the demand detection server 102. When items are added to the online platform 136 (e.g., products are added to the online platform operator's catalog) to meet the needs identified in the demand data 134, the demand analysis module 122 can detect the change or be alerted to the change, and can send a notification message to the user about the newly added object matching the object previously searched by the user. In other embodiments, the online platform 136 can provide functionality for generating such notification messages. For example, the online platform 136 can provide the user with the option to submit an email address to which the user wishes to receive notification messages about new products.

[0038] Figure 2 An example of a process 200 for detecting user needs based on a query image 125 is depicted, in accordance with certain embodiments of the present disclosure. Figure 2 The description contemplates process 200 being performed by demand detection server 102 as described above. However, process 200 may alternatively be performed by online platform 136 (eg, by executing the described modules on platform server 138 and storing associated data in online platform database 140).

[0039] At block 202, the image annotation module 106 receives a query image 125. In some examples, the query image 125 is retrieved from query image data 126 stored in the data store 124. In some examples, the image annotation module receives one or more query images from the online platform 136, wherein the query image generation module 150 is configured to generate a query image based on user input, generally as Figure 1 As described in.

[0040] At block 204, the image annotation module 106 generates a plurality of query image tags 128 that identify and describe the content of the query image 125. In some examples, the image annotation module 106 implements an AI-based object recognition model to identify elements in one or more query images 125. An element may include an object (or entity) and associated attributes. An object may be a person, a place, or a thing. An attribute may be an object name, an object category, an object feature (such as color, shape, action, emotion), etc. In some examples, the image annotation module 106 extracts image tags based on data associated with the query image 125, such as data about a user request to make changes to the original query image (e.g., by providing a new prompt or other input to generate a new query image 125 that reflects changes to certain portions of an object shown in the original query image) or direct user feedback about the query image 125 (e.g., favorite, like, or dislike the query image).

[0041] At block 206, the image content analyzer module 108 analyzes the plurality of image tags based on the knowledge graph 142 to create the query image feature data 130. The online platform 136 may be a market platform that provides commercial products. The knowledge graph 142 may be an e-commerce knowledge graph that includes product information for various products that are currently provided or not provided on the online platform 136. The knowledge graph 142 may be generated by the online platform 136 or a third-party platform and may be accessed by the demand detection server 102 or the online platform 136 via an API. The image content analyzer module 108 maps the query image tags 128 (generated for one or more query images 125) by the image annotation module 106 to the knowledge graph 142 to determine the semantic context of the plurality of image tags based on the knowledge graph. For example, the image content analyzer module 108 may determine whether the image tag refers to a product, an attribute of a product (such as size or color), or a sub-part of a larger item. For another example, the image content analyzer module 108 may determine one or more category names for an object depicted in the query image. The query image feature data simplifies and enriches the image tags. For example, based on the fact that the image tags of the query image include “rubber duck” and “yellow”, the query image feature data including “rubber duck” can also be referred to as “rubber duckie”, “yellow” is the color of “rubber duckie”, “rubber duckie” belongs to the category of “toy” or “bathtub toy”, and “rubber duckie” can be inferred and created from the knowledge graph.

[0042] In some examples, the image annotation module 106 and the image content analyzer module 108 process query images 125 associated with multiple user queries from different users during a period of time. The query image feature data 130 of the different user queries during the period of time may be aggregated to generate aggregated query image feature data.

[0043] At block 208, the topic identification module 120 identifies query topics 132 based on the aggregated query image feature data 130. The topic identification module 120 may use a clustering model to cluster the aggregated query image feature data 130 based on commonality or similarity. The clustering model may be a connectivity-based model (e.g., a hierarchical clustering algorithm), a density-based model (e.g., density-based spatial clustering of applications, DBSCAN), a distribution-based model (e.g., a Gaussian mixture model), a centroid model (e.g., K-means clustering), or a neural-based model (e.g., an unsupervised neural network). The clustering model may also be a topic extraction model based on Latent Dirichlet allocation (LDA) or other generative probabilistic models. The clustering model may generate one or more clusters for the aggregated query image feature data 130. Each cluster corresponds to a query topic 132, which describes the commonality of the query image feature data 130 in the cluster. The query subject 132 may include information about object categories, object names, object attributes, or other object descriptors from the aggregated query image feature data 130 .

[0044] At block 210, the need analysis module 122 generates need data 134 by comparing the query subject 132 with the catalog data 144 of the online platform 136, the need data 134 indicating a user need for an object corresponding to the query subject 132. In some examples, the need analysis module 122 compares the query subject 132 with the catalog data 144 of the online platform 136 using a similarity model. The similarity model may determine a similarity score to measure the similarity between the query subject 132 and the catalog data 144. For example, the similarity model determines a cosine similarity between an embedding vector of the query subject 132 and an embedding vector of the catalog data 144. If the similarity score is below a predetermined threshold, the query subject 132 is determined to represent an unmet need from the online platform 136. The need analysis module 122 then generates need data 134 based on the query subject 132 having a similarity score below the predetermined threshold. The need data 134 may include a synthetic object image generated based on the query subject representing a detected need, regardless of whether the need has been met. The demand data 134 may also include recommendations of certain suppliers for products to be sold by the online platform 136. The demand analysis module 122 sends the demand data 134 to the online platform 136 or otherwise displays the demand data 134 for viewing by the operator of the online platform 136.

[0045] Figure 3300 is an example workflow for detecting user demand for items provided by an online platform 136 according to certain embodiments of the present disclosure. In this example, a user 302 of an online platform 136 enters a text input 304 (i.e., a prompt) via a GUI of the online platform 136. In this example, the online platform 136 is a market platform that provides commercial products. The text input 304 includes one or more keywords that describe the product of interest that the user 302 is searching for on the online platform 136. The query image generation module 150 on the platform server 138 generates one or more query images 125 based on the text input 304. The user 302 can provide feedback on the one or more query images 125. The query image generation module 150 regenerates the query image 125 based on the user feedback. The platform server 138 also compares the generated / regenerated query image 125 with the catalog data 144 (e.g., by implementing the query image search module 139) to determine whether any product provided on the online platform 136 matches or is sufficiently similar to the query image 125, thereby generating a query result 308. The query results may be displayed to the user in the form of images of certain products that match or are similar to the query image 125. The user may select a product to purchase, or provide feedback indicating that none of the suggested products are what the user 302 is looking for.

[0046] Meanwhile, the query image 125 (e.g., the originally generated and the regenerated associated metadata regarding the user feedback) may be processed by the image annotation module 106 on the demand detection server 102 to generate query image tags 128. The query image tags 128 include categories, names, attributes, or other descriptors regarding the objects depicted in the query image 125. In some examples, the image annotation module 106 compares different versions of the query image 125 taking into account the user action metadata to determine the query image tags 128 that are important to the user 302. For example, the user 302 chooses to keep "cap sleeves" in the shirt image, and therefore, one of the image tags may include "cap sleeves".

[0047] The image content analyzer module 108 can compare the query image tag 128 with the e-commerce knowledge graph 142 to enrich the query image tag based on the e-commerce knowledge graph 142. The e-commerce knowledge graph 142 has its own vocabulary for describing products that can be provided on the e-commerce platform. If the product category included in the query image tag 128 can be mapped to a term used in the e-commerce knowledge graph 142, the term is extracted as the inferred product category. The mapping is based on semantic meaning. A product can belong to more than one category, so there can be multiple inferred product categories. If the product name included in the query image tag 128 can be mapped to a term used in the e-commerce knowledge graph 142, the term is extracted as the inferred product name. Similarly, if the product attribute included in the query image tag 128 can be mapped to a term used in the e-commerce knowledge graph 142, the term is extracted as the inferred product attribute. In addition, the image content analyzer module 108 can also provide context for the query image tag 128. For example, the image content analyzer module 108 determines whether the query image tag 128 is referred to as a category, name, or attribute of an object in the query image 125 based on the e-commerce knowledge graph 142. The contextual and enriched query image tags 128 constitute the query image feature data 130 corresponding to the query image 125 associated with the text input 304.

[0048] The query image feature data 130 extracted from different query images 125 generated for different users 302 may be aggregated to be stored in a storage device as aggregated query image feature data 310. The aggregated query image feature data 310 may be processed to obtain query topics 132. For example, the topic identification module 120 of the demand detection server 102 implements a clustering model to cluster the aggregated query image feature data 310 into different clusters. The query topics 132 may be trends or patterns identified from the aggregated query image feature data 310.

[0049] The identified query subject 132 may be compared to the catalog data 144 of the online platform 136, for example, by the demand analysis module 122 on the demand detection server 102, to determine whether any product offered on the online platform 136 matches the identified query subject 132. If no product matches the identified query subject 132, this indicates that there is an unmet demand for the product corresponding to the identified query subject 132. Alternatively, if some products match the identified query subject 132, this indicates that the online platform 136 has already met the user's demand for the product corresponding to the identified query subject 132. In some embodiments, the demand data 134 (indicating an unmet demand or a met demand) may include images showing products that match or are similar to the product that the user searched for. For example, the query subject is pink clothing because 10% of user queries are about different pink clothes. One or more images of pink clothing (e.g., pink shirts, pink dresses) may be generated and included in the demand data 134, which is sent to or displayed to the operator 312 of the online platform 136.

[0050] Figure 4 An example of a GUI 400 of an online platform 136 for inputting a user query according to certain embodiments of the present disclosure is depicted. The GUI 400 enables a user to search for any product of interest on the online platform 136. In this example, the online platform 136 is a marketplace platform. The user can enter a query text 404 in a search box 402. The query text 404 describes what the user is searching for in the user's text. It may not be the exact term used by the online platform 136. It may not be grammatically correct. In this example, the query text 404 is "women's turtleneck short-sleeved green shirt for office use". The GUI 400 provides three possible search methods. The first method is a text-based search 406, in which the platform server 138 searches the query text 404 entered into the search box 402 in the catalog data 144 of the online platform 136. The second method is a generative AI-based search 408, in which the platform server 138 generates one or more query images 125 based on the query text 404 entered in the search box 402, and identifies images in the catalog data 144 of the online platform 136 that are similar to the generated query images 125. The third method is an uploaded image-based search 410, in which a user can upload a query image 125 depicting a product of interest, and the platform server 138 searches the catalog data 144 of the online platform 136 to identify products similar to the image uploaded by the user. Figure 5-Figure 7 In this example continued in , the user selects generative AI-based search 408 after entering query text 404 .

[0051] Figure 5An example of a GUI 500 for generating a query image 125 according to certain embodiments of the present disclosure is depicted. Following the example above, a prompt is provided to the query image generation module 150 on the platform server 138 based on the query text 404, such as "women's" 502, "tops" 504, and "turtleneck" 506. The user may also specify certain criteria for generating the query image 125, such as the criteria included in the criteria box 516. For example, the user sets the color palette and tint 508 to "neutral" and "fluorescent," sets the size 510 to "medium," and selects "model display" 512. Any criteria selected by the user are converted into prompts and submitted to the query image generation module 150 along with the prompt derived from the query text 404. The user may click or touch the button 514 to submit the prompt and generate the query image 125.

[0052] Figure 6 An example of a GUI 600 for providing feedback about a generated query image 125 is depicted in accordance with certain embodiments of the present disclosure. Following the example above, after a user clicks or touches button 514, the query image generation module 150 on the platform server 138 generates four query images 602, 604, 606, and 608. At the same time, the query image search module 139 searches the catalog data 144 of the online platform 136 and finds 28 products that are similar to the generated query images, as shown in area 610. The user can provide feedback about the generated query images 602, 604, 606, and 608, or interact with the similar products displayed in area 610. For example, the user can "favorite" certain query images, or mark certain query images as "mismatched" by what the user has in mind. In the example above, the query image generation module 150 generates four query images 602, 604, 606, and 608. Figure 6, the user marks query images 604 and 608 as “not matching”. The user can also provide feedback on specific parts in the query image. These parts can be generated as interactive based on the query text 404. The user can hover the cursor over the interactive part and choose to keep or regenerate the interactive part. Note that the name of the interactive part can also be displayed to inform the user. For example, the collar 612 of the shirt shown in the query image 602 is interactive and is shown as a “funnel collar”. The user can choose to keep the funnel collar by clicking the reuse button 613. Then, when the user clicks or presses the regenerate button 618, the regenerated image can retain the funnel collar feature and provide alternative representations for all other aspects of the generated image. Alternatively, the user can choose to deliberately exclude the funnel collar in subsequent image generation by clicking the delete button 614. Then, when the user clicks or presses the regenerate button 618, the regenerated image may not include the funnel collar feature, but may have a round collar or a roll collar or other type of high collar feature. Additionally, other aspects of the shirt, such as the sleeves 615 of the shirt shown in the query image 602, may be interactive, so that the user may choose to keep or exclude that aspect of the shirt. This functionality may also be used for other generated shirts. At the same time, the user may update certain criteria in the criteria box 516. For example, the user may deselect the "shown on model" option 616, which will cause the query images 602, 604, 606, 608 to be regenerated to show various shirts that are not worn on a model. The user may also click on a button 618 to regenerate all query images 602, 604, 606, 608, with or without changing any of the original prompts, such as the query text 404 or other specified criteria. Whenever one or more of the query images 602, 604, 606, 608 are regenerated, the area 610 is updated to show products that match or are sufficiently similar to the updated set of query images 602, 604, 606, 608.

[0053] Figure 7 An example of a GUI 700 is depicted that illustrates a situation where the online platform 306 does not provide a product that matches or is sufficiently similar to the generated or regenerated query image. Following the above example, when the user clicks or touches button 618 ( Figure 6 After regenerating the query image using the query image generation module 150 on the platform server 138, the query image generation module 150 regenerates the four images 702, 704, 706, and 708. At the same time, the query image search module 139 searches the catalog data 144 of the online platform 136 and does not find any similar products. Therefore, no similar products are displayed in area 710. However, area 710 displays a message to the user that the online platform 136 can let the user know when it sells similar products to the user's "favorites" and also sends a 10% coupon.

[0054] The user can provide feedback on the regenerated query images 702, 704, 706, and 708, similar to Figure 6 The user may click or touch button 712 to regenerate the query image based on the user's feedback on the generated query image. Figure 7 , the user likes the regenerated images 704 and 708. The query image generation module 150 of the platform server 138 can store the user's mark on the regenerated images 704 and 708 (e.g., a heart symbol indicating that the user likes the object depicted in the query image), and can send a notification to the user when a similar product is added to the online platform 136.

[0055] Any suitable computing system or group of computing systems may be used to perform the operations described herein. For example, Figure 8 An example of a computing system 800 for implementing certain embodiments of the present disclosure is depicted. Implementations of the computing system 800 may be used to implement the demand detection server 102. In other embodiments, Figure 8 A single computing system 800 that combines devices similar to those depicted in FIG. 8 (eg, processor, memory, etc.) Figure 1 One or more operations depicted as separate systems.

[0056] The depicted example of computing system 800 includes a processor 802 communicatively connected to one or more memory devices 804. Processor 802 executes computer executable program code stored in memory device 804, accesses information stored in memory device 804, or both. Examples of processor 802 include a microprocessor, an application-specific integrated circuit ("ASIC"), a field-programmable gate array ("FPGA"), or any other suitable processing device. Processor 802 may include any number of processing devices, including a single processing device.

[0057] Memory device 804 includes any suitable non-transitory computer-readable medium for storing program code 805, program data 807, or both. Computer-readable media may include any electronic storage device, optical storage device, magnetic storage device, or other storage device capable of providing computer-readable instructions or other program codes to the processor. Non-limiting examples of computer-readable media include disks, memory chips, ROMs, RAMs, ASICs, optical storage devices, tapes or other magnetic storage devices, or any other medium from which a processing device can read instructions. Instructions may include processor-specific instructions generated by a compiler or interpreter from a code written in any suitable computer programming language (including, for example, C, C++, C#, Visual Basic, Java, Python, Perl, JavaScript, and ActionScript).

[0058] The computing system 800 executes program code 805, which configures the processor 802 to perform one or more operations described herein. In various embodiments, examples of program code 805 include an application executed by the image annotation module 106 to generate image tags for a query image, an application executed by the image content analyzer module 108 to analyze image tags based on a knowledge graph associated with an online platform to obtain query image feature data, an application executed by the topic identification module 110 to identify query topics based on aggregated query image feature data, an application executed by the demand detection module 112 to detect object demand by comparing the query topic with the catalog data of the online platform to generate object demand data, or other suitable applications that perform one or more operations described herein. The program code may reside in the memory device 804 or any suitable computer-readable medium and may be executed by the processor 802 or any other suitable processor.

[0059] In some embodiments, one or more memory devices 804 store program data 807, which includes one or more data sets and models described herein. Examples of these data sets include generated query images, query image tags, query image features, query clusters, query topics, detected object requirements, etc. In some embodiments, one or more of the data sets, models, and functions are stored in the same memory device (e.g., one of the memory devices 804). In additional or alternative embodiments, one or more of the programs, data sets, models, and functions described herein are stored in different memory devices 804 accessible via a data network. One or more buses 806 are also included in the computing system 800. The bus 806 communicatively couples one or more components of a corresponding computing system 800.

[0060] In some embodiments, the computing system 800 also includes a network interface device 810. The network interface device 810 includes any device or group of devices suitable for establishing a wired or wireless data connection to one or more data networks. Non-limiting examples of the network interface device 810 include an Ethernet adapter, a modem, etc. The computing system 800 can communicate with one or more other computing devices (e.g., the user computing device 146) via a data network using the network interface device 810.

[0061] The computing system 800 may also include a plurality of external or internal devices, an input device 820, a display device 818, or other input or output devices. For example, the computing system 800 is shown as having one or more input / output ("I / O") interfaces 808. The I / O interface 808 may receive input from an input device or provide output to an output device. The input device 820 may include any device or device group suitable for receiving visual input, auditory input, or other suitable input for controlling or affecting the operation of the processor 802. Non-limiting examples of the input device 820 include a touch screen, a mouse, a keyboard, a microphone, a separate mobile computing device, and the like. The display device 818 may include any device or device group suitable for providing visual output, auditory output, or other suitable sensory output. Non-limiting examples of the display device 818 include a touch screen, a monitor, a speaker, a separate mobile computing device, and the like.

[0062] although Figure 8 Input device 820 and display device 818 are depicted as being local to the computing device executing demand detection server 102, but other implementations are possible. For example, in some embodiments, one or more of input device 820 and display device 818 may comprise a remote client computing device that communicates with computing system 800 via network interface device 810 using one or more data networks described herein.

[0063] Numerous specific details are set forth herein to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will appreciate that the claimed subject matter may be practiced without these specific details. In other cases, methods, devices, or systems known to those of ordinary skill in the art are not described in detail in order to avoid obscuring the claimed subject matter.

[0064] Unless otherwise specifically stated, it should be understood that terms such as "processing", "computing", "calculating", "determining" and "identifying" used throughout the discussion of this specification refer to the action or process of a computing device (such as one or more computers or similar one or more electronic computing devices) to manipulate or transform data represented as physical electronic or magnetic quantities within the memory, registers or other information storage devices, transmission devices or display devices of a computing platform.

[0065] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems that access stored software that programs or configures the computing system from a general-purpose computing device to a dedicated computing device that implements one or more embodiments of the present subject matter. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained in the software used in programming or configuring a computing device herein.

[0066] Embodiments of the methods disclosed herein may be performed in the operation of such a computing device. The order of the blocks presented in the above examples may vary, for example, the blocks may be reordered, combined and / or divided into sub-blocks. Certain blocks or processes may be performed in parallel.

[0067] The use of "adapted to" or "configured to" herein refers to open, inclusive language that does not exclude devices that are adapted to or configured to perform additional tasks or steps. In addition, the use of "based on" is meant to be open and inclusive, as a process, step, calculation, or other action that is "based on" one or more recorded conditions or values ​​may actually be based on additional conditions or values ​​beyond those recorded conditions or values. The headings, lists, and numbers included herein are for ease of explanation only and are not meant to be limiting.

[0068] Although the subject matter has been described in detail with respect to specific embodiments, it will be understood that those skilled in the art can easily make substitutions, changes and equivalents to these embodiments after understanding the foregoing. Accordingly, it should be understood that the present disclosure has been presented for the purpose of illustration rather than limitation, and does not exclude the inclusion of such modifications, changes and / or additions to the subject matter, which are obvious to those of ordinary skill in the art.

Claims

1. A method performed by one or more processing devices, comprising: Receiving a query image by an image annotation module; The image annotation module generates a plurality of image tags for the query image, wherein the image tags describe the content of the query image; Analyzing the plurality of image tags based on the knowledge graph by an image content analyzer module to create query image feature data; The query topic is identified by a topic identification module based on the aggregated query image feature data; as well as Demand data indicating user demand for an object corresponding to the query subject is generated by a demand analysis module, wherein the demand data is generated by comparing the query subject with catalog data of an online platform.

2. The method of claim 1, wherein the query image is a synthetic image generated by a generative artificial intelligence model.

3. The method according to claim 1, further comprising: A plurality of query images associated with a user query on the online platform is received, wherein the plurality of query images include image metadata including user action data associated with an aspect of one or more query images of the plurality of query images. The method according to claim 1 , wherein the plurality of image tags include object categories, object names, and object attributes. 5 . The method of claim 1 , wherein the online platform is a marketplace platform, and wherein the knowledge graph includes product information for products associated with the marketplace platform.

6. The method of claim 1, wherein the query image feature data comprises an inferred semantic description of the plurality of image tags, wherein analyzing the plurality of image tags based on the knowledge graph associated with the online platform by the image content analyzer module to create the query image feature data comprises: Mapping the multiple image tags to the knowledge graph; as well as The semantic contexts of the multiple image tags are determined based on the knowledge graph to obtain the inferred semantic descriptions of the multiple image tags.

7. The method according to claim 1, further comprising: aggregating the query image feature data generated from a plurality of query images corresponding to a plurality of user queries to generate aggregated query image feature data; as well as The aggregated query image feature data is stored.

8. The method according to claim 1, further comprising: The requirement data is transmitted to the online platform, wherein the requirement data includes a synthetic object image generated based on the requirement data.

9. The method according to claim 1, further comprising: A notification message is transmitted to a user computing device, the notification message indicating that the object has been newly added to the online platform.

10. A system comprising: an image annotation module configured to generate a plurality of image tags for the content of a query image; an image content analyzer module configured to analyze the plurality of image tags based on the knowledge graph to create query image feature data; a topic identification module configured to identify a query topic based on the aggregated query image feature data; as well as A demand detection module is configured to generate demand data by comparing the query subject with catalog data of an online platform, wherein the demand data indicates user demand for an object corresponding to the query subject.

11. The system according to claim 10, wherein the demand detection module is further configured to: determining a query result by comparing the query image associated with a user query with the catalog data of the online platform; and The query results are provided to a user computing device associated with the user query.

12. The system of claim 10, wherein the image annotation module is configured to receive a plurality of query images associated with a user query on the online platform, wherein the plurality of query images include image metadata including user action data associated with aspects of one or more of the plurality of query images.

13. The system of claim 10, wherein the query image feature data comprises an inferred semantic description of the plurality of image tags, wherein the image content analyzer module is further configured to: Mapping the plurality of image tags to the knowledge graph; and The semantic context of the semantic context of the multiple image tags is determined based on the knowledge graph to obtain the inferred semantic description of the multiple image tags.

14. The system of claim 10, wherein the topic identification module comprises a clustering model configured to cluster the aggregated query image feature data into one or more query clusters, wherein the one or more query clusters correspond to one or more query topics. 15 . The system of claim 10 , wherein the demand detection module is further configured to transmit the demand data to the online platform, wherein the demand data comprises a synthetic object image generated based on the demand data.

16. A non-transitory computer-readable medium storing executable instructions that, when executed by a processing device, cause the processing device to perform operations comprising: receiving one or more query images via the online platform; generating a plurality of image tags describing the content of the one or more query images; analyzing the plurality of image tags based on a knowledge graph associated with the online platform to create query image feature data; identifying a query subject based on the aggregated query image feature data; as well as A step for generating demand data indicative of user demand for an object corresponding to the query subject. 17 . The non-transitory computer-readable medium of claim 16 , wherein the one or more query images include image metadata including user action data associated with aspects of the one or more query images.

18. The non-transitory computer readable medium of claim 16, wherein the executable instructions, when executed by a processing device, cause the processing device to perform further operations, the further operations comprising: determining a query result by comparing the one or more query images with catalog data of the online platform; as well as The query results are provided to a user computing device associated with the user query.

19. The non-transitory computer-readable medium of claim 16, wherein the query image feature data comprises an inferred semantic description of the plurality of image tags, wherein the executable instructions, when executed by a processing device, cause the processing device to perform further operations, the further operations comprising: Mapping the multiple image tags to the knowledge graph; as well as The semantic context of the semantic context of the multiple image tags is determined based on the knowledge graph to obtain the inferred semantic description of the multiple image tags.

20. The non-transitory computer readable medium of claim 16, wherein the executable instructions, when executed by a processing device, cause the processing device to perform further operations, the further operations comprising: The requirement data is transmitted to the online platform, wherein the requirement data includes a synthetic object image generated based on the requirement data.

Images