Method and system for providing recommendations

Abstract

This invention provides a method and system for making recommendations based on user queries and social media data. The system integrates data from social media platforms to perform trend analysis and generate product recommendations. This invention enhances the user experience by leveraging real-time social media trends, user behavior, and interactions, thereby enabling more personalized and relevant product recommendations.

Description

Technical Field

[0001] The present invention generally relates to methods and systems for making recommendations, and more specifically, to methods and systems for making recommendations based on user queries with trend analysis. Background Technology

[0002] As online shopping platforms become increasingly complex and offer a wider variety of products, consumers often struggle to find items that match their preferences or interests. Traditional search engines require users to describe their desired products using specific terminology, which can be challenging when users cannot accurately articulate the characteristics or quality of the products they are looking for. Furthermore, mismatches may exist between the terminology or descriptions used for a particular product by consumers and sellers, or between different sellers of the same product. Because users may find it difficult to express their desired product descriptions with precise wording, effectively locating similar or related products becomes challenging, thus reducing the effectiveness of the online shopping experience.

[0003] Currently, social media platforms such as TikTok, YouTube, Twitter (now X), and Instagram have gained significant popularity among the general public and play a crucial role in influencing consumer purchasing behavior. However, while these platforms are known to provide information about consumer trends, they do not offer actionable insights into trending products that consumers can use to make purchasing decisions. Unless the users posting content on the platform specifically select and market the consumer products in the videos, the products appearing in social media content are often unidentifiable. Even when products are specifically marketed, social media platforms typically incentivize users to link to affiliated or exclusive online shopping platforms. Furthermore, it is common on social media platforms to provide different experiences to different users based on each user's personal network or characteristics, which can lead to an inaccurate or incomplete picture of which consumer products are trending across the entire group from any single individual's social media presence. Therefore, traditional systems miss the opportunity to leverage real-time social media insights to deliver relevant, trend-based recommendations on e-commerce platforms.

[0004] Therefore, it is necessary to develop an improved online shopping method that addresses these limitations by integrating real-time social media insights and trend analysis. Such a method could provide consumers with recommendations aligned with current market trends, making it easier and more efficient for them to discover products of interest. Furthermore, it could infer the most likely products based on incomplete or inaccurate descriptions of consumer goods by leveraging items known to be trending across the population.

[0005] Furthermore, other desirable features and characteristics will become apparent from the following detailed description and appended claims, taken in conjunction with the accompanying drawings and the background of this disclosure. Summary of the Invention

[0006] According to one aspect of the present invention, a method for providing recommendations is provided, the method comprising: receiving a query from a user at a first platform; performing trend analysis based on the query; generating one or more recommendations based on the results of the trend analysis; and providing the one or more recommendations to the user via the first platform.

[0007] In some embodiments, the query is received in the form of text input by the user.

[0008] In some embodiments, the method further includes: receiving the query from the user; analyzing the query; and classifying the intent of the query as general or specific based on the analysis; wherein if the query is determined to be a general query, the method further includes forwarding the query to a second platform to find trending items on the second platform.

[0009] In some embodiments, the step of performing trend analysis based on the query further includes: sending the query to a second platform; retrieving multiple videos from the second platform based on the query; identifying at least one trending product from the multiple videos; generating multiple products by the first platform by matching them with the at least one trending product identified in the second platform; and recommending the multiple products to the user via the first platform.

[0010] In some embodiments, the method further includes: taking screenshots at different time points in the plurality of videos; using an image search model to identify one or more products in the screenshots; determining the similarity between the identified one or more products; selecting the product with the highest similarity score as the at least one trending product from the second platform; matching the products from the first platform with the at least one trending product from the second platform; and determining a recommendation based on the matched products from the platform.

[0011] In some embodiments, the method further includes: tracking queries submitted by users within a predetermined time period; identifying multiple search queries based on the frequency of submissions during the predetermined time period; performing a search for the identified search queries and retrieving video results in a second platform; identifying trending products from the retrieved videos; and pre-storing the multiple search queries and the identified trending products.

[0012] In some embodiments, the method further includes: receiving the query from the user; and matching the query with the plurality of search queries; wherein if the query matches the plurality of search queries, then accessing pre-stored trending products of the corresponding plurality of queries; or, if the query does not match the plurality of queries, then processing the query in real time.

[0013] In some embodiments, the method further includes: identifying a plurality of trending products related to the query based on the trend analysis; and displaying the identified trending products in a balanced manner on the first platform.

[0014] In some embodiments, the query is received in the form of the user interacting with video content.

[0015] In some embodiments, the method further includes: receiving at least one video frame from the video content; identifying one or more products using an image search model by processing the at least one video frame; matching the one or more products to identify one or more products from the first platform; creating a link for each of the one or more products; and displaying the link to the user through a user interface within the video content.

[0016] In some embodiments, the step of matching the one or more products includes: using image search to match the one or more products with a product pool in the first platform.

[0017] In some embodiments, the step of identifying one or more products further includes: creating a search string based on the identified one or more products.

[0018] In some embodiments, the step of matching the one or more products includes: performing a search on the first platform using the search string.

[0019] In some embodiments, one or more products identified are different from the products currently promoted in the video content.

[0020] In some embodiments, the step of using an image search model to identify one or more items includes: receiving an image comprising one or more objects; using an object detection algorithm to identify one or more bounding boxes around each of the one or more objects in the image; extracting features from each of the one or more objects within the identified bounding boxes; comparing the extracted features from the image with a reference image to determine similarity; generating a similarity score that quantifies the degree of similarity between each of the one or more objects in the image and the reference image; and determining whether the similarity score of the one or more objects is higher than a threshold.

[0021] In some embodiments, the step of identifying one or more objects includes: embedding extracted features into a feature vector using an image embedding model, wherein the image embedding model converts each of the extracted features into a high-dimensional vector representation; and comparing the reference image and the feature vector of the image to identify the one or more objects.

[0022] In some embodiments, the method further includes the step of: fine-tuning the image embedding model using a Bidirectional Encoder Representations from Transformers (BERT) model for semantic understanding, wherein the BERT model is used to capture the contextual relationships between the extracted features.

[0023] According to another aspect of the invention, a system for providing recommendations is provided, the system comprising: a processor; a memory in electronic communication with the processor; and instructions stored in the memory and executable by the processor to cause the system to perform any of the methods discussed above.

[0024] According to another aspect of the invention, an apparatus for providing recommendations is provided, the apparatus comprising components for performing any of the methods discussed above.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed in a computer, it causes the computer to perform any of the methods discussed above.

[0026] The embodiments described herein are not exhaustive and additional features and variations of the invention may be incorporated. Various other advantages and novel features of the invention will become apparent from the following detailed description when considered in conjunction with the accompanying drawings. Attached Figure Description

[0027] This disclosure will be better understood when considered in conjunction with non-limiting examples and the accompanying drawings, in which: Figure 1 This is an architecture diagram of an online shopping system according to an embodiment of the present invention.

[0028] Figure 2 This is a flowchart of a method for an online shopping system according to an embodiment of the present invention.

[0029] Figure 3 This is an architecture diagram of a user terminal according to an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of an online shopping system using a social media platform according to an embodiment of the present invention.

[0031] Figure 5 This is a flowchart of an online shopping system using a social media platform according to another embodiment of the present invention.

[0032] Figure 6 This is a flowchart of an online shopping system that uses a social media platform to detect trending products, according to another embodiment of the present invention.

[0033] Figure 7 This is a flowchart of a method for identifying general queries using a social media platform for online shopping, according to an embodiment of the present invention.

[0034] Figure 8 This is a flowchart of a method for online shopping using a social media platform with pre-stored trending products, according to an embodiment of the present invention.

[0035] Figure 9 This is a flowchart of a method for online shopping by detecting multiple products according to an embodiment of the present invention.

[0036] Figure 10 This is an architecture diagram of an online live shopping system for product identification according to an embodiment of the present invention.

[0037] Figure 11 This is a flowchart of a method for displaying product information in an online shopping live streaming system according to an embodiment of the present invention.

[0038] Figure 12a This is a schematic diagram of a live streaming interface according to an embodiment of the present invention.

[0039] Figure 12b This is a schematic diagram of an online live streaming interface for identifying products in an image, according to an embodiment of the present invention.

[0040] Figure 12c This is a schematic diagram of an online live streaming interface for a product with an identifier, according to an embodiment of the present invention.

[0041] Figure 13 This is a flowchart of a method for product identification according to another embodiment of the present invention.

[0042] Figure 14a This is a flowchart of an image processing method for detecting goods according to an embodiment of the present invention.

[0043] Figure 14b This is a flowchart of an image processing method for detecting goods according to another embodiment of the present invention.

[0044] Figure 15 This is a flowchart of a method for displaying recommended products in an online shopping system according to an embodiment of the present invention. Detailed Implementation

[0045] In the following detailed description, reference is made to the accompanying drawings, which form a part thereof. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. Unless otherwise stated, the terms “comprising” and “including” and their grammatical variations as used herein are intended to mean “open-ended” or “inclusive” language, such that they include the elements set forth, but also allow for the inclusion of additional, unset elements.

[0046] Various embodiments are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. Embodiments described in the context of one system or method are equally effective for other systems or methods. Similarly, embodiments described in the context of a method are equally effective for a system, apparatus, or computer program, and vice versa.

[0047] Features described in the context of this embodiment may be correspondingly applied to other embodiments, even if not explicitly described in those other embodiments. Furthermore, additions and / or combinations and / or alternatives to features described in the context of this embodiment may be correspondingly applied to the same or similar features in other embodiments.

[0048] When purchasing products, users sometimes lack a clear or specific idea of ​​what they are looking for. In many cases, users begin shopping with broad, general, or unclear queries, such as searching for "Christmas clothes," without knowing the exact item or style they are interested in. Traditional search systems often struggle to handle such open-ended queries, typically producing a large number of broad and unfocused results. These results can overwhelm users, or be inconsistent with their personal preferences or current trends, making the search process unnecessarily time-consuming and potentially frustrating.

[0049] Another common scenario arises when users see products of interest in videos or other media where these media don't necessarily explicitly market or sell the product in question. This can include television programs, over-the-top videos, online streaming, short-format videos, or other contemporary visual or audiovisual media. Identifying and searching for specific products based on video content can be challenging, often requiring users to manually provide descriptions or estimates, which may result in incomplete or inaccurate results. Traditional methods like reverse image search are insufficient because it's difficult to capture images in a video stream that can be effectively compared to existing images of the product in question. The difficulty in translating visual inspiration into specific search queries in this situation adds inconvenience, thus reducing the smoothness of the shopping experience.

[0050] This lack of certainty or targeted queries often leads to time-consuming searches or missed opportunities to discover relevant products. This invention addresses this challenge by using data from a second platform to analyze and identify emerging trends relevant to a user's potential interests. By analyzing user queries within the context of popular trends and other relevant social media data, the system refines recommendations, presenting options that are not only trend-driven but also aligned with the user's implicit preferences. For example, if a user searches for "Christmas clothes" without specifying a particular style, the system can narrow down the options by highlighting trending seasonal styles, helping the user quickly discover trendy fashion products that match their personal style. Traditional product recommendation systems rely solely on sales information completed on a given platform, limiting their ability to provide responsive suggestions based on emerging trends. It should be noted that shopping systems encompass all forms of platforms, such as e-commerce platforms where users can purchase products. This includes, but is not limited to, platforms like Amazon, Taobao, Shopee, and similar online marketplaces. Shopping systems can also be integrated via simple links embedded in any video, image, or social media website, allowing users to access product listings directly from video or social media platforms such as YouTube or Instagram. For example, when a user watches a YouTube video, an embedded link to a product listing on a first platform can be provided. Users can interact with the embedded link within the YouTube video to access products on the primary platform.

[0051] Figure 1 This is an architecture diagram of an online shopping system according to an embodiment of the present invention. Figure 1 As shown, the online shopping system 100 includes, but is not limited to, user terminal 101, seller terminal 102, online shopping server 103, and social media server 104. Both user terminal 101 and seller terminal 102 can be referred to as user terminals. Both users and sellers can be referred to as users. These components within system 100 can be interconnected via network 105.

[0052] User terminal 101 and seller terminal 102 can be any type of computing device capable of transmitting messages via network 105. For example, user terminal 101 and seller terminal 102 can be smartphones, tablets, laptops, desktop computers, etc., but are not limited thereto. Applications including social media functionality can be installed and run on user terminal 101 or seller terminal 102. User terminal 101, seller terminal 102, online shopping server 103, and social media server 104 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this.

[0053] User terminal 101 can transmit search queries or inquiries to online shopping server 103. Online shopping server 103 can also transmit search queries to social media server 104. Social media server 104 can return popular resources to online shopping server 103. Social media platforms associated with social media server 104 can utilize various resources, such as videos, images, and publications. Those skilled in the art will recognize that different types of social media platforms may exist, such as platforms branded as TikTok and Instagram. User terminal 101 can receive recommended products returned by online shopping server 103. Therefore, user terminal 101 can display recommended products based on product popularity on social media platforms, allowing users to interact through user terminal 101 to conduct further online shopping processes.

[0054] It should be noted that although social media platforms are used as examples in the embodiments of the present invention, other platforms capable of identifying trending products may also be used without departing from the scope of the present invention. For example, platforms specifically designed for trending product analysis are also within the scope of the present invention.

[0055] It should be noted that search queries or queries can take various formats. In some embodiments, queries are received through user interaction with video content. For example, in the context of live streaming, a user can watch a live stream or click on certain products displayed in the live stream. In other embodiments, queries are received by the user in the form of text input or a combination of the above forms.

[0056] User terminal 101 and seller terminal 102 represent examples of the terminal types involved. However, this is only one possible scenario. Those skilled in the art will recognize that the number of terminals can vary widely; there may be only one terminal, or there may be dozens, hundreds, or even more terminals. Specific details regarding the number and type of devices used are not limited to this embodiment of the application.

[0057] Online shopping server 103 and social media server 104 can take various forms, including standalone physical servers, server clusters, or cloud-based solutions. As a cloud server, it can provide a range of cloud computing services, such as cloud storage, databases, computing power, network services, middleware, domain name services, security features, content delivery networks (CDNs), big data analytics, and artificial intelligence capabilities.

[0058] In some embodiments of the present invention, the number of servers involved may vary; depending on the application's needs, there may be a single server or multiple servers. Furthermore, the servers may include other dedicated servers to provide a wider range of services and enhance the overall functionality of the system.

[0059] Network 105 may include a combination of local area network (LAN), wide area network (WAN), terrestrial network, wireless network, data bus, telephone network, point-to-point network, satellite network, token ring network and / or hub network.

[0060] Figure 2 This is a flowchart of a method for an online shopping system according to an embodiment of the present invention.

[0061] The various methods disclosed herein are not necessarily limiting. Steps may be added, omitted, and / or performed simultaneously without departing from the scope of the appended claims. These methods may include any number of additional or alternative tasks, and the method may be incorporated into a more comprehensive program or process with additional functionality not described in detail herein. Furthermore, one or more tasks may be omitted from embodiments of the method, provided that the intended overall functionality remains unchanged. The illustrated method may also be stopped at any time. The method is computer-implemented because the various tasks or steps performed in conjunction with the method can be performed by software, hardware, firmware, or any combination thereof.

[0062] In some embodiments, users are required to authenticate with the system before they can use the online shopping system. User authentication via login ensures a secure and personalized experience. Before accessing system features, users need to log in using their credentials (e.g., username and password). This process verifies their identity and protects sensitive information, including personal details and payment methods. Authentication also allows the system to provide a tailored shopping experience by remembering user preferences and past purchases.

[0063] refer to Figure 2 Taking the method executed by the user terminal as an example, the method includes the following steps: At step 201, the method begins when a user submits a query to the first platform. The query may include specific keywords related to a product, product category, or other identifying characteristics. The query may also be broad or general, such as "clothing." The first platform may take various forms, such as an e-commerce platform or any other platform capable of recommending products to users.

[0064] At step 202, once a query is received, the method processes the query by performing trend analysis. Trend analysis can be performed by sending the query to a second platform (e.g., a social media platform, an e-commerce platform, or any platform that provides trend information) to analyze relevant trends. Trend analysis including social media platform data may consider, but is not limited to, the amount of references provided to or published on the social media platform or the amount of content including a given product, text descriptions attached to the social media content by the creator of the content or other users of the social media platform, the level of engagement of users on the social media platform with a particular content item, or the presence or absence of a particular content item among users based on its popularity or activity level on the social media platform. In some embodiments, trend analysis considers not only social media trends but may also include data such as historical data about users, product sales data, and product reviews. In other embodiments, trend analysis may assign different weights based on metadata applicable to the content on the platform; for example, this may include giving additional weights to users who are less active on the platform and reduced weights to users who participate in traditional marketing campaigns (e.g., paid influencers or product reviewers). In still other embodiments, trend analysis may assign weights in different ways.

[0065] At step 203, the method generates recommendations. By merging identified trend factors and / or products, the method determines which products available on the first platform are most likely to correspond to trending products, and thus responds to user queries. Alternatively, the method determines which multimedia content available on the first platform is most likely to correspond to the query, and thus provides multimedia content that helps the user identify products in response to the query. Multimedia content may include images, videos, or live streams, which further enhances the recommendations by providing dynamic content.

[0066] At step 204, recommended products are displayed to the user via a first platform (e.g., an e-commerce platform).

[0067] In some embodiments, this method more accurately identifies trending products by monitoring and aggregating data from various social media platforms. These trends are then analyzed in real time and correlated with users' general preferences or past behavior, enabling the system to provide highly relevant product recommendations even when a user's query is general or broad. This approach ensures that users are presented with products most likely to align with emerging or existing trends within their relevant groups, thereby enhancing the overall shopping experience.

[0068] In some embodiments, the present invention also relates to a system and method for automatically recommending products to users based on image / video content from social media platforms, e-commerce platforms, or any platform that provides trend information. The system analyzes text, image, or video data from social media platforms, e-commerce platforms, or any platform that provides trend information to identify relevant content and make personalized product recommendations without requiring users to enter search queries.

[0069] This system utilizes video recognition and analysis technology to detect and understand trending products or context presented in videos. These detected products are then compared to products in a database, and recommendations are generated accordingly. Based on this analysis, the system generates product recommendations relevant to the video content, thereby enhancing the user experience and providing more accurate and timely product suggestions. The user interface is designed to present these recommendations in a user-friendly manner, enabling users to easily explore product suggestions and take action.

[0070] Figure 3 This is an architecture diagram of a user terminal according to an embodiment of the present invention.

[0071] According to an exemplary embodiment, the user terminal 300 may include a user interface 301, a search module 302, a social media interface module 303, an image search module 304, a product detection module 304a, a product list module 305, one or more memories 306, and one or more processors 307.

[0072] User interface 301 is the main interaction point between the user and the terminal. User interface 301 includes both an input interface 301a and an output interface 301b to provide a user-friendly experience. User interface 301 can receive user input, such as a search query from the user, can present a list of search results to the user, can obtain product selections from the user, and can present a product list to the user.

[0073] For input interface 301a, user interface 301 allows users to input search queries, enter information, make selections, and interact with the system. Common input methods include touchscreen, keyboard, voice commands, and mouse input. The input interface supports search queries, navigation, and user commands.

[0074] For output interface 301b, user interface 301 displays information and feedback to the user. User interface 301 includes visual elements such as a screen or monitor where the user can view product details, recommendations, and live content. The output interface also includes audio components for notifications and alarms.

[0075] Search module 302 enables users to quickly and efficiently find products. Search module 302 processes user input to perform searches based on keywords, categories, or filters. The module is equipped with algorithms that match user queries with a list of relevant products and present the results in an organized manner.

[0076] The social media interface module 303 is used for integration with various social media platforms. For example, the social media interface module 303 can forward and analyze a user's search query to the social media platform and determine trending products recommended to the user corresponding to the provided search query. In some embodiments, it also allows users to share their shopping experiences, post reviews, interact with others, and recommend products based on the social media platform. This information can be selectively applied in a variety of ways, including but not limited to determining whether a large amount of activity is based on positive and favorable reviews of the product, or on negative or defamatory reviews. This module supports integration with social media feeds, share buttons, and login functionality for platforms such as TikTok, Facebook, Twitter, and Instagram.

[0077] Image search module 304 processes and manipulates image data. Image search module 304 supports tasks such as image enhancement, object recognition, and visual search. This module enables functions such as identifying products from images to recommend products based on detected items.

[0078] In some embodiments, the image search module 304 may include a product detection module 304a. The product detection module 304a is responsible for identifying and labeling products within image, video, or live stream data. The image search module 304 uses advanced algorithms and machine learning techniques to detect and categorize products based on their visual characteristics. This module is crucial for image search and real-time product detection of images, videos, or live streams.

[0079] In some embodiments, the user terminal includes a product list module 305. The product list module 305 searches for products in the online shopping system based on detected product characteristics and manages the presentation of product information to the user. The product list module 305 organizes and displays the product list.

[0080] In some embodiments, the user terminal includes a memory 306. The memory 306 may include one or more computer-readable storage media, which may be non-transitory. The memory 306 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 306 is configured to store at least one piece of program code. This at least one piece of program code is configured to be executed by a processor 307 to implement the online live streaming processing, video processing, and image processing provided in the method embodiments of the present invention. Sufficient memory ensures that the terminal can handle multiple tasks simultaneously, such as running a search module, processing images, and managing user interactions without latency.

[0081] In some embodiments, the user terminal includes a processor 307. The processor 307 can be implemented using various technologies and architectures designed to implement the described functions. It can be implemented as a general-purpose processor, content-addressable memory, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device, discrete gate or transistor logic, discrete hardware components, or combinations thereof. Furthermore, the processor can take the form of a microprocessor, a controller, a microcontroller, or a state machine. Additionally, the processor can be implemented through a combination of computing devices, such as combining a digital signal processor with a microprocessor, utilizing multiple microprocessors, integrating one or more microprocessors with a digital signal processor core, or employing any other suitable configuration. In some embodiments, the processor 307 may also include an artificial intelligence (AI) processor. The AI ​​processor is configured to handle computational operations related to machine learning for image processing to identify goods. A powerful processor enables the terminal to efficiently handle complex operations, such as real-time image processing, running search algorithms, and managing user input.

[0082] Those skilled in the art will recognize that, Figure 3 The structure depicted is not limited to user terminal 300. User terminal 300 may include more or fewer components than those illustrated in the figure, some components may be combined, or alternative component configurations may be used. The user terminal 300 described herein is not limited to local processing capabilities and can be integrated with cloud-based processing systems. This allows compute-intensive tasks to be delegated to the cloud, enabling efficient processing and real-time updates while ensuring that user terminal 300 remains lightweight and responsive.

[0083] Figure 4 This is a schematic diagram of an online shopping system using a social media platform according to an embodiment of the present invention.

[0084] As shown in 410, after a search is performed on an online shopping platform, an interface displaying a list of relevant products is presented to the user. In some embodiments, search results are organized in a grid or list format, displaying product images, names, prices, and brief descriptions. Each product entry includes basic details such as ratings, availability, and any current discounts.

[0085] In some embodiments, the interface has filtering and sorting options, allowing users to refine their searches based on criteria such as price range, brand, or user ratings. Additionally, users can hover over or click on individual products to view more detailed information, including specifications, user reviews, and additional images.

[0086] To enhance the shopping experience, interfaces are often integrated with social media platforms to identify trending products on those platforms, thereby improving the relevance of products recommended to users in 420.

[0087] In some embodiments, when a list of videos is identified from a social media platform, the system switches to image search model 430 to identify products. The primary objective is to detect trending products from social media platforms. In addition to video content, various other forms of social media data, such as images, live streams, and other multimedia formats, can be analyzed to find trending products.

[0088] In some embodiments, data from multiple social media platforms is collected and aggregated to identify trending products. By analyzing different sources, the system can detect emerging trends across various platforms, thereby enhancing the accuracy and relevance of recommendations made to users.

[0089] After identifying the products, the system sends the detected main products to the product list module 440. The product list module then searches the list for similar products based on the detected products to make recommendations.

[0090] After identifying the product, the product is displayed to the user in interface 410.

[0091] In some embodiments, a user may become interested in clothing or shoes of a certain style after watching videos on a social media platform. However, the user may not know how to accurately or completely describe the style. Using this invention, a user can search for general queries, such as "clothing," on an online shopping platform. The system will then look for trending clothing designs on the social media platform and recommend products to the user. Such a system can be configured to make this determination based on available information about the user submitting the query, either on overall trending items on the social media platform or on trending items within a subgroup or other user set on the social media platform.

[0092] Overall, online shopping systems enhance the shopping experience, making it easier for users to discover products that match trending items or find more search results that interest them, ultimately driving higher engagement and sales.

[0093] Figure 5 This is a flowchart of an online shopping system using a social media platform according to another embodiment of the present invention. Taking a method applied to the interaction between a terminal and a server as an example, such as... Figure 5 As shown, the method includes: At step 501, the user enters a search query in the user terminal. The user terminal receives the user query and processes it to identify the user's intent.

[0094] In some embodiments, the user interface includes social media platform controls, and the online shopping platform will recommend products to users based on the product's popularity on the social media platform only when a trigger action regarding social media recommendations is received.

[0095] In step 502, the user terminal sends a query to the social media server to perform a search on the social media platform based on the user's query. The search retrieves content results related to the query, such as videos, images, or other media.

[0096] At step 503, the social media server processes the search results and identifies specific content based on factors such as relevance or popularity, such as the top 50 or 100 videos.

[0097] At step 504, the social media server sends the video to the online shopping server.

[0098] At step 505, the online shopping server uses an image search model to perform product recognition on the video to obtain at least one trending product.

[0099] In step 506, after receiving the detected product data, the online shopping server processes the information to search for matching products in the online shopping product database. The server uses algorithms to find products that closely match the characteristics of the detected product. For example, if a specific type of clothing is identified, the server searches for other clothing from a list of online shopping platforms with similar characteristics.

[0100] In some embodiments, the server stores user preferences and purchase history so that the system can recommend more relevant products to the user based on the user preferences or purchase history.

[0101] The online shopping server further ranks the products in the recommended product list, with higher-ranked items displayed at the top. This ranking prioritizes products based on relevance, popularity, and user preferences, ensuring that the most suitable and popular options are recommended to the user. Therefore, it increases the likelihood of a purchase and improves the user experience.

[0102] In some embodiments, before transmitting product links, the server sorts the products based on various parameters, such as price index, sales volume index, and other relevant factors. This sorting ensures that the most suitable products are prioritized and presented to the user.

[0103] In step 507, the online shopping server transmits the list of recommended products to the user's terminal.

[0104] In step 508, after the server transmits the recommended product list back to the user's terminal, the terminal displays the product list on the interface. The user can interact with these recommendations, view detailed product information, and access links to purchase the recommended products.

[0105] At step 509, the user can interact with the recommended product by clicking a purchase link. This action takes the user to the product's details page, where they can add the item to their cart and continue the purchase process.

[0106] Figure 6 This is a flowchart of an online shopping system that uses a social media platform to detect trending products, according to another embodiment of the present invention.

[0107] The system retrieves videos from social media platforms based on user queries. In step 601, the system captures screenshots from the videos at different time points for further analysis or processing.

[0108] At step 602, the system detects goods from the video. Specifically, the system uses an image search model to identify the main goods in the image, which uses a machine learning model to detect objects in the video frames. Techniques such as convolutional neural networks (CNNs) can be used to identify goods. For each detected goods, the system also extracts features. These features may include color, shape, texture, and other visual characteristics.

[0109] At step 603, the system checks the similarity between different items. The system uses a similarity metric (e.g., cosine similarity, Euclidean distance, or other relevant metrics depending on the feature type) to compare the features of the items identified in different videos. The system may use clustering algorithms such as k-means or hierarchical clustering to group similar items together to identify items that are visually or contextually similar in the videos.

[0110] At step 604, the system selects the items with the highest similarity scores, and these items are trending items from the social media platform. In some embodiments, the system also analyzes the frequency of occurrence of the selected highly similar items in the videos. Items that appear frequently and have high similarity scores are likely to become trending. Therefore, the system can combine frequency and popularity metrics to determine overall trending items from different videos. These are the items that appear most frequently and are most similar in the videos, indicating potential trends. In some embodiments, the system also considers additional metrics, such as the context in which the items appear (e.g., in popular scenes or associated with popular topics), to identify trends. Therefore, this method can systematically detect, compare, and identify trending items in multiple videos based on visual and contextual similarity.

[0111] At step 605, the system identifies a list of products from the online shopping system. The system attempts to match these product images uploaded by the seller based on an image search model. In some embodiments, the system also compares the detected products with a product database to retrieve detailed information, including product name, description, and purchase options.

[0112] At step 606, the system displays matching products to the user. This method collects matching products from the product list as search results. The matching products are ranked based on relevance and returned to the user.

[0113] Figure 7 This is a flowchart of a method for identifying general queries using a social media platform for online shopping, according to an embodiment of the present invention.

[0114] When online shopping system search queries are general / broad, it's difficult to find the style or product that the user (e.g., the buyer) truly wants. Click-through rates for general query results are significantly lower than for specific queries.

[0115] Traditional search engines often struggle to accurately determine whether a user's search query reflects general interests or a specific need. Understanding user intent is crucial for delivering relevant search results. This invention provides a method for analyzing and categorizing search queries to determine whether user intent is general or specific. This categorization helps deliver more relevant search results and improves click-through rates.

[0116] Figure 7This invention provides a method for addressing the aforementioned problems. In some embodiments, the invention offers a method for classifying user search queries into general or specific intents using advanced natural language processing (NLP) algorithms, such as large language models (LLM). The system processes the search query using LLM to extract contextual and semantic meaning, which is then used to classify the intent. This classification improves the accuracy of search results and enhances the overall user experience. Only when the search query is classified as general will the method subsequently rely on social media platforms to improve search results.

[0117] In some embodiments, a predefined list of general queries is maintained. If a user's query matches an entry in this list, it is classified as a general query.

[0118] At step 701, the method begins by receiving a search query from the user, which serves as input for subsequent processing steps.

[0119] At step 702, in some embodiments, the search query undergoes several preprocessing steps to enhance its accuracy and relevance. This includes tokenization, which breaks the query down into individual words or tokens; normalization, which standardizes the text by converting it to a consistent format; and vectorization, which transforms the text into numerical vectors to represent its semantic meaning in a form suitable for analysis.

[0120] At step 703, the preprocessed query is then analyzed using an LLM (High-Level Model). This model interprets the query by understanding its context and semantics. The LLM captures deeper meaning and relationships within the query, thus helping to more accurately interpret what the user is searching for.

[0121] At step 704, based on the analysis from the LLM, the method determines whether the user's intent is general or specific. This step involves evaluating the query to discern whether the user is looking for broad information or has a specific, targeted request. For example, if the search query is "clothing," then the search query is categorized as a general search query. In contrast, a specific search query might be "yellow clothing with floral patterns for a Christmas party."

[0122] At step 705, if it is determined that the search query is a general query, the search query is forwarded to a social media platform to find trending products on the social media platform. Otherwise, at step 706, if it is determined that the search query is a specific query, a real-time search is performed based on the specific search query without referring to the social media platform.

[0123] Figure 8 This is a flowchart of a method for online shopping using a social media platform with pre-stored trending products, according to an embodiment of the present invention.

[0124] In some embodiments, the present invention provides a video search system that includes a pre-search step for identifying and caching the most frequently searched queries within a specified time period (e.g., a week). By pre-storing the results of these popular queries, the system enables users to access relevant videos quickly and efficiently without waiting for real-time search processing. Therefore, it enhances the user experience by reducing search time.

[0125] At step 801, the method tracks search queries submitted by a user over a period of time (e.g., a week). This involves recording each query and its search frequency.

[0126] At step 802, at the end of the tracking period, the method analyzes the recorded data to identify popular queries based on frequency. Popular queries are then selected for further processing.

[0127] At step 803, for each identified popular query, the method performs a search and retrieves relevant content from social media platforms. Videos are processed to identify trending products, including referencing metadata where appropriate, such as descriptions, reviews, tags, or other data used by users to annotate or add information about the content. In some embodiments, the method also identifies product listings from an online shopping system based on trending products.

[0128] Video results, trending products, and / or recommended products for popular queries are stored in a cache or pre-stored database. This pre-stored data is organized and optimized for performance to facilitate fast retrieval.

[0129] At step 804, when the user enters a new search query, the system first checks whether the query matches or is potentially related to any of the pre-stored popular queries.

[0130] At step 805, if the query matches or is potentially relevant to pre-stored popular queries, cached results of trending products are retrieved and a product list is recommended based on the trending products.

[0131] At step 806, if the query does not match any of the pre-stored popular queries, a live search is performed on the social media platform and the query is processed as usual.

[0132] Therefore, this invention improves search efficiency by pre-searching and pre-storing the results of the most frequently searched queries within a given time period. The system minimizes user wait times by providing direct access to pre-stored results of popular queries and processing other searches in real time.

[0133] Figure 9 This is a flowchart of a method for online shopping by detecting multiple products according to an embodiment of the present invention.

[0134] In some embodiments, the present invention relates to a system and method for identifying multiple trending products, recommending products based on these trends, and displaying these products in a balanced and user-friendly manner.

[0135] In step 901, a video is received from a social media platform for a search query.

[0136] At step 902, multiple trending items for a general search query are detected from the video. For example, when a user enters a query related to clothing, the system detects three main clothing styles that are currently trending on social media platforms.

[0137] At step 903, different recommended product sets are identified based on the detected trending items. For example, after detecting three main clothing styles, the system will identify three product sets in the online shopping system to match the three clothing styles identified on the social media platform.

[0138] At step 904, after generating different product sets based on multiple trending items, a combined product list is displayed to the user. The interface displays the recommended product list in a balanced and proportional manner, ensuring that different categories of products are represented equally and avoiding over-representation of any single category in the user interface.

[0139] With the rise of e-commerce and social media platforms, live shopping has become a popular method for attracting consumers and driving sales. However, existing systems lack efficient methods for real-time product identification and recommendations during live streams. Users often manually search for products shown in the live stream, which can lead to a suboptimal shopping experience.

[0140] In some embodiments, recommended products are featured in live video feeds used in online shopping systems. Therefore, the system provides a novel system and method for enhancing the live shopping experience by incorporating real-time product identification, recommendations, and purchase functionality.

[0141] Figure 10 This is an architecture diagram of an online live-streaming shopping system for product identification according to an embodiment of the present invention. Figure 10 As shown, the online live shopping system includes, but is not limited to, user terminal 1001, seller terminal 1002, and server 1003. These components within system 1000 can be interconnected via network 1004. The system may also include a social media server.

[0142] The seller logs into their account on their terminal, opens a live stream room for viewers, and begins capturing live video through their camera. The seller terminal 1002 transmits the live stream data to the server 1003. The user terminal 1001 receives the live stream data returned by the server 1003 and then decodes the received data to obtain audio and video frames. Therefore, the user terminal 1001 can display the live stream, allowing users to watch the live stream on their user terminal 1001 for further online shopping.

[0143] The video stream is then transmitted to a server. The server plays a crucial role in encoding the audio and video frames from the live stream and distributing the encoded live data to user terminals.

[0144] User terminal 1001 and seller terminal 1002 represent examples of the terminal types involved. However, it is important to note that this is only one possible scenario. Those skilled in the art will recognize that the number of terminals can vary widely; there may be only one terminal, or there may be dozens, hundreds, or even more terminals. Specific details regarding the number and type of devices used are not limited to this embodiment of the application.

[0145] It should be noted that although live streaming is used as an example in the embodiments of the present invention, other types of image or video platforms capable of identifying products and recommending products to users may also be used without departing from the scope of the present invention. For example, this includes any image or video platform that can recommend products to users.

[0146] Figure 11 This is a flowchart illustrating a method for displaying product information in an online shopping live-streaming system according to an embodiment of the present invention. The online shopping platform incorporates live-streaming functionality, allowing users to watch live streams directly on the platform.

[0147] Current live streams and videos only allow users to purchase specific items sold by content creators during the live stream or video. However, users may also be interested in purchasing other items shown in the video / live stream. This invention aims to enable real-time detection of products displayed during live streams on online shopping platforms.

[0148] refer to Figure 11 Taking the terminal execution method as an example, the method includes the following steps: At step 1101, live stream data is displayed to the user. The live stream data is selected by the user or recommended to the user based on a user search query. The method begins by displaying a live stream with a live streamer (host) on a live online shopping platform. The live stream includes at least one target product being promoted by the host. The live stream captures not only the target product but also other products presented in the live stream, such as accessories or related products. For example, a seller displays and sells her clothing in her live stream, which also showcases other accessories such as earrings. The live stream consists of multiple frames from the live stream, which can be used for image processing to identify the products displayed in the images.

[0149] In some embodiments, the live stream is displayed in a live stream interface that overlays elements such as live stream viewer room information, a comment section, and product link controls. These controls include links to products recommended by the streamer. Viewers can interact with these links to access details and make purchases.

[0150] At step 1102, the method detects products from the live stream. During the live stream, the host recommends specific products, and these recommendations are highlighted in the live stream. Viewers can select these product links to be redirected to the product purchase interface. In addition to recommended products, viewers can also use product recognition operations to identify and interact with other products displayed in the live stream image, thereby providing a broader shopping experience.

[0151] In some embodiments, a live viewer room is a feature within a shopping platform specifically designed for live streaming. Within this viewer room, the live streamer may include product links that guide viewers to purchase the displayed products. When viewers click on one of these product links, they are taken to a product purchase page where they can complete the transaction.

[0152] In addition to the products directly linked by the host, viewers can also choose to use the product identification feature to identify and interact with other products shown in the live stream. This feature allows users to explore and interact with additional products displayed during the live stream, extending their shopping experience beyond the host's recommendations.

[0153] In some embodiments, when a live streamer recommends clothing, the live stream image includes the clothing recommended by the live streamer, and the live stream also includes other merchandise (e.g., accessories worn by models and room decorations).

[0154] At step 1103, after identifying a product (e.g., detecting earrings), the method then initiates a search based on the characteristics and attributes of the identified product. The method attempts to match the detected product shown in the live stream with images uploaded by sellers in the online product list of the online shopping platform. If any product is found that correlates with a high similarity score, it is identified as the target product.

[0155] In some embodiments, it extracts key attributes from the identified products, such as category, brand, function, price range, and user reviews. This information forms the basis for finding similar products. Using the extracted attributes, the system performs a comprehensive search in its product database to generate a list of products that match or complement the selected products. This includes products in the same category, products with similar functions, or related accessories that the user might be interested in.

[0156] To address the different approaches to user identification and product search, this invention combines two main product identification methods: text-based search and image-based search.

[0157] The first method is text-based search, which relies on keywords or search strings. In this case, the system interprets detected products and generates search strings based on them. For example, the system can generate a search string for "black silk cheongsam" based on detected products and use it to retrieve related products. The strength of this method lies in its ability to identify specific attributes of detected products and generate search strings based on them. Furthermore, this method can incorporate trending terms and phrases from social media platforms, enriching search results with the latest and most in-demand products.

[0158] The second approach is image-based search, which focuses on product identification through image search. Here, the system uses image search methods to match the design of detected goods seen in images or videos with similar products. For example, after detecting a product from a video, the system can also retrieve comparable products from the platform via image search. By analyzing design attributes (e.g., color, pattern, silhouette, or fabric type) in the original image, the system compares these attributes with product images available on the platform. This visual search approach is particularly useful for users who may lack the specific language to describe what they are looking for, or those inspired by visual effects rather than product names. It enhances accessibility by enabling users to more intuitively find goods that match their visual preferences. Products identified through text-based or image-based search can be identified as trending products for recommendation to users.

[0159] In step 1104, a purchase link is displayed to the user, allowing them to click the link to make a purchase. Recommended products are displayed to the user in a dedicated area on the shopping platform or live streaming interface. Users can view these recommendations next to the original products and choose to explore more details or make additional purchases. When a user selects a product link, they are directed to the product purchase interface. Here, they can complete the purchase and finalize the transaction.

[0160] Figure 12aThis is a schematic diagram of a live streaming interface according to an embodiment of the present invention.

[0161] like Figure 12a As shown, the live streaming interface 1200 features a live streamer 1210 actively broadcasting. The streamer is selling clothing in the live stream room, which also includes other items such as the streamer's earrings or a clock on the wall. One or more links 1201 and 1202 are displayed for the clothing the streamer is selling. Users can then click these links to continue purchasing.

[0162] In some embodiments, the live streaming interface 1200 is designed to provide additional interactive elements common to live streaming interfaces, including live stream viewer room information 1203, comment information 1204, and product link controls 1205. The live stream viewer room information 1203 displays detailed information such as the name of the live stream viewer room and the title of the live stream. The comment information 1204 shows public comments made by users within the live stream viewer room. The product link controls 1205 manage product links for items recommended by the live stream host. By interacting with the product link controls 1205, users can decide whether to trigger the identification of new products for recommendation. If triggered, the system will provide new product links for consideration. Otherwise, the identification process will not be initiated, and no new recommendations will be generated. This setup ensures that viewers not only enjoy the live stream content but also have access to interactive features, thereby enhancing their viewing experience and facilitating product discovery.

[0163] Figure 12b This is a schematic diagram of a live streaming interface for identifying products in an image, according to an embodiment of the present invention.

[0164] like Figure 12b As shown, the live streaming interface has a real-time detection function to monitor products during the live stream. If the online shopping system... Figure 12a If a product is found in the list through interaction with the product link control 1205, this function can be enabled by system control. This function is disabled by default. In some embodiments, this function can be enabled by the user. When a trigger operation is received on the product recognition control, the system switches to an image search model to identify the product. The primary objective is to detect the main product on a screenshot from a live stream.

[0165] After identifying the products, the detected main products are sent to the product list module 1230. The product list module 1230 searches for similar products from the list based on the detected products used for recommendations. Product links are also generated for display in the live stream.

[0166] Figure 12c This is a schematic diagram of a live streaming interface for a product with an identifier according to an embodiment of the present invention.

[0167] like Figure 12cAs shown, after detecting the recommended product, the product link for the earrings is displayed on the live stream interface 1200. Users can interact with the product link 1240 to purchase the product.

[0168] Figure 13 This is a flowchart of a method for product identification according to another embodiment of the present invention. Taking a method applied to the interaction between a user terminal and a server as an example, such as... Figure 13 As shown, the method includes: At step 1301, the user terminal displays a live stream, and the live stream includes a live streamer performing in the live stream environment and at least one item located in the live stream environment.

[0169] User terminals receive the live stream from the server. The live stream is continuously displayed to viewers, allowing them to watch the live streamer and view the products showcased in the environment.

[0170] In some embodiments, the live streaming interface includes a product recognition control, and upon receiving a trigger operation on the product recognition control, it is determined that the terminal has received a product recognition operation. The product recognition operation is configured to perform recognition on at least one product in the live streaming image.

[0171] At step 1302, in response to the product recognition operation, the user terminal obtains the live stream by capturing a screenshot of the live stream and transmits the image to the server.

[0172] The terminal transmits a product identification request to the server. The product identification request is configured to instruct product identification to be performed on the live stream.

[0173] In some embodiments, the product identification request carries a timestamp corresponding to the live stream to be identified, so that the server can obtain the live stream from the cached live stream data based on the timestamp.

[0174] The product identification request also carries user information, and the server stores user preferences and purchase history so that the system can recommend more relevant products to the user based on user preferences or purchase history.

[0175] In step 1303, the server receives the live stream image and performs product recognition on the live stream to obtain at least one product included in the live stream. The object detection module classifies the detected products based on predefined categories and attributes. Each identified object is analyzed to determine its type and other relevant characteristics.

[0176] In step 1304, after receiving the detected product data, the server processes the information to search for matching products in the online shopping product database. The server uses algorithms to find products that closely match the characteristics of the detected product. For example, if a specific type of earring is identified, the server searches the list for other earrings with similar characteristics.

[0177] At step 1305, the server generates at least one product link to recommend to the user.

[0178] The server generates a list of recommended products based on object detection results. This list includes products that are similar to or complementary to the objects detected in the live stream. The recommendations are tailored to match the audience's interests with the products showcased by the live streamer.

[0179] At step 1306, the server transmits at least one product link to the terminal.

[0180] The server transmits the recommended product list back to the user's device. This information is integrated into the live streaming interface, allowing viewers to see product recommendations while watching the live stream.

[0181] In step 1307, the terminal displays product links based on the display area of ​​the products in the live stream.

[0182] The user terminal displays recommended products within the live stream interface. Viewers can interact with these recommendations, view detailed product information, and access links to purchase the recommended items.

[0183] At step 1308, users can interact with the recommended product by clicking the purchase link. This action takes users to the product's details page, where they can view additional information, add the item to their cart, and continue the purchase process.

[0184] Viewers can choose recommended products to view more details or make purchases directly through the app. In some embodiments, the terminal receives a click action on a product link. The user terminal facilitates these interactions, ensuring a seamless shopping experience and integrating it with live streaming.

[0185] Figure 14a This is a flowchart of an image processing method for detecting goods according to an embodiment of the present invention.

[0186] When this method performs product recognition on an image through a user terminal or server to obtain at least one product in the image, the process can be roughly divided into the following steps: receiving the image, preprocessing, feature extraction and selection, comparing the extracted features, calculating the similarity score, and making a product detection decision.

[0187] At step 1401, the method receives an image. In some embodiments, the method captures and processes a live feed in real time. The live feed processor captures and processes video feeds from a live stream. This component continuously receives video data and then converts it into individual frames for further analysis. The processor ensures that the video feed is processed in real time and kept synchronized with the live stream.

[0188] At step 1402, a preprocessing step is applied to the image. This method performs initial data extraction from the image or video feed, including frame extraction and basic quality enhancement. This step prepares the image or video frames for detailed analysis by subsequent components.

[0189] In some embodiments, preprocessing primarily refers to denoising, smoothing, and transform operations in image processing to improve image quality and consistency. Tasks include normalization to adjust brightness and contrast, enhancement to improve frame features, and denoising to reduce visual noise. Preprocessing ensures that subsequent feature extraction processes run on high-quality data.

[0190] At step 1403, the method extracts features from the image. Feature extraction involves capturing and quantifying various attributes that describe the object. The method employs advanced image processing techniques and machine learning algorithms to extract features. In some embodiments, it utilizes a CNN to extract visual features from the image. CNNs are particularly effective in detecting and classifying patterns in images due to their hierarchical structure, which allows them to capture spatial and temporal features. CNNs are trained on large datasets of product images, enabling them to recognize a wide variety of products.

[0191] In some embodiments, key features may include attributes such as shape, color, texture, and other depth features. Shape features can be extracted using techniques such as edge detection (e.g., the Canny edge detector) and contour analysis, while geometric features can be quantized using descriptors such as Hu moments or shape context. Color features can be extracted using techniques such as color histograms (e.g., RGB histograms) or color moments to capture the object's color attributes. Texture features describe the object's surface pattern. Techniques such as gray-level co-occurrence matrix (GLCM) and local binary mode (LBP) are used to quantize texture characteristics.

[0192] At step 1404, the method compares the extracted features. After feature extraction, the extracted features are compared with a database of product images and descriptions. This comparison is performed using similarity metrics and scoring algorithms that evaluate how well the features match known products. The unit identifies the best match based on these scores, thereby achieving accurate product identification.

[0193] Various metrics and algorithms are used to measure similarity. In some embodiments, Euclidean distance is used to measure the straight-line distance between feature vectors. In some embodiments, cosine similarity is used to measure the cosine of the angle between feature vectors to determine their alignment. The cosine of the angle between two vectors is calculated, and it is determined whether the two vectors point approximately in the same direction. In some embodiments, the Structural Similarity Index (SSIM) is used. SSIM measures the similarity between two images based on brightness, contrast, and structure terms.

[0194] At step 1405, a similarity score is calculated. The similarity score quantifies the degree of matching between the items in the input image and the items in the reference image. A higher score indicates greater similarity. These scores can be used to rank or categorize the detected items.

[0195] At step 1406, one or more items are detected. In some embodiments, the system evaluates each item by comparing its similarity score to a predefined threshold. If the similarity score exceeds the threshold, the system confirms the detection of the item and provides a result accordingly. Conversely, if the score falls below the threshold, the item is not detected, and no result is generated. In some cases, when the similarity score is below the threshold, the system may issue a notification indicating that the item cannot be confidently identified. This approach ensures that only items with high similarity are detected.

[0196] Figure 14b This is a flowchart of an image processing method for detecting goods according to another embodiment of the present invention.

[0197] The image processing steps can be divided into two stages. In stage 1, the bounding box of each item in the product list is determined. In stage 2, different image processing algorithms are used to determine the similarity between images to identify the products.

[0198] In Phase 1, this step involves detecting objects within the image using bounding boxes at step 1410. A bounding box is a rectangular box drawn around each detected object to define its location and extent. To achieve this, various image processing algorithms and techniques are employed, including: In some embodiments, advanced machine learning algorithms are employed to identify bounding boxes. For example, algorithms such as YOLO (You Only Look Once), CNN, Faster R-CNN (region-based convolutional neural network), or SSD (Single Shot Multiple Box Detector) are used to detect and locate objects by predicting bounding boxes.

[0199] These algorithms typically use pre-trained models trained on large datasets such as COCO (Common Objects in Context) or ImageNet. These models can identify a wide variety of objects and provide bounding boxes around them.

[0200] In some embodiments, after the initial detection of bounding boxes, post-processing techniques such as nonmaximum suppression (NMS) are applied to eliminate redundant boxes and retain the most accurate boxes.

[0201] In phase 2, at step 1420, it uses a different algorithm to determine the similarity between images to identify the goods.

[0202] In some embodiments, image processing is performed using a Contrastive Language–ImagePre-training (CLIP) model 1430 to identify goods.

[0203] The CLIP model learns to understand images and text by being trained on a large image-text pair dataset. The model aligns visual and textual information in a shared embedding space, enabling it to perform a variety of tasks, including object detection, by leveraging image and textual descriptions.

[0204] The detailed steps are as follows: The input image is processed by an image encoder using the CLIP model to obtain a feature vector. This vector captures the visual information of the image in a high-dimensional space. When analyzing an image, its key features are first extracted, which may include characteristics such as color, shape, texture, and spatial layout. These features are then processed by an image embedding model, which transforms each feature into a high-dimensional vector representation called a feature vector.

[0205] Once the feature vector of the image is generated, it is compared with the embedded feature vector of the reference image. This comparison is based on mathematical similarity measures, such as cosine similarity or Euclidean distance, which measure how close the vector representations are. If the similarity score between the embedded feature vectors of the image and the reference image exceeds a predetermined threshold, the system identifies one or more items in the image as matching the reference image.

[0206] This embedding-based comparison allows for robust product identification even when scale, orientation, or lighting conditions vary in the input image. By converting image features into a high-dimensional vector space, the system can perform more accurate and efficient product detection.

[0207] In some embodiments, the Vision Transformer (ViT) model is used in the CLIP model. ViT applies a transformer architecture commonly used in NLP tasks to image processing. Unlike CNNs that rely on hierarchical feature extraction, ViT processes image patches as sequences and treats them in a manner similar to word processing in NLP. This approach leverages a self-attention mechanism to capture long-range dependencies and contextual information in the image. The ViT model divides the image into fixed-size blocks, correctly embeds each block within the block, and uses the position embedding as input to the transformer encoder.

[0208] In some embodiments, the BERT model 1440 is combined with the CLIP model 1430 to enhance object detection by leveraging advanced text processing and image understanding.

[0209] BERT's bidirectional encoding captures the subtle meanings and contexts of words, enabling more accurate similarity assessments between image content and text descriptions. BERT processes text cues to generate embeddings that capture the context of each description. Object-related text descriptions or cues are preprocessed and tokenized. BERT encodes these tokenized texts into dense embeddings. BERT's bidirectional approach allows it to understand the context of each word in the cues relative to other words, enhancing semantic understanding of the descriptions. Using the CLIP model's text encoder, the BERT-encoded text descriptions are mapped to the same feature space as the image embeddings.

[0210] By integrating the BERT model, object detection systems can better understand complex text descriptions, thereby improving the classification of objects within bounding boxes.

[0211] In some embodiments, to adapt the CLIP model to a specific object detection task, a fine-tuning step 1450 is performed to improve its performance on a specialized dataset or to focus on a specific object category. This involves adjusting the weights of the pre-trained model to better fit new data while retaining the general knowledge gained during initial training. Fine-tuning can improve detection accuracy for a specific task.

[0212] Therefore, object detection using CLIP models combines powerful visual and textual understanding to achieve accurate and flexible detection capabilities. CLIP models enable complex object detection processes by leveraging their ability to align image and text representations, handling a wide variety of objects and descriptions with minimal task-specific training.

[0213] Sophisticated image processing algorithms are used to ensure high accuracy in product recognition, thereby improving the reliability of product information and enhancing user satisfaction.

[0214] Figure 15 This is a flowchart of a method for displaying recommended products in an online shopping system according to an embodiment of the present invention.

[0215] At step 1501, the output of the detected item from the image processing step is received.

[0216] In step 1502, the product list of the online shopping platform is searched based on the detected products. The online shopping platform has a database of products for sale.

[0217] The system extracts key attributes from detected products. These attributes may include product category, brand, color, size, material, and any other relevant characteristics. These attributes provide the basis for finding similar products.

[0218] In some embodiments, the system constructs a search query using extracted attributes. This query is designed to find products that match or are similar to the selected product. The search query may include keywords, filters, and other parameters derived from product attributes. The search query is sent to a product database, where it is used to search for products that meet the criteria. The database includes a wide variety of products, and the search process identifies products whose attributes closely match those of the selected product.

[0219] At step 1503, the method identifies similar products. The method identifies a list of similar products based on search results. This list includes products that match the attributes of the selected product or belong to the same category. The method may rank these products based on relevance, popularity, and / or user preference. The method then generates product recommendations for the user.

[0220] In some embodiments, the method recommends products based on product attributes and the user's past interactions with similar products. The system analyzes product characteristics and user preferences to provide suggestions consistent with the user's known interests.

[0221] In some embodiments, a relevance score for recommended products is calculated based on factors such as user preferences, browsing history, and current trends. These scores help rank and present the most relevant recommendations to the user. The products with the highest scores are recommended to the user.

[0222] At step 1504, for each recommended product, the method generates a purchase link. This link directs the user to the product's details page, where they can view more information and complete the purchase. The link is typically created using a unique identifier for the product within a database.

[0223] In step 1505, the recommendation and its corresponding purchase link are displayed to the user.

[0224] Therefore, with respect to this invention, by providing real-time product identification, users can obtain instant information about the products they see, thereby creating a more engaging and interactive shopping experience. Users can easily identify and access product details without manually searching for products, thus simplifying the shopping process and potentially increasing conversion rates.

[0225] By addressing the need for enhanced interactivity and efficiency in online shopping, this invention provides novel and valuable solutions for both users and retailers in the digital marketplace.

[0226] For this invention, routines for specific embodiments can be implemented using any suitable programming language, such as C, C++, Java, and assembly language. Different programming techniques can be employed, such as procedural or object-oriented programming. These routines can execute on a single processing device or multiple processors.

[0227] Specific embodiments may be implemented in a computer-readable storage medium (also known as a machine-readable storage medium) for use by or in conjunction with an instruction execution system, apparatus, system, or device. Specific embodiments may be implemented in the form of control logic in software or hardware, or a combination of both. The control logic, when executed by one or more processors, may be operable to perform the content described in the specific embodiments.

[0228] While various aspects and embodiments have been disclosed herein, it will be apparent to those skilled in the art, upon reading the foregoing disclosure, that various other modifications and adjustments to the invention will be apparent without departing from the spirit and scope of the invention, and all such modifications and adjustments are intended to be included within the scope of the appended claims. The aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting; the true scope and spirit of the invention are indicated by the appended claims.

Claims

1. A method for providing recommendations, the method comprising: Receive queries from users on the first platform; Perform trend analysis based on the query; One or more recommendations are generated based on the results of the trend analysis. as well as The one or more recommendations are provided to the user via the first platform.

2. The method of claim 1, wherein the query is received in the form of text input by the user.

3. The method according to claim 2, further comprising: Receive the query from the user; Analyze the query; as well as Based on the analysis, the intent of the query is classified as general or specific; If the query is determined to be a general query, the method further includes forwarding the query to a second platform to find trending products on the second platform.

4. The method according to any one of claims 1 to 3, wherein the step of performing trend analysis based on the query further comprises: Send the query to the second platform; Based on the query, retrieve multiple videos from the second platform; Identify at least one trending product from the plurality of videos; Multiple products are generated by the first platform by matching them with at least one trending product identified in the second platform; as well as The multiple products are recommended to the user via the first platform.

5. The method according to claim 4, further comprising: Take screenshots at different times from the multiple videos; Use an image search model to identify one or more items in the screenshot; Determine the similarity between one or more identified products; Select the product with the highest similarity score as the at least one trending product from the second platform; Matching products from the first platform with at least one trending product from the second platform; and The one or more recommendations are determined based on the products matched on the first platform.

6. The method according to any one of claims 1 to 5, further comprising: Track queries submitted by users within a predetermined time period; Multiple search queries are identified based on the frequency of submission during the predetermined time period; The second platform performs a search for the identified search query and retrieves video results; Identify trending products from the retrieved videos; as well as The multiple search queries and identified trending products are pre-stored.

7. The method according to claim 6, further comprising: Receive the query from the user; as well as Match the query with the plurality of search queries; in If the query matches one of the multiple search queries, then the pre-stored trending products for the corresponding multiple queries are accessed; or If the query does not match any of the multiple queries, the query is processed in real time.

8. The method according to any one of claims 1 to 7, further comprising: Based on the trend analysis, identify multiple trending products related to the query; as well as The first platform displays one or more recommendations based on the identified trending products in a balanced manner.

9. The method of claim 1, wherein the query is received in the form of the user interacting with video content.

10. The method of claim 9, wherein the method further comprises: Receive at least one video frame from the video content; One or more items are identified by using an image search model through processing at least one video frame; Match the one or more items to identify one or more products from the first platform; Create a link for each of the one or more products; as well as The link is displayed to the user through the user interface within the video content.

11. The method of claim 10, wherein the step of matching the one or more goods comprises: Image search is used to match the one or more products with the product library in the first platform.

12. The method of claim 10, wherein the step of identifying one or more goods further comprises: Create a search string based on one or more identified products.

13. The method of claim 12, wherein the step of matching the one or more goods comprises: Use the search string to perform a search on the first platform.

14. The method according to any one of claims 10 to 13, wherein one or more of the identified products are different from the products currently promoted in the video content.

15. The method of claim 5 or 10, wherein the step of using an image search model to identify one or more items comprises: Receive an image that includes one or more objects; Use an object detection algorithm to identify one or more bounding boxes around each of the one or more objects in the image; Extract features from each of the one or more objects within the identified bounding box; The extracted features from the image are compared with a reference image to determine similarity; A similarity score is generated, which quantifies the degree of similarity between each of the one or more objects in the image and the reference image; as well as Determine whether the similarity score of the one or more objects is higher than a threshold.

16. The method of claim 15, wherein the step of identifying one or more objects comprises: The extracted features are embedded into a feature vector using an image embedding model, wherein each feature in the extracted features is converted into a high-dimensional vector representation; and The reference image and the feature vector of the image are compared to identify the one or more objects.

17. The method of claim 16, further comprising the step of: The image embedding model is fine-tuned using a Bidirectional Encoder Representation (BERT) model approach from the transformer for semantic understanding, wherein the BERT model is used to capture the contextual relationships between the extracted features.

18. A system for providing recommendations, the system comprising: processor; A memory that communicates electronically with the processor; as well as Instructions, which are stored in the memory and can be executed by the processor to cause the system to perform the method according to any one of claims 1 to 17.

19. A computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed in a computer, causes the computer to perform the method according to any one of claims 1 to 17.

Images