Information recommendation method, device and equipment and readable storage medium

Abstract

The invention discloses an information recommendation method, device and equipment and a readable storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: receiving a first input for a first application interface; in response to the first input, displaying recommendation information on the first application interface; wherein the recommendation information is information which is output based on recommendation intention description information generated by performing semantic intention analysis on recommendation content information and context scene information corresponding to the first input through a multi-modal large model.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an information recommendation method, apparatus, device, and readable storage medium. Background Technology

[0002] In today's information age, users face a massive amount of data, making it crucial to efficiently and accurately obtain personalized recommendations. Traditional methods often require users to manually migrate information between multiple applications to retrieve specific recommendations based on existing data, resulting in lengthy processes and a fragmented user experience. Furthermore, existing recommendation methods typically rely on a single data source, making it difficult to integrate cross-platform and multimodal information, leading to a lack of richness and accuracy in recommendations. Summary of the Invention

[0003] The purpose of this application is to provide an information recommendation method, apparatus, device, and readable storage medium that can improve the convenience of obtaining recommendation information while also improving the accuracy of the recommendation information.

[0004] In a first aspect, embodiments of this application provide an information recommendation method, the method comprising: Receive the first input to the first application interface; In response to the first input, recommended information is displayed on the first application interface; The recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

[0005] Secondly, embodiments of this application provide an information recommendation device, the device comprising: The first receiving module is used to receive the first input to the first application interface; The first display module is used to display recommended information in response to the first input on the first application interface; The recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

[0006] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment of the application, a first input to a first application interface can be received; in response to the first input, recommended information is displayed on the first application interface; wherein, the recommended information is generated by semantic intent parsing of the recommended content information and contextual information corresponding to the first input through a multimodal large model, generating recommended intent description information, and outputting information based on the recommended intent description information.

[0011] This allows for direct response to input on the first application interface, displaying recommended information directly within that interface without requiring cross-application operations. This simplifies the workflow and improves the ease of accessing recommended information. Furthermore, it supports recommendations for various information types, including text, images, audio, video, and links, adapting to different input scenarios. Leveraging the semantic parsing capabilities of a multimodal large-scale model, combined with contextual information, it extracts recommendation intent descriptions from complex inputs to output corresponding recommendations. This ensures that the recommended information more closely matches the user's true intent, improving the accuracy of the recommendations. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the information recommendation method provided in some embodiments of this application; Figure 2 This is a schematic flowchart of a scenario embodiment of the information recommendation method provided by some embodiments of this application; Figure 3 This is a schematic flowchart of another scenario embodiment of the information recommendation method provided by some embodiments of this application; Figure 4 These are schematic diagrams of the information recommendation device provided in some embodiments of this application; Figure 5 These are schematic diagrams of the structure of electronic devices provided in some embodiments of this application; Figure 6 These are schematic diagrams of the hardware structure of electronic devices provided in some embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The information recommendation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0016] Figure 1 This is a flowchart illustrating an information recommendation method provided in some embodiments of this application. The information recommendation method may include: Step 101: Receive the first input to the first application interface.

[0017] In step 101, the first application interface can be the interface of various types of applications such as news and information or instant messaging. In order to facilitate the description of the information recommendation method provided in the embodiments of this application, the following description will take the interface of an instant messaging application as the first application interface.

[0018] The first input may be an input that triggers the selection of at least some of the multimodal information in the first application interface. The selected multimodal information may be the recommended content information corresponding to the first input, and may be one or more of text, images, audio, video and links, without specific limitations here.

[0019] For example, the first input can be an input that selects at least part of the content in the first application interface, and the information within the selected range can be the recommended content information corresponding to the first input.

[0020] In some examples, the first application interface can be a chat interface where a user discusses lipsticks with an instant messaging contact. This chat interface contains text messages, voice messages, and images of lipsticks. By selecting these messages, recommended content information can be obtained, and the recommended content information can be lipsticks.

[0021] In other examples, the recommended content information can indicate a location, and the corresponding recommendation information may include travel information such as route planning, ride-hailing services, or nearby restaurants. The recommended object can also be a TV series or movie, and the corresponding recommendation information may include information such as the streaming source, rating details, and ticket purchase links. The recommended object can also be a smart home device, and the corresponding recommendation information may include control information such as pre-activating smart home devices.

[0022] Understandably, in instant messaging applications, users can activate recommendation services through an interactive method registered at the operating system or application plugin level. Examples include long-pressing the power button, long-pressing the navigation bar, or pulling up the sidebar. After activating the recommendation service, the main flow can begin, meaning the application can now receive initial input. Taking "finger selection" as an example, users can then define their interests using gestures to determine the recommended content corresponding to their initial input.

[0023] Step 102: In response to the first input, display recommended information on the first application interface; The recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

[0024] In some examples, the recommended content information includes at least one of the following: text, images, audio, video, and links; the contextual information includes at least one of the following: user context information and device context information.

[0025] In step 102, in response to the first input, the system immediately collects the recommended content information corresponding to the first input, such as multimodal content within the user's selected area, including one or more of text, images, audio, video, and links, and marks it as key content. Simultaneously, it also collects contextual information related to this first input operation.

[0026] Contextual information refers to auxiliary information associated with the target content that helps understand the user's true intent. It may include at least one of the following: user context information and device context information. User context information may include historical dialogue information, user profile information, and real-time preference information. Device context information may include the time and location of the terminal device at the time of the interaction, and other terminal device status information.

[0027] The collected recommendation content information and contextual information can be input into a multimodal big data model for processing. The multimodal big data model performs semantic intent parsing on the recommendation content information and contextual information to deeply understand the user's true intent and generate recommendation intent description information.

[0028] Based on the recommendation intent description information, results matching the recommendation intent description information can be found from other related applications to obtain recommendation information, which can then be output through a multimodal large model.

[0029] The corresponding recommendation information can be displayed directly in the first application interface through floating windows, message pop-ups, or message cards.

[0030] In this embodiment, the information recommendation method can receive a first input to a first application interface; in response to the first input, it displays recommendation information on the first application interface; wherein, the recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information; the recommendation content information includes at least one of the following: text, image, audio, video, and link; the contextual information includes at least one of the following: user context information and device context information.

[0031] This allows for direct response to input on the first application interface, displaying recommended information directly within that interface without requiring cross-application operations. This simplifies the workflow and improves the ease of accessing recommended information. Furthermore, it supports recommendations for various information types, including text, images, audio, video, and links, adapting to different input scenarios. Leveraging the semantic parsing capabilities of a multimodal large-scale model, combined with contextual information, it extracts recommendation intent descriptions from complex inputs to output corresponding recommendations. This ensures that the recommended information more closely matches the user's true intent, improving the accuracy of the recommendations.

[0032] In some embodiments, displaying recommendation information in response to a first input may include: In response to the first input, obtain the recommended content information and contextual information corresponding to the first input; The recommended content information and contextual information are preprocessed to obtain structured information; Structured information is input into a multimodal big model, which then performs semantic intent parsing on the structured information to generate recommendation intent description information. A query request is generated based on the recommendation intent description information through a multimodal large model and sent to N second applications, where N is a positive integer; Receive query results returned by N second applications; Recommendations are generated based on query results using a multimodal large model; Recommended information is displayed on the first application interface.

[0033] In this embodiment, as mentioned above, in response to the first input, the system can collect the recommended content information corresponding to the first input, as well as the contextual information related to the first input operation.

[0034] The recommended content information and contextual information can be preprocessed to obtain structured information. This structured information can include attribute information and the attribute value corresponding to each attribute. The attribute information can include the interaction type of the first input, and its corresponding attribute value can be a gesture selection, an image fragment within the selected area (stored as a Base64 encoded string), text information within the selected area, text transcribed from voice messages within the selected area, and contextual information related to the selection operation, including chat history before the selection operation, user profile information, and device status information.

[0035] Structured information can be input into a multimodal large model, which then performs semantic intent parsing on the structured information to generate recommendation intent description information.

[0036] For example, a first cue word can be used to indicate that the multimodal big model deeply understands the user's true intent in order to generate recommendation intent description information based on structured information.

[0037] Taking the recommended content information corresponding to the first input, including product images, chat content related to the product, etc., as an example, the first suggestion word could be: "You will extract information from the image according to the given format. The extracted information will be mainly used for product purchase decisions."

[0038] Extraction format requirements: - desc: A complete description of the product information in the image. - items: A list of products. This lists all the products you identified in the image. - categories: Product categories.

[0039] - desc: Detailed description of this product. Includes shape, color, brand, and other information you deem important. - subject: true / false. Is this product the main product in the image? This can be determined based on the product's visual position or the area selected by the user.

[0040] - Conversations: Users' text conversation history. Extracting chat logs between the user and the other party based on common instant messaging software layouts. The conversation history must include all records within the complete screenshot. The multimodal large model can output recognition results based on the first prompt word. The recognition results can be: “items: - Categories: [Beauty, Lipstick] Description: A red lipstick with a velvety texture. It features a black and gold tube design with the words "VELVET KISS" printed on it.

[0041] subject: true desc: A screenshot from a mobile phone shows a conversation about lipstick. In the conversation, one person shares a picture of a lipstick, saying it suits them perfectly. The other person replies that it looks great and suggests using it for photos at the beach next time. The circled part in the picture is an open lipstick.

[0042] conversations: The other person: Look! This lipstick suits me perfectly! The other party: [Image] Me: It looks great! When are you planning to use it? The other party: [Voice message 56s] Me: Oh! I see XXX did wear this color before, we can use it for our next beach photoshoot! By combining multimodal large models with the contextual information in structured information, a structured description of the recommendation intent can be inferred and generated.

[0043] A query request can be generated based on the recommendation intent description information using a multimodal large model and sent to one or more second applications. It's understandable that, taking the user's true intent as purchasing goods as an example, the second application could be a shopping platform software.

[0044] For example, the system packages the structured recommendation intent description information generated in the previous step into an anonymous query request and distributes it to multiple mainstream shopping platforms. Understandably, if a shopping platform does not provide a smart agent interface, it may reduce the dimensionality of the query request. The content sent depends on the shopping platform's agent's natural language understanding capabilities and the specific format of its API interface. No specific limitations are specified here.

[0045] It can receive query results returned by these second applications. For example, each shopping platform can return a list of products that match the recommendation intent description information, including at least one product information.

[0046] Multimodal large models can be used to generate recommendation information based on the query results returned by these second applications. For example, a multimodal large model can integrate product information returned by multiple shopping platforms and then output product recommendations.

[0047] The recommended information output by the multimodal large model can then be displayed on the first application interface.

[0048] In this way, on the one hand, preprocessing transforms raw multimodal data into structured information, reducing the computational load of large multimodal models, improving inference efficiency, and optimizing resource consumption. On the other hand, sending query requests to multiple secondary applications aggregates results from different sources, avoiding the data limitations of a single platform and improving the diversity of recommendations.

[0049] In some embodiments, when N is an integer greater than 1, generating recommendation information based on the query results using a multimodal large model may include: Using a multimodal large model, information processing is performed on the query results returned by N second applications to obtain M recommendation sub-information; information processing includes at least one of aggregation processing and deduplication processing; Using a multimodal large model, M recommendation sub-information items are sorted according to a preset sorting rule to generate a recommendation list, where M is a positive integer.

[0050] In this embodiment, a multimodal large model can be used to aggregate and / or deduplicate the query results returned by N second applications to obtain M recommendation sub-information pieces. These M recommendation sub-information pieces are then sorted to generate a recommendation list. For example, the aggregation, deduplication, and sorting here can use a model-as-a-judge (LLM) strategy, which is not specifically limited here.

[0051] This process deduplicates results from N secondary applications, avoiding information redundancy and improving user browsing efficiency. By sorting recommended sub-information based on multiple dimensions, the most relevant or optimal options are prioritized for display, making the recommendation list more aligned with users' actual needs and preferences, thus increasing click-through rates.

[0052] In some embodiments, the recommended content information indicates a product as the recommended object, and the second application is a shopping platform; Using a multimodal large model, M recommendation sub-information items are sorted according to a preset sorting rule to generate a recommendation list, which may include: By using a multimodal large model, the weighted score value corresponding to each recommendation sub-information is determined based on K dimensions; Then, using a multimodal large model, the M recommendation sub-information are sorted according to a preset sorting rule based on the weighted score values ​​of the M recommendation sub-information to generate a product recommendation list; K is a positive integer; The K dimensions include at least one of the following: The semantic matching degree between the recommended sub-information and the recommended intent description information; The degree of matching between recommended sub-information and user profile information; Confidence level of product quality corresponding to the recommended sub-information; The product price corresponding to the recommended sub-information; The confidence level of the shopping platform corresponding to the recommended sub-information.

[0053] In this embodiment, when the recommended object indicated by the recommended content information is a product and the second application is a shopping platform, a multimodal large model can be used to determine the weighted score value corresponding to each recommended sub-information based on K dimensions; according to a preset sorting rule, the M recommended sub-information items are sorted based on the weighted score values ​​of the M recommended sub-information items to generate a product recommendation list.

[0054] For example, the K dimensions may include one or more of the following: semantic matching degree between the recommendation sub-information and the recommendation intent description information; matching degree between the recommendation sub-information and the user profile information; confidence degree of product quality corresponding to the recommendation sub-information; product price corresponding to the recommendation sub-information; and confidence degree of shopping platform corresponding to the recommendation sub-information.

[0055] For example, the higher the semantic match between the recommended sub-information and the description of the recommendation intent, the higher the rating of that sub-information. Similarly, the higher the match between the recommended sub-information and the user profile information, the higher the rating. The confidence level of the product quality can be calculated based on the positive review rate and sales volume of the recommended sub-information; the higher the confidence level, the higher the rating. The lower the price of the product associated with the recommended sub-information, the higher its rating. Finally, the higher the confidence level of the shopping platform associated with the recommended sub-information, i.e., the higher the platform's reputation, the higher its rating.

[0056] We can perform a weighted calculation based on the score values ​​of each recommendation sub-information in K dimensions and the preset weights corresponding to the K dimensions to obtain the weighted score value of each recommendation sub-information.

[0057] The weighted scores of M recommendation sub-information items can be sorted according to preset sorting rules. For example, the preset sorting rules can be sorting the weighted scores from largest to smallest, or from smallest to largest. Alternatively, the maximum weighted score can be used as the middle position in the sorting, with the weighted score decreasing towards both ends from the middle position. The specific settings can be configured according to actual needs, and no specific limitations are made here.

[0058] The sorted M recommendation sub-information can be used to generate a product recommendation list.

[0059] The product recommendation list can be displayed on the primary application interface. For example, it can be presented as a lightweight, semi-transparent floating card. Each card clearly displays the product's main image, title, price, and source platform. In some examples, the card also includes an AI-generated "Recommendation Reason," directly addressing the user's deeper intent, such as, "This is the same model mentioned in the chat, and it's the lowest priced across all platforms," ​​assisting the user's decision-making. In other examples, the system can also help users find previously used products based on its memory, aiding in deeper decision-making. The recommendation reasons here can be dynamically adjusted, representing the most compelling reasons for purchasing the product given by a multimodal big data model based on a comprehensive understanding of the user.

[0060] In this way, by using K dimensions to weight the score, we can avoid bias from a single dimension, improve the accuracy of the recommendation information, and make the recommendation list more in line with individual needs.

[0061] In some embodiments, after displaying recommendation information on the first application interface, the method may further include: Obtain user interaction information regarding the recommendation list; Based on the interactive operation information, determine the corresponding feedback results for the recommendation list; Based on the feedback, update the preset sorting rules.

[0062] In this embodiment, taking a product recommendation list as an example, user interaction information on the product recommendation list can be obtained. This interaction information may include user actions such as clicking to view product information or purchasing a product. It may also include relevant information from user-provided feedback.

[0063] For example, each product card in the product recommendation list can have simple "like" and "dislike" controls. If a user clicks the "dislike" control, the system will immediately record it and reduce the number of similar products recommended in the future. If a user clicks the "like" control, the system will immediately record it and add similar products to the recommendations in the future. Next to each product card, a question mark or "why" control can also be provided. Clicking this can allow the system to explain the reason for the recommendation in natural language, such as: "Based on your mention of 'velvet texture' and 'beach photos' in your chat, we recommend this lipstick to you." Users can directly hide product cards or the entire pop-up window that they are not interested in; this behavior also serves as negative feedback, helping the system better understand the user's real-time preferences. A text or voice input box can also be provided, allowing users to specify why they dislike a particular recommended product, such as: "I don't like this color," "The price is too high," or "I don't like this brand."

[0064] Based on interactive operation information, the system can determine the feedback results corresponding to the product recommendation list and update the preset sorting rules based on the feedback results.

[0065] For example, the entire process of generating the product recommendation list can be modeled as a sequential decision Markov process, aiming to directly optimize the final presentation quality of the product recommendation list. This sequential decision Markov process may include the following components: State: State S t It not only includes user profile information and historical behavior sequences, but also dynamically includes the first t-1 ordered items already generated in the current product recommendation list. This design allows subsequent decisions to fully utilize the contextual information of "what is already available".

[0066] Action: In state S t Below, the model's action A t The process involves selecting the t-th most suitable item from a massive pool of candidate items and placing it at the t-th position in the list. This process is repeated iteratively until a complete, ordered list of recommended items containing M sub-items is generated.

[0067] Policy: The large multimodal model itself is the policy π(A) t │S t The goal is to learn an optimal sequence generation strategy to maximize the expected total reward of the final list.

[0068] To accurately guide the optimization of the strategy model, a comprehensive reward function for the entire product recommendation list can be designed. This function is calculated after a complete user interaction, i.e., after the user browses and interacts with a product recommendation list, and its structure is as follows: It integrates multi-dimensional feedback signals. The reward function integrates all explicit and implicit feedback from users on this list, including but not limited to purchases, clicks, effective dwell time, ignoring, quick swipes, likes / dislikes, etc.

[0069] Position bias correction. The function has a built-in mechanism to correct for the "position bias" that users naturally tend to focus on at the top of the list. Specifically, positive interactions that occur later in the list, such as clicks and purchases, will be given significantly higher reward weights because this better reflects the item's appeal to the user, rather than its position.

[0070] Alignment with ranking metrics. The reward function is designed to be directly linked to core list evaluation metrics in the industry. User interactions, such as clicking the third item and skipping the first two, are directly interpreted as implicit confirmation of the local ranking item3>{item1,item2}. This ranking preference information is integrated into the final reward value, ensuring end-to-end consistency between model optimization and the business objective function.

[0071] Incentivize diversity and novelty. The reward function can include a penalty to punish situations where items in the list are too homogeneous, or an incentive to reward those who successfully guide users to explore new areas, such as when a user clicks on a recommended list of a certain category for the first time.

[0072] This list-level reinforcement learning paradigm can fundamentally improve model performance across multiple dimensions: First, the model no longer understands each product in isolation, but learns the synergistic and substitutive relationships between products. For example, the model learns that displaying products with similar functions in a list will reduce the overall click-through rate, while displaying complementary products, such as cameras and tripods, can increase the overall value.

[0073] Secondly, by analyzing users' browsing and clicking sequences in the product recommendation list, the model can more accurately capture users' complex and dynamic consumption intentions. For example, a user's behavior of browsing high-priced products first and then clicking on mid-priced products will be interpreted by the model as an intention to "pursue cost-effectiveness," rather than simply being interested in mid-priced products.

[0074] Third, the optimization objective has shifted from "maximizing the expected return of a single point" to "maximizing the overall expected return of the list." The model will generate a product recommendation list with coherent internal logic and a strong narrative, which may include "high-popularity products, high-precision matching products, and exploratory products," maximizing user dwell time, interaction depth, and final conversion rate as a whole, achieving a globally optimal sorting strategy.

[0075] In this way, the sorting rules are adjusted in real time based on the feedback obtained from user interaction, enabling the recommendation system to continuously iterate, enhance personalization, and improve long-term recommendation performance.

[0076] In some embodiments, the recommendation information is product recommendation information, which includes at least one product information; After displaying the recommended information on the first application interface, the method may also include: Receive a second input regarding the first product information; the first product information is any one of the product information included in the product recommendation information; In response to the second input, the third application is launched; the third application is the application corresponding to the first product information. The detailed information of the first product is displayed on the application interface of the third application.

[0077] In this embodiment, the recommendation information can be product recommendation information, which may include at least one product information. In other words, at least one product information can be displayed in a floating window on the first application interface.

[0078] Users can proceed to the next step within the floating window. For example, clicking the floating card corresponding to the first product information will seamlessly link to the corresponding third application details page for purchase.

[0079] In some examples, interactive operations such as "add to comparison list", "share with friends" or "set price drop reminders" can also be performed within the floating window.

[0080] In this way, after clicking on a recommended product, users can be directly redirected to the source application so that they can continue with the purchase process, reducing the number of steps required.

[0081] In some embodiments, user context information includes user profile information; after displaying recommendation information on the first application interface, the method may further include: Obtain user interaction information regarding recommended information; Update user profile information based on interactive operation information.

[0082] In this embodiment, user context information may include user profile information. As mentioned above, user interaction information with the product recommendation list can be obtained. This interaction information may include user actions such as clicking to view product information or purchasing a product. It may also include relevant information related to feedback directly provided by the user.

[0083] User preference information can be determined based on interactive operation information, and user profile information can be updated based on user preference information.

[0084] For example, continuously collected user feedback is used to refine and iterate user profile information. Within the list-level optimization framework, the system can not only learn users' isolated preferences, such as "likes brand A," but also capture their deep contextual selection preferences, such as "after seeing a jacket from brand A, the user is more likely to browse casual pants from brand B." This fine-grained modeling of the interactions between items gives user profile information unprecedented contextual understanding.

[0085] Understandably, through the analysis of the complex decision-making logic of massive numbers of users, despite the vast differences in their decision-making paths, their final purchasing behavior and core preferences often exhibit significant "convergence." For example, in a specific season, most users' shopping needs will concentrate on a few categories, such as winter clothing. At specific marketing moments, users' attention will be significantly attracted to top-selling products. This convergence reveals the principle of "the finiteness of user profiles." That is, in commercial recommendation scenarios, although individual users are unique, the number of user profile information prototypes that constitute effective commercial value is vast but finite. These can be summarized into a series of representative group patterns, such as "college students seeking cost-effectiveness" and "young mothers concerned about infant health."

[0086] Based on the convergence of user behaviors and the limitations of user profiles, a unified multimodal model can be used to meet the needs of the vast majority of users. This is not only feasible but also highly efficient. This unified multimodal model, with its vast parameter space and powerful learning capabilities, can solidify tens of thousands of user profile prototypes and their complex contextual decision-making logic as "patterns" within the model. During actual recommendations, the system only needs to identify the user's current contextual clues to activate the most matching pattern from this multimodal model, thereby generating highly personalized recommendation results. This avoids the enormous technical and computational overhead of maintaining independent models for each segmented group or even individual, truly achieving scalable, highly efficient, and deeply personalized services.

[0087] In this way, user profile information is dynamically updated based on user actions related to recommended information, ensuring that the recommendation strategy is consistent with the user's latest preferences or interests and improving the accuracy of recommended information.

[0088] In some examples, during the information recommendation process described above, when the system encounters a problem at any step, a corresponding error handling mechanism is needed to ensure user experience and avoid process interruption.

[0089] When no results are found or the recommendation information is mismatched, if the multimodal large model fails to generate effective, structured descriptions of the recommendation intent, or if the query results returned by the second application have a very low match with the user's intent, the system should not directly display a blank page or error message. Instead, it should provide a user-friendly, non-disruptive alternative solution and offer other entry points. For example: "Sorry, no perfectly matching products were found. Please try other methods: image search, or manually enter keywords." When a network or system failure occurs, the user should be clearly notified of the network failure, and retry and network diagnostic functions should be provided.

[0090] When a user's intent is unclear, if the multimodal big data model is ambiguous or uncertain about the user's intent, the system can proactively guide the user to provide more information. For example, it can ask the user via a dialog box or voice, "Could you tell me more specific thoughts?" This not only improves the accuracy of recommendations but also makes the user feel that the system is more intelligent and human-like.

[0091] To facilitate understanding of the information recommendation method provided in the above embodiments, the following describes the information recommendation method using a specific scenario embodiment. Figure 2 A schematic flowchart of a scenario embodiment of the information recommendation method provided for some embodiments of this application.

[0092] like Figure 2 As shown, this scenario embodiment may specifically include the following steps: Step 201: Receive gesture selection input on the first application interface to trigger the product recommendation service; Step 202: Obtain the recommended content information and contextual information corresponding to the gesture selection input; Step 203: Semantic intent parsing is performed using a multimodal large model to generate recommendation intent description information; Step 204: Determine if the recommendation intent description information contains a product recommendation intent; if yes, proceed to step 205; otherwise, end. Step 205: Generate a query request based on the recommendation intent description information and send it to N second applications, and receive the query results returned by N second applications; Step 206: Determine whether all query results do not match the recommended intent description information; if not, proceed to step 207; otherwise, proceed to step 208. Step 207: Generate product recommendation information based on the query results that match the description information of the recommendation intent; Step 208: Display product recommendation information via a floating window on the first application interface; Step 209: In response to input of any product information in the product recommendation information, redirect to the corresponding product details page of the application to make a purchase; Step 210: The first application interface displays "No suitable product found".

[0093] like Figure 3 As shown, some embodiments of this application provide a multi-party interactive scenario embodiment of the information recommendation method. This scenario embodiment takes an instant messaging application as the first application, a product as the recommendation object, and a shopping platform as the second application for illustration.

[0094] This scenario embodiment may include: Step 301: The user makes a selection in the chat interface of the client. Step 302: The client obtains the recommended content information and contextual information corresponding to the selection operation; Step 303: The client sends the recommended content information and contextual information to the edge model for preprocessing; Step 304: The client-side model returns structured information to the client. Step 305: The client sends structured information to the cloud server; the cloud server deploys a multimodal large model. Step 306: The cloud server performs semantic intent parsing on the structured information using a multimodal large model to generate recommendation intent description information; Step 307: The cloud server generates a query request based on the recommendation intent description information using a multimodal large model; Step 308: The cloud server distributes the query request to the shopping platform; Step 309: Each shopping platform returns the query results to the cloud server; Step 310: The cloud server processes and intelligently sorts the query results using a multimodal large model to obtain a product recommendation list; Step 311: The cloud server returns a list of recommended products to the client; the client can then display this list of recommended products. Step 312: The user clicks on the first product in the product recommendation list; the first product can be any product in the product recommendation list. Step 313: The client launches the shopping platform corresponding to the first product information and displays the detailed content of the first product information on the shopping platform's display interface.

[0095] The information recommendation method provided in this application can be executed by an information recommendation device. This application uses an information recommendation device to perform the information recommendation method as an example to illustrate the information recommendation device provided in this application.

[0096] like Figure 4 As shown, the information recommendation device 400 provided in this application embodiment may include: The first receiving module 401 is used to receive the first input to the first application interface; The first display module 402 is used to display recommended information in response to the first input on the first application interface; The recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

[0097] This allows for direct response to input on the first application interface, displaying recommended information directly within that interface without requiring cross-application operations. This simplifies the workflow and improves the ease of accessing recommended information. Furthermore, it supports recommendations for various information types, including text, images, audio, video, and links, adapting to different input scenarios. Leveraging the semantic parsing capabilities of a multimodal large-scale model, combined with contextual information, it extracts recommendation intent descriptions from complex inputs to output corresponding recommendations. This ensures that the recommended information more closely matches the user's true intent, improving the accuracy of the recommendations.

[0098] In some embodiments, the first display module 402 may include: The acquisition unit is used to acquire recommended content information and contextual information corresponding to the first input in response to the first input. The preprocessing unit is used to preprocess the recommended content information and contextual information to obtain structured information; The generation unit is used to input structured information into the multimodal large model, and then use the multimodal large model to perform semantic intent parsing on the structured information to generate recommendation intent description information; The sending unit is used to generate a query request based on the recommendation intent description information through the multimodal large model and send it to N second applications, where N is a positive integer; The receiving unit is used to receive query results returned by N second applications; The output unit is used to generate recommendation information based on the query results using a multimodal large model; The display unit is used to display recommendation information on the first application interface.

[0099] In this way, on the one hand, preprocessing transforms raw multimodal data into structured information, reducing the computational load of large multimodal models, improving inference efficiency, and optimizing resource consumption. On the other hand, sending query requests to multiple secondary applications aggregates results from different sources, avoiding the data limitations of a single platform and improving the diversity of recommendations.

[0100] In some embodiments, when N is an integer greater than 1, the output unit may include: The processing subunit is used to process information from N query results returned by the second application through a multimodal large model to obtain M recommendation sub-information; the information processing includes at least one of aggregation processing and deduplication processing; The sorting subunit is used to sort M recommendation sub-information according to preset sorting rules using a multimodal large model to generate a recommendation list, where M is a positive integer.

[0101] This process deduplicates results from N secondary applications, avoiding information redundancy and improving user browsing efficiency. By sorting recommended sub-information based on multiple dimensions, the most relevant or optimal options are prioritized for display, making the recommendation list more aligned with users' actual needs and preferences, thus increasing click-through rates.

[0102] In some embodiments, the recommended content information indicates a product as the recommended object, and the second application is a shopping platform; The sorting subunit can also be used for: By using a multimodal large model, the weighted score value corresponding to each recommendation sub-information is determined based on K dimensions; Then, using a multimodal large model, the M recommendation sub-information are sorted according to a preset sorting rule based on the weighted score values ​​of the M recommendation sub-information to generate a product recommendation list; K is a positive integer; The K dimensions include at least one of the following: The semantic matching degree between the recommended sub-information and the recommended intent description information; The degree of matching between recommended sub-information and user profile information; Confidence level of product quality corresponding to the recommended sub-information; The product price corresponding to the recommended sub-information; The confidence level of the shopping platform corresponding to the recommended sub-information.

[0103] In this way, by using K dimensions to weight the score, we can avoid bias from a single dimension, improve the accuracy of the recommendation information, and make the recommendation list more in line with individual needs.

[0104] In some embodiments, the information recommendation device 400 may further include: The first acquisition module is used to acquire user interaction information on the recommendation list; The determination module is used to determine the feedback result corresponding to the recommendation list based on the interaction operation information; The update module is used to update the preset sorting rules based on the feedback results.

[0105] In this way, the sorting rules are adjusted in real time based on the feedback obtained from user interaction, enabling the recommendation system to continuously iterate, enhance personalization, and improve long-term recommendation performance.

[0106] In some embodiments, the recommendation information is product recommendation information, which includes at least one product information; the information recommendation device 400 may further include: The second receiving module is used to receive a second input of the first product information; the first product information is any one of the product information included in the product recommendation information; The startup module is used to launch the third application in response to the second input; the third application is the application corresponding to the first product information. The second display module is used to display detailed information about the first product on the application interface of the third application.

[0107] In this way, after clicking on a recommended product, users can be directly redirected to the source application so that they can continue with the purchase process, reducing the number of steps required.

[0108] In some embodiments, user context information includes user profile information; the information recommendation device 400 may further include: The second acquisition module is used to acquire user interaction information on the recommendation information; The update module is used to update user profile information based on interactive operation information.

[0109] In this way, user profile information is dynamically updated based on user actions related to recommended information, ensuring that the recommendation strategy is consistent with the user's latest preferences or interests and improving the accuracy of recommended information.

[0110] The information recommendation device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0111] The information recommendation device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0112] The information recommendation device provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0113] Optionally, such as Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501 and a memory 502. The memory 502 stores a program or instructions that can run on the processor 501. When the program or instructions are executed by the processor 501, they implement the various steps of the above-described information recommendation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0114] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0115] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0116] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0117] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0118] The user input unit 607 can be used to: receive a first input to the first application interface; The display unit 606 can also be used to: display recommended information in response to the first input on the first application interface; The recommendation information is generated by semantic intent parsing of the recommendation content information and contextual information corresponding to the first input through a multimodal large model, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

[0119] This allows for direct response to input on the first application interface, displaying recommended information directly within that interface without requiring cross-application operations. This simplifies the workflow and improves the ease of accessing recommended information. Furthermore, it supports recommendations for various information types, including text, images, audio, video, and links, adapting to different input scenarios. Leveraging the semantic parsing capabilities of a multimodal large-scale model, combined with contextual information, it extracts recommendation intent descriptions from complex inputs to output corresponding recommendations. This ensures that the recommended information more closely matches the user's true intent, improving the accuracy of the recommendations.

[0120] In some embodiments, the processor 610 may be used for: In response to the first input, obtain the recommended content information and contextual information corresponding to the first input; The recommended content information and contextual information are preprocessed to obtain structured information; Structured information is input into a multimodal big model, which then performs semantic intent parsing on the structured information to generate recommendation intent description information. A query request is generated based on the recommendation intent description information through a multimodal large model and sent to N second applications, where N is a positive integer; Receive query results returned by N second applications; Recommendations are generated based on query results using a multimodal large model; The display unit 606 can also be used to display recommendation information on the first application interface.

[0121] In this way, on the one hand, preprocessing transforms raw multimodal data into structured information, reducing the computational load of large multimodal models, improving inference efficiency, and optimizing resource consumption. On the other hand, sending query requests to multiple secondary applications aggregates results from different sources, avoiding the data limitations of a single platform and improving the diversity of recommendations.

[0122] In some embodiments, when N is an integer greater than 1, the processor 610 can also be used for: Using a multimodal large model, information processing is performed on the query results returned by N second applications to obtain M recommendation sub-information; information processing includes at least one of aggregation processing and deduplication processing; Using a multimodal large model, M recommendation sub-information items are sorted according to a preset sorting rule to generate a recommendation list, where M is a positive integer.

[0123] This process deduplicates results from N secondary applications, avoiding information redundancy and improving user browsing efficiency. By sorting recommended sub-information based on multiple dimensions, the most relevant or optimal options are prioritized for display, making the recommendation list more aligned with users' actual needs and preferences, thus increasing click-through rates.

[0124] In some embodiments, the recommended content information indicates a product as the recommended object, and the second application is a shopping platform; Processor 610 can be used for: By using a multimodal large model, the weighted score value corresponding to each recommendation sub-information is determined based on K dimensions; Then, using a multimodal large model, the M recommendation sub-information are sorted according to a preset sorting rule based on the weighted score values ​​of the M recommendation sub-information to generate a product recommendation list; K is a positive integer; The K dimensions include at least one of the following: The semantic matching degree between the recommended sub-information and the recommended intent description information; The degree of matching between recommended sub-information and user profile information; Confidence level of product quality corresponding to the recommended sub-information; The product price corresponding to the recommended sub-information; The confidence level of the shopping platform corresponding to the recommended sub-information.

[0125] In this way, by using K dimensions to weight the score, we can avoid bias from a single dimension, improve the accuracy of the recommendation information, and make the recommendation list more in line with individual needs.

[0126] In some embodiments, the processor 610 may be used for: Obtain user interaction information regarding the recommendation list; Based on the interactive operation information, determine the corresponding feedback results for the recommendation list; Based on the feedback, update the preset sorting rules.

[0127] In this way, the sorting rules are adjusted in real time based on the feedback obtained from user interaction, enabling the recommendation system to continuously iterate, enhance personalization, and improve long-term recommendation performance.

[0128] In some embodiments, the recommendation information is product recommendation information, which includes at least one product information; The user input unit 607 can also be used to: receive a second input on the first product information; the first product information is any one of the product information included in the product recommendation information; The processor 610 can also be used to: launch a third application in response to the second input; the third application is the application corresponding to the first product information; The display unit 606 can also be used to display detailed information about the first product in the application interface of a third application.

[0129] In this way, after clicking on a recommended product, users can be directly redirected to the source application so that they can continue with the purchase process, reducing the number of steps required.

[0130] In some embodiments, user context information includes user profile information; the processor 610 can be used for: Obtain user interaction information regarding recommended information; Update user profile information based on interactive operation information.

[0131] In this way, user profile information is dynamically updated based on user actions related to recommended information, ensuring that the recommendation strategy is consistent with the user's latest preferences or interests and improving the accuracy of recommended information.

[0132] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0133] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0134] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0135] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information recommendation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0136] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0137] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described information recommendation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0138] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0139] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described information recommendation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0140] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0142] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An information recommendation method characterized by comprising: The method comprises: receiving a first input of a first application interface; in response to the first input, displaying recommendation information on the first application interface; wherein the recommendation information is generated by a multi-modal large model performing semantic intent analysis on recommendation content information and context scenario information corresponding to the first input, generating recommendation intent description information, and outputting information based on the recommendation intent description information.

2. The method of claim 1, wherein, The response to the first input, displaying recommendation information on the first application interface, comprises: in response to the first input, obtaining recommendation content information and context scenario information corresponding to the first input; preprocessing the recommendation content information and the context scenario information to obtain structured information; inputting the structured information into the multi-modal large model, performing semantic intent analysis on the structured information by the multi-modal large model, and generating recommendation intent description information; generating a query request based on the recommendation intent description information by the multi-modal large model, and sending it to N second applications, N being a positive integer; receiving query results returned by the N second applications; generating recommendation information based on the query results by the multi-modal large model; displaying the recommendation information on the first application interface.

3. The method of claim 2, wherein, In the case where N is an integer greater than 1, the multi-modal large model generates recommendation information based on the query results, comprising: performing information processing on the query results returned by the N second applications by the multi-modal large model to obtain M recommendation sub-information; the information processing includes at least one of aggregation processing and de-duplication processing; sorting the M recommendation sub-information according to a preset sorting rule by the multi-modal large model to generate a recommendation list, M being a positive integer.

4. The method of claim 3, wherein, The recommendation object indicated by the recommendation content information is a commodity, and the second application is a shopping platform; The multi-modal large model sorts the M recommendation sub-information according to a preset sorting rule to generate a recommendation list, comprising: determining a weighted score value corresponding to each recommendation sub-information based on K dimensions by the multi-modal large model; and sorting the M recommendation sub-information according to a preset sorting rule based on the weighted score value of the M recommendation sub-information by the multi-modal large model to generate a commodity recommendation list; K is a positive integer; wherein the K dimensions include at least one of the following: semantic matching degree of the recommendation sub-information and the recommendation intent description information; matching degree of the recommendation sub-information and user portrait information; commodity quality confidence corresponding to the recommendation sub-information; commodity price corresponding to the recommendation sub-information; shopping platform confidence corresponding to the recommendation sub-information.

5. The method of claim 3, wherein, After displaying the recommendation information on the first application interface, the method further comprises: obtaining user interaction operation information of the recommendation list; determining a feedback result corresponding to the recommendation list based on the interaction operation information; updating the preset sorting rule based on the feedback result.

6. The method of claim 2, wherein, The recommendation information is commodity recommendation information, and the commodity recommendation information includes at least one commodity information; The method further includes, after displaying the recommendation information in the first application interface: receiving a second input of first commodity information; the first commodity information is any commodity information included in the commodity recommendation information; in response to the second input, starting a third application; the third application is an application corresponding to the first commodity information; displaying detailed content of the first commodity information in an application interface of the third application.

7. The method of claim 1, wherein, The context scenario information includes user portrait information; The method further includes, after displaying the recommendation information in the first application interface: obtaining interaction operation information of the user on the recommendation information; updating the user portrait information based on the interaction operation information.

8. An information recommendation device characterized by comprising: The device includes: a first receiving module configured to receive a first input of a first application interface; a first display module configured to display recommendation information in the first application interface in response to the first input; wherein the recommendation information is information output by a multi-modal large model based on semantic intent analysis of recommendation content information and context scenario information corresponding to the first input, generation of recommendation intent description information, and the recommendation intent description information.

9. An electronic device, comprising: A processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-7.

10. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-7.

Images