Interaction information display method, electronic device, and storage medium

Abstract

Embodiments of the present application disclose an interactive information display method, an electronic device and a storage medium, comprising: receiving query information input by a user; identifying a query intention corresponding to the query information according to the query feature and personalized information corresponding to the user; determining a scene disassembly corresponding to the query information, and recalling a target object corresponding to the query information and an associated object associated with the target object based on the query intention and the scene disassembly; combining the target object and the associated object according to a combination mode of the target object and the associated object to obtain a recall object information set corresponding to the query information; and generating a plurality of multimedia contents corresponding to the recall object information set through a pre-trained generation model, and displaying the plurality of multimedia contents in an information display interface. The present scheme can meet the demand of the user for a comprehensive solution to a complex problem, and improve the comprehensiveness of information display.

Description

Technical Field

[0001] This application relates to the field of Internet technology, specifically to an interactive information display method, electronic device, and storage medium. Background Technology

[0002] With the development of internet technology and the widespread use of smart devices, people are increasingly accustomed to using various search platforms to search for information, such as product information, educational resources, service resources, fashion tips, and gift recommendations. Existing search engines typically return a large number of webpage links and information, requiring users to spend a significant amount of time reading and filtering relevant content.

[0003] Furthermore, existing search engines often fail to fully understand the user's context and precise needs, resulting in search results that are independent and lack relevance. Moreover, existing search platforms rely on keywords contained in the user's query, aiming to find the most relevant single result and displaying related search results in order of relevance. However, many users' needs are contextualized and vague, difficult to express precisely with simple keywords. Users actually need a comprehensive solution, not just a few related single search results. Therefore, existing search platforms cannot meet users' needs for comprehensive solutions to complex problems. Summary of the Invention

[0004] This application provides an interactive information display method, electronic device, and storage medium, which can meet users' needs for comprehensive solutions to complex problems and improve the comprehensiveness of information display.

[0005] In a first aspect, embodiments of this application provide an interactive information display method, including:

[0006] Receive query information input by the user;

[0007] Based on the query information and the user's corresponding personalized information, identify the query intent of the query information;

[0008] Based on the scenario breakdown corresponding to the query intent and query information, the target object corresponding to the query information and the associated objects associated with the target object are recalled;

[0009] The target object and the associated object are combined according to the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information; and

[0010] Multiple multimedia contents corresponding to the recall object information set are generated by a pre-trained generative model and displayed in the information display interface.

[0011] Secondly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program as described in any of the methods above.

[0012] Thirdly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0013] In the information display scheme provided in this application, personalized information and query information are used to identify the user's query intent. Then, the scenario decomposition corresponding to the query information is determined. Based on the query intent and scenario decomposition, the target object and its associated objects are recalled. Finally, the target object and associated objects are combined using a combination method to obtain a recalled object information set. Multiple multimedia contents corresponding to the recalled object information set are generated through a pre-trained generative model and displayed on the information display interface. On the one hand, based on the query intent and scenario decomposition, the target object corresponding to the query information can be accurately searched. On the other hand, it can also extend to the associated objects related to the target object. Thus, it can meet the user's need for comprehensive solutions to complex problems and improve the comprehensiveness of information display. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating an interactive information display method provided in an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of a front-end interactive interface provided in an embodiment of this application;

[0017] Figure 3 This is another schematic diagram of the front-end interactive interface provided in the embodiments of this application;

[0018] Figure 4 This is a schematic diagram of an information display interface provided in an embodiment of this application;

[0019] Figure 5 This is another schematic diagram of the information display interface provided in the embodiments of this application;

[0020] Figure 6This is yet another schematic diagram of the information display interface provided in the embodiments of this application;

[0021] Figure 7 This is a schematic diagram of an object information interface provided in an embodiment of this application;

[0022] Figure 8 This is another flowchart illustrating an interactive information display method provided in an embodiment of this application;

[0023] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] This application provides an interactive information display method, electronic device, and storage medium, which can meet users' needs for comprehensive solutions to complex problems and improve the comprehensiveness of information display.

[0026] This interactive information display method can be applied to electronic devices, such as servers or terminals. The server can be a standalone server, a distributed server, or a server cluster consisting of multiple servers. The terminal can include mobile phones, tablets, personal computers (PCs), etc.

[0027] The interactive information display method includes: receiving query information input by a user; identifying the query intent corresponding to the query information based on the query information and the user's corresponding personalized information; determining the scene breakdown corresponding to the query information, and recalling the target object and associated objects related to the target object based on the query intent and the scene breakdown; combining the target object and associated objects according to the combination method of the target object and associated objects to obtain the recall object information set corresponding to the query information; and generating multiple multimedia contents corresponding to the recall object information set through a pre-trained generative model, and displaying the multiple multimedia contents in the information display interface.

[0028] The following combination Figures 1 to 9 The interactive information display method, electronic device, and storage medium in the embodiments of this application will be described in detail. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating an interactive information display method provided in an embodiment of this application. The specific flow of this interactive information display method can be as follows:

[0030] S101. Receive query information input by the user.

[0031] For example, a search engine provides a front-end interactive interface that supports multimodal user input, allowing users to input multimodal information, such as text, voice, images, videos, links, and other input methods. This front-end interactive interface may specifically include elements such as text search boxes, voice input controls, image upload controls, and video upload controls.

[0032] A query is a search request submitted by a user to the multimodal front-end interface of a search engine to find specific data, information, or knowledge. Query information can include keywords, phrases, questions, or anything that expresses the user's information needs. Query information can be in text form, such as keywords entered in the search box; it can be in voice form, such as a question asked through a voice assistant; it can be in image form, such as an uploaded image used for image search; or it can be a link. This article does not impose specific restrictions on the form of query information.

[0033] Please see Figure 2 , Figure 2This is a schematic diagram of the front-end interactive interface provided in this application embodiment. The front-end interactive interface includes a text search box 21, a voice input control 22, an image upload control 23, a video upload control 24, and a confirmation control 25. Users can enter query text in the text search box 21 and then click the confirmation control 25 to search for the query text. Users can also input voice by long-pressing or clicking the voice input control 22. The information display system then converts the input voice into text and displays the text corresponding to the input voice in the text search box. Users can then click the confirmation control 25 to search for the input voice. It is understood that the text corresponding to the input voice is displayed in the text search box, and users can edit the text, such as adding or deleting text. Users can also upload images by clicking the image upload control 23. The image can be captured in real-time or pre-saved locally. After receiving the image, the information display system identifies objects in the image, such as items, background, and / or image style, and then performs a search based on the identification results. Similarly, users can also upload videos by clicking the video upload control 24. These videos can be captured in real-time or pre-saved locally. After receiving the video, the information display system identifies the style of objects such as items, backgrounds, and / or images in each frame of the video, and then performs searches based on the identification results.

[0034] It is understood that, since the embodiments of this application support users to perform multimodal searches, users can enter text in the text search box and upload images or videos at the same time.

[0035] Please continue reading. Figure 2 Users can enter "white dress" in the text search box 21 and upload an image of the dress through the image upload control 23. The information display system can then perform a search based on the query text and the query image.

[0036] Furthermore, in some embodiments of this application, after a user inputs multimodal query information through a front-end interactive interface, the front-end interactive interface can be switched to an intent-assisted interface to facilitate the assisted identification of the intent of the query information.

[0037] For example, please refer to Figure 3The user inputs a query for "apple," which includes three intents: querying for a mobile phone, querying for fruit, and querying for nutritional information. Therefore, the interface can display interactive buttons 31, 32, and 33 to assist in understanding these intents. Optionally, interactive button 31 corresponds to the intent to query for a mobile phone, interactive button 32 corresponds to the intent to query for the price of fresh apples, and interactive button 33 corresponds to the intent to query for the nutritional information of apples. The user can click the corresponding interactive button according to their actual needs. After the user clicks the corresponding interactive button, the intent corresponding to the interactive option can be added to the personalized information for easy subsequent intent recognition.

[0038] S102. Obtain the query features corresponding to the query information and the personalized information corresponding to the user, and identify the query intent corresponding to the query information based on the query features and personalized information.

[0039] Query intent refers to the specific goal or purpose a user intends to achieve when submitting a query. Query characteristics are data that can be processed by a computer, transformed from the user's query information. Personalized information refers to data related to a specific user, which can be used to customize and optimize the user experience. In information retrieval, recommendation systems, and search engines, personalized information helps the system better understand user needs and preferences, thereby providing more accurate and personalized services. Personalized information can include user behavior data, user preference settings, user feedback, user social data, user habits, and user personal information, etc. Specifically, personalized information can be collected through user behavior tracking, questionnaires, user registration information, etc. Before users use a search platform, personalized information can be obtained through information surveys or interactive methods. For example, through interface queries, the user's gender, location, and areas of interest (such as sports or music) can be determined.

[0040] Taking the query information as text as an example, the word2vec algorithm can be used to convert the user's query text into query features. Then, the corresponding features are extracted from the personalized information, and the extracted features are combined with the query features. Finally, the query intent of the query information is identified based on the combination result.

[0041] In some embodiments of this application, the step of "identifying the query intent corresponding to the query information based on query features and personalized information" may specifically include:

[0042] Retrieve the query features corresponding to the query information;

[0043] Extract personalized features that are associated with user intent from personalized information;

[0044] Personalized features are concatenated into query features to obtain enhanced query features;

[0045] Based on the enhanced query features, the query intent corresponding to the query information is identified.

[0046] In this embodiment, after obtaining the query features corresponding to the query information and the user's personalized information, features that help identify the user's intent are extracted from the user's personalized information, such as the user's most frequently searched keywords and the topics of most interest. Then, the query features and personalized feature vectors are concatenated into a longer vector, namely, the enhanced query features. This enhanced vector contains the semantic information of the original query and the user's personalized information. Then, the query intent corresponding to the enhanced query features can be identified by an intent recognition model based on the Transformer architecture or a large language model.

[0047] Through this process, the system can accurately identify the user's query intent by combining the user's query content and personalized information.

[0048] It is understood that in this application, user input supports multimodal input; therefore, query information may include not only query text but also query images. That is, optionally, in some embodiments of this application, the step "obtaining query features corresponding to the query information" may specifically include:

[0049] Extract query text and query image from query information;

[0050] The query text and query image are processed using a preset processing model to obtain the query features corresponding to the query information.

[0051] To further improve the accuracy and comprehensiveness of subsequent information display, after obtaining the enhanced query features, the enhanced query features can be processed by a pre-trained intent recognition model. Then, the query intent of the query information can be determined from the intent recognition results by combining the user's real-time data.

[0052] Optionally, in some embodiments of this application, the step of "identifying the query intent corresponding to the query information based on the enhanced query features" may specifically include:

[0053] The enhanced query features are processed by a pre-trained intent recognition model to obtain the intent recognition result corresponding to the enhanced query features.

[0054] Based on real-time user data contained in personalized information, the query intent corresponding to the query information is determined from the intent recognition results.

[0055] In this embodiment, a pre-trained intent recognition model processes the enhanced query features to identify intent and outputs an intent recognition result, which can be a probability distribution representing the likelihood of different intents. Then, this is combined with real-time user data from personalized information, such as current location, time, and device type. This information provides additional context to help determine the user's intent. For example, if a user searches for "beach" in summer and is located in a coastal city, it indicates that the user is looking for nearby beaches. Therefore, the user's query intent can be further refined by combining the result of the intent recognition model with the user's personalized information. If the intent recognition model predicts that the user may be looking for information about "beach vacation," and the user's personalized information shows that they are interested in high-end resorts, then the final determined query intent might be "looking for high-end beach vacations."

[0056] For example, a user submits a query request, "I want a lightweight jacket suitable for spring." The user's profile shows a preference for fashionable casual style, frequent purchases of mid-to-high-end brands, and an interest in outdoor gear. A pre-trained NLP model (such as BERT) is used to convert the query text into text vectors. Assuming the user also uploads a reference image of the type of jacket they want, a pre-trained image recognition model converts the image into image vectors. Next, the text and image vectors are fused into an enhanced query feature. Then, a pre-trained intent recognition model (such as Transformer) processes the enhanced query feature to obtain the intent recognition result. Finally, considering the user's real-time preferences and historical behavior, it can be inferred that the user is looking for a lightweight jacket suitable for spring outdoor activities. Combining the user's brand preferences and style preferences, the recognition result is refined to "looking for a mid-to-high-end brand lightweight jacket suitable for spring outdoor activities."

[0057] S103. Determine the scenario breakdown corresponding to the query information, and based on the query intent and the scenario breakdown corresponding to the query information, recall the target object corresponding to the query information and the associated objects associated with the target object.

[0058] Scenario decomposition is the process of analyzing and deconstructing information from a user's query request. Its purpose is to extract key contextual information from the query to construct a specific use case. The target object refers to the specific information, data, or content directly related to the user's submitted query request. The target object is the result retrieved by the search engine from a database or knowledge base based on the user's query intent. Related objects refer to other information or data that are directly or indirectly related to the user's target object. Related objects expand the query results, providing users with a more comprehensive information view. For example, if the target object is a jacket, then its related objects might include jacket brands, jacket prices, styling suggestions, and washing guidelines, etc.

[0059] For example, NLP techniques, such as word segmentation, part-of-speech tagging, and syntactic analysis, are used to analyze user queries to identify keywords and entity words. For instance, in the query "new spring jackets," "spring" and "jacket" are keywords. Machine learning models, such as Support Vector Machines (SVM) and neural networks, are used to classify query intent. The user's main query purpose is determined, such as finding products, obtaining information, or comparing prices. Based on the identified query intent and the determined scenario breakdown, information directly related to the query is retrieved from the database. For the query "new spring jackets," all information on new spring jackets is retrieved. Association rules between the target object and potential related objects are defined based on business logic and domain knowledge. For jackets, related objects may include styling suggestions, user reviews, brand stories, and care guidelines. Based on the association rules, related objects related to the target object are retrieved from the database or knowledge base. For example, information on trousers, shoes, and other clothing items that can be paired with the selected jacket is retrieved. Optionally, in some embodiments of this application, the related objects can be adjusted in conjunction with the user's personalized information to provide more personalized recommendations. For example, if a user prefers a certain color or style, relevant matching suggestions will be displayed first.

[0060] Optionally, in some embodiments of this application, the step "based on query intent and scenario decomposition, recalling the target object corresponding to the query information and the associated objects associated with the target object" may specifically include:

[0061] Retrieve candidate objects corresponding to the query intent;

[0062] Based on personalized information, the scenario breakdown, and the object information of candidate objects, the target object corresponding to the query information and the associated objects associated with the target object are recalled.

[0063] For example, for the query "lightweight jacket suitable for spring," identify "spring" and "lightweight jacket" as key information. Based on the query intent, retrieve all potentially relevant candidates from the database. For "lightweight jacket," retrieve all jacket products tagged "lightweight." Analyze user personalization information, such as search history, purchase records, and preferences, to build a user profile. Based on the characteristics of the target object and the user's personalization information, identify related objects associated with the target object. For "lightweight jacket," related objects may include styling suggestions, similar product recommendations, and user reviews. Retrieve related objects associated with the target object from the database or knowledge base. For example, retrieve information on pants, shoes, and other clothing items that can be paired with the selected lightweight jacket, as well as user reviews and styling suggestions.

[0064] Optionally, in some embodiments of this application, the step "determining the scenario breakdown corresponding to the query information" may specifically include:

[0065] Identify the entity words contained in the query information;

[0066] Based on the query intent and the identified entity words, construct the scenario breakdown corresponding to the query information.

[0067] For example, NER (Network Entity Retrieval) technology can be used to extract specific entity words from query information, such as names, locations, organizations, and times. For instance, a user queries "What clothes should I wear to New York tomorrow?". The word segmentation result is: ["tomorrow", "go", "New York", "should", "wear", "what", "clothes"]; "New York" is a location entity. "New York" is identified as a location. Based on the query "What clothes should I wear to New York tomorrow?", the intent is to query the "weather". Combining the intent of "querying the weather" with the context of "New York" and "tomorrow", the scenario is broken down into "weather query before a trip".

[0068] In this way, the system can accurately understand the user's query intent and construct a scenario breakdown closely related to the user's needs, thereby providing more accurate and personalized search results.

[0069] S104. Combine the target object and related objects according to the combination method of the target object and related objects to obtain the recall object information set corresponding to the query information.

[0070] For example, a user submits a query like "help me search for a red dress," meaning the target object is "dress." Based on fashion matching rules and user preferences, related objects associated with "red dress" are identified, such as "high heels," "handbag," and "necklace." Then, based on color matching, occasion suitability, or style consistency, "red dress" can be combined with "high heels," "handbag," and "necklace" to form a recall object information set. Optionally, in some embodiments of this application, the user's contextual information can be combined, such as the user possibly looking for an outfit suitable for a specific occasion (e.g., a dinner party, a wedding), to combine the target object and related objects and obtain the recall object information set corresponding to the query information.

[0071] Specifically, the target item is a dress, and the associated items are high heels, handbags, and necklaces. The combination method is as follows: the target item "dress" is combined with the color "red," and the matching "high heels," "handbags," and "necklaces" are considered. All red dresses are retrieved from the database, and based on fashion matching rules, the high heels, handbags, and necklaces that are paired with them are selected. The retrieved dress information may include images, brand, user reviews, and / or price, and information on the matching high heels, handbags, and necklaces is also retrieved.

[0072] In some embodiments of this application, the step "combining the target object and the associated object according to the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information" may specifically include:

[0073] Obtain the first object feature corresponding to the target object and the second object feature corresponding to the associated object, respectively;

[0074] Based on personalized information and the combination of target objects and associated objects, the first object features and the second object features are combined to obtain the recall object information set corresponding to the query information.

[0075] The first set of object features consists of attributes and information directly related to the target object, while the second set consists of attributes and information directly related to associated objects. For example, the target object is a "red dress," whose features include color (red) and material (cotton). The associated objects are "high heels" and a "handbag," whose features include the style (stiletto heels) and color (black) of the high heels, and the material (leather) and capacity (medium) of the handbag. This constructs a recall object information set containing the "red dress" and its associated "high heels" and "handbag."

[0076] In some embodiments of this application, the step "combining the first object feature and the second object feature according to the personalized information and the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information" may specifically include:

[0077] Based on the combination method of the target object and the associated object, the first object feature and the second object feature are combined to obtain the feature combination set after the first object feature and the second object feature are combined;

[0078] Determine the feature combination preferences corresponding to the personalized information from the feature combination set;

[0079] Based on feature combination preferences and query intent, determine the display weight of each feature combination in the feature combination set;

[0080] Based on the display weights and feature combination sets, a recall object information set corresponding to the query information is generated.

[0081] In this embodiment, firstly, based on the logical relationship between the target object and related objects, the first object features and the second object features are combined into a feature combination set. For example, if a user queries "red dress," the first object features are the color and style of the dress, and the second object features are the color and style of the shoes and bag worn with it. Then, the user's personalized information, such as historical behavior, is analyzed to determine the user's preferences for different feature combinations. Based on the user's personalized information, the user's preferred feature combinations are identified. For example, if a user frequently views dresses paired with high heels, then high heels are a preferred feature combination. Based on the user's feature combination preferences, each feature combination in the feature combination set is assigned a display weight, representing the relative importance of that combination in the final display. It should be noted that the weights can be dynamically adjusted based on the user's latest behavior and feedback to ensure that the displayed content always aligns with the user's latest preferences. Combining the feature combination set and the display weights, a recall object information set corresponding to the query information is generated. The recall object information set will include detailed information about the target object and related objects, as well as their visual elements. Finally, the recall object information set is sorted according to the display weights, prioritizing the display of feature combinations with higher weights.

[0082] S105. Generate multiple multimedia contents corresponding to the recall object information set through a pre-trained generative model, and display the multiple multimedia contents on the information display interface.

[0083] Multimedia content refers to information content that combines different media formats, including elements such as text, images, audio, and video. Displaying multiple multimedia contents on an information display interface is a dynamic and visual way of presenting content, which can provide a rich and continuous user experience.

[0084] For example, when a user searches for "new spring clothing," the information display interface showcases selected images or videos of new spring clothing as the focus of the media stream. Specifically, a waterfall layout can be used to display clothing combinations, accessory recommendations, and styling suggestions from fashion bloggers related to "new spring clothing." Furthermore, users can click on any clothing image to view more details or swipe to browse more styling suggestions. Based on user clicks and browsing behavior, the content in the media stream is dynamically updated to display more clothing styles that users may be interested in. This satisfies users' need for comprehensive solutions to complex problems and improves the comprehensiveness of information display.

[0085] For example, such as Figure 4 As shown, a user uploads a picture of jeans and enters "how to style jeans" in the text search box. Computer vision technology is used to identify the style, color, and other features of the uploaded jeans (target object 41). Natural language processing is then performed on the user's query "how to style jeans" to extract keywords such as "style" and "jeans." Then, related items for styling jeans are recalled, including tops 42, shoes 43, watches 44, sunglasses 45, and handbags 46. The uploaded picture of jeans is integrated with the recalled multimedia content (such as images, videos, and 3D models) of styling suggestions. Finally, the integrated visual elements are arranged into a media stream to form a coherent styling suggestion.

[0086] Optionally, in some embodiments of this application, the recommended pairing suggestions can be adjusted based on the user's personalized information and historical behavior.

[0087] Optionally, in some embodiments of this application, more suitable matching suggestions can be provided by combining the user's real-time contextual information, such as the current season and fashion trends.

[0088] Optionally, in some embodiments of this application, the information display method of this application may further include:

[0089] In response to the user's element addition operation, determine the visual element corresponding to the element addition operation;

[0090] Generate a personalized element library for each user based on visual elements.

[0091] For example, a user can search for jeans, tops, and sunglasses all at once. Then, the system retrieves the visual elements corresponding to jeans, tops, and sunglasses. Next, the user selects the corresponding visual element. In other words, in response to the user's element addition operation, the system determines the visual element corresponding to the element addition operation. Finally, the user can add these visual elements to their personal collection to form a complete collection of outfits or lifestyles.

[0092] Optionally, in some embodiments of this application, the information display method of this application may further include:

[0093] Obtain the image material corresponding to the element splicing operation;

[0094] Generate personalized materials;

[0095] Personalized splicing elements are obtained by splicing personalized materials and image materials using element splicing operations.

[0096] For example, the system first receives user input (such as text descriptions or voice input) to identify the user's current mood. Then, it retrieves image materials related to the user's mood (uploaded by the user) or searches a database for image materials matching the user's description. Specifically, the user can click the stitching button on the information display interface to trigger an element stitching operation. Next, combining the user's mood and uploaded item images, personalized materials are generated that reflect the user's emotions and personality. Finally, image editing technology is used to stitch the personalized materials together with the user-uploaded item images to form a collage.

[0097] Optionally, in some embodiments of this application, the interactive information display method may further include:

[0098] In response to an embedding operation on a target object, display the embedding object corresponding to the embedding operation;

[0099] Embed the visual elements corresponding to the target object into the embedded object.

[0100] An embedded object is an object or entity created and displayed in a digital environment. For example, a user can select or upload a target object of interest, such as a piece of clothing or an accessory, through an interface and request a virtual try-on (e.g., triggered by an embedding button on the interface). A virtual mannequin (the embedded object) is then displayed; this mannequin can be a 3D model or a 2D image. Finally, image processing is used to blend the visual elements with the virtual mannequin, effectively "wearing" the visual elements onto the virtual mannequin.

[0101] Optionally, in some embodiments of this application, the step of "embedding the visual element corresponding to the target object into the embedded object" may further include:

[0102] Dynamically display the embedded object after the target object is embedded.

[0103] Specifically, the embedded scene is rendered in real time using a graphics rendering engine to ensure that users can see instant changes, such as the effect of clothes being worn on a doll in real time, including the folds of the clothes and changes in light and shadow.

[0104] Optionally, in some embodiments of this application, it may further include:

[0105] Displays comparison information between each recalled object in the recall object information set.

[0106] For example, when a user searches for a smartphone, detailed data on different smartphone models can be collected from the recall target information set. The data is analyzed and a comparison table (i.e., comparison information) is generated, listing the performance of each phone in key features.

[0107] Optionally, in some embodiments of this application, after the step of "generating multiple multimedia contents corresponding to the recall object information set through a pre-trained generative model and displaying the multiple multimedia contents in the information display interface", it may further include:

[0108] In response to content editing operations on the information display interface, it can edit multiple multimedia contents and / or generate adjustment suggestions for query information.

[0109] For example, please refer to Figure 5 Taking the display of multimedia content on the information display interface as an example, this multimedia content is a user's search for outfit suggestions, including pants 51, tops 52, and shoes 53. If the user is not satisfied with the multimedia content, they can edit it by clicking on the content editing item 54 in the information display interface. This content editing item 54 is associated with the multimedia content; that is, if the multimedia content is an outfit suggestion, then one content editing item 54 corresponds to one outfit style. Furthermore, each content editing item 54 corresponds to a content editing button 55, which the user can click to select the corresponding outfit style. In addition, the user can also input their custom query intent through the intent input box 57 to supplement the query information. The information display interface can also display the user's personalized tags 56, which are the results of the user's selection of their own personalized attributes, such as body shape, interests, gender, etc.

[0110] For example, please refer to Figure 6 For example, if a multimedia content is displayed on the information display interface, the multimedia content is a set of outfit suggestions searched by the user. In response to the content editing operation on the information display interface, such as entering how to match in the input box 61, multiple suggestion tags 62 can be generated in the preset position of the multimedia content, such as outfit suggestions for dinner, Christmas or commuting.

[0111] For example, please refer to Figure 7After clicking on the associated object 72, the user can switch the information display interface to the object information interface, which displays detailed information 77 of the associated object 72, similar results 78 of the associated object 72, and try-on information 79 of the associated object 72.

[0112] Optionally, in some embodiments of this application, please refer to Figure 8 Step S105 may further include:

[0113] S106. Generate extended results of the recall object information set, and reorganize multiple multimedia contents in the information display interface according to the relevant query intents corresponding to the extended results.

[0114] For example, image recognition content search technology and large language models can be combined to process query information and obtain extended results of the recall object information set in the network. These extended results are similar search results and / or related search results of the recall object information set. Next, the relevant query intents corresponding to the similar search results and / or related search results are determined. Based on the user's interaction with the relevant query intents, such as selecting a specific relevant intent from the determined relevant intent results, a secondary query and result redrawing are performed in combination with the user's input query information to achieve interactive and personalized result reorganization, that is, reorganizing multiple multimedia contents in the information display interface.

[0115] In the interactive information display method provided in this application, personalized information and query information are used to identify the user's query intent. Then, the scene decomposition corresponding to the query information is determined. Based on the query intent and scene decomposition, the target object and the associated objects of the target object are recalled. Finally, the target object and the associated objects are combined using the combination method of the target object and the associated objects to obtain the recalled object information set. Multiple multimedia contents corresponding to the recalled object information set are generated through a pre-trained generative model and displayed on the information display interface. On the one hand, based on the query intent and scene decomposition, the target object corresponding to the query information can be accurately searched. On the other hand, the associated objects associated with the target object can also be extended. Thus, the user's need for comprehensive solutions to complex problems can be met, and the comprehensiveness of information display is improved.

[0116] It should be noted that the method flow of this application is not limited to the specific steps and their order described and shown. Those skilled in the art can make various changes, modifications and additions, or change the order of the steps after understanding the principles of this application.

[0117] In addition, to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device.

[0118] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. The electronic device 3 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 3 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. Figure 9 As shown, the electronic device 3 includes at least one processor 301 and a memory 302. Those skilled in the art will understand that... Figure 9 The structure of the electronic device 3 shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0119] Specifically, memory 302 can be used to store computer programs or computer-executable instructions. Memory 302 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0120] Processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. Processor 301 is the control center of the electronic device 3, connecting various parts of the electronic device 3 via various interfaces and lines. By running or executing computer programs or computer-executable instructions stored in memory 302, and by calling other data stored in memory 302, it performs various functional applications and data processing of the electronic device, thereby providing overall control of the electronic device 3 and implementing the interactive information display method in the above method embodiments.

[0121] like Figure 9As shown, the electronic device 3 may further include an input unit 303 and an output unit 304. The input unit 303 can be used to receive input digital or character information and generate signal inputs related to user settings and function control, such as touch screen, keyboard, mouse, etc. The output unit 304 may include a display unit, an auxiliary lighting device (e.g., LED), and a haptic feedback device (e.g., vibration motor). The display unit may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED) display, and a plasma display. In some embodiments, the display unit may be a touch screen.

[0122] Specifically, in this embodiment, the processor 301 of the electronic device 3 reads and executes the computer program or computer-executable instructions stored in the memory 302 to implement any of the interactive information display methods provided in the above-described method embodiments of this application. For example, the processor 301 can execute the computer program or computer-executable instructions stored in the memory 302 to perform the following operations:

[0123] The system receives query information input by the user; identifies the query intent corresponding to the query information and the user's personalized information; determines the scene breakdown corresponding to the query information, and recalls the target object and associated objects related to the target object based on the query intent and the scene breakdown; combines the target object and associated objects according to the combination method of the target object and associated objects to obtain the recall object information set corresponding to the query information; and generates multiple multimedia content corresponding to the recall object information set through a pre-trained generative model, and displays the multiple multimedia content in the information display interface.

[0124] For details on the specific implementation of each of the above operations, please refer to the preceding method implementation examples, which will not be repeated here.

[0125] In the interactive information display scheme provided in this application, personalized information and query information are used to identify the user's query intent. Then, the scenario decomposition corresponding to the query information is determined. Based on the query intent and scenario decomposition, the target object and its associated objects are recalled. Finally, the target object and associated objects are combined using a combination method to obtain a recalled object information set. Multiple multimedia contents corresponding to the recalled object information set are generated through a pre-trained generative model and displayed on the information display interface. On the one hand, based on the query intent and scenario decomposition, the target object corresponding to the query information can be accurately searched. On the other hand, it can also extend to the associated objects related to the target object. Thus, it can meet the user's need for comprehensive solutions to complex problems and improve the comprehensiveness of information display.

[0126] Those skilled in the art will understand that all or part of the steps in the various methods of the above method embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0127] To this end, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program or computer-executable instructions. This computer program or computer-executable instructions can be loaded by a processor to execute the steps in the interactive information display method provided in embodiments of this application. For example, the computer program or computer-executable instructions can perform the following operations:

[0128] The system receives query information input by the user; identifies the query intent corresponding to the query information and the user's personalized information; determines the scene breakdown corresponding to the query information, and recalls the target object and associated objects related to the target object based on the query intent and the scene breakdown; combines the target object and associated objects according to the combination method of the target object and associated objects to obtain the recall object information set corresponding to the query information; and generates multiple multimedia content corresponding to the recall object information set through a pre-trained generative model, and displays the multiple multimedia content in the information display interface.

[0129] For details on the specific implementation of each of the above operations, please refer to the preceding method implementation examples, which will not be repeated here.

[0130] The non-transient computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0131] This application also provides a computer program product. When the instructions in this computer program product are executed by the processor of an electronic device, they implement any of the interactive information display methods in the above-described method embodiments. For example, the instructions in this computer program product can perform the following operations:

[0132] The system receives query information input by the user; identifies the query intent corresponding to the query information and the user's personalized information; determines the scene breakdown corresponding to the query information, and recalls the target object and associated objects related to the target object based on the query intent and the scene breakdown; combines the target object and associated objects according to the combination method of the target object and associated objects to obtain the recall object information set corresponding to the query information; and generates multiple multimedia content corresponding to the recall object information set through a pre-trained generative model, and displays the multiple multimedia content in the information display interface.

[0133] For details on the specific implementation of each of the above operations, please refer to the preceding method implementation examples, which will not be repeated here.

[0134] Since the computer program or computer executable instructions stored in the computer-readable storage medium and the instructions in the computer program product can be executed to implement the steps in the interactive information display method provided in the method embodiments of this application, the computer-readable storage medium and the computer program product can achieve the beneficial effects that any interactive information display method provided in the method embodiments of this application can achieve. For details, please refer to the previous method embodiments, which will not be repeated here.

[0135] The foregoing has provided a detailed description of an interactive information display method, electronic device, computer-readable storage medium, and computer program product provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An interactive information display method, characterized in that, include: Receive query information input by the user; Based on the query information and the user's corresponding personalized information, identify the query intent corresponding to the query information; Determine the scenario breakdown corresponding to the query information, and based on the query intent and the scenario breakdown, recall the target object corresponding to the query information and the associated objects associated with the target object; The target object and the associated object are combined according to the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information; as well as Multiple multimedia contents corresponding to the recall object information set are generated by a pre-trained generative model and displayed in the information display interface.

2. The interactive information display method according to claim 1, characterized in that, The step of identifying the query intent corresponding to the query information based on the query information and the personalized information includes: Obtain the query features corresponding to the query information; Extract personalized features associated with the user's intent from the personalized information; The personalized features are concatenated into the query features to obtain the enhanced query features; Based on the enhanced query features, the query intent corresponding to the query information is identified.

3. The interactive information display method according to claim 2, characterized in that, The step of identifying the query intent corresponding to the query information based on the enhanced query features includes: The enhanced query features are processed by a pre-trained intent recognition model to obtain the intent recognition result corresponding to the enhanced query features. Based on the real-time user data contained in the personalized information, the query intent corresponding to the query information is determined from the intent recognition result.

4. The interactive information display method according to claim 3, characterized in that, The query information includes query text and query image, and obtaining the query features corresponding to the query information includes: Extract the query text and query image from the query information; The query text and query image are processed using a preset processing model to obtain the query features corresponding to the query information.

5. The interactive information display method according to claim 1, characterized in that, The process of determining the scenario breakdown corresponding to the query information includes: Identify the entity words contained in the query information; Based on the query intent and the entity words, construct the scenario breakdown corresponding to the query information.

6. The interactive information display method according to claim 1, characterized in that, The step of recalling the target object corresponding to the query information and the associated objects associated with the target object based on the query intent and the scenario breakdown includes: Obtain the candidate object corresponding to the query intent; Based on the personalized information, the scene breakdown, and the object information of the candidate objects, the target object corresponding to the query information and the associated objects associated with the target object are recalled.

7. The interactive information display method according to claim 1, characterized in that, The step of combining the target object and the associated object according to the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information includes: The first object feature corresponding to the target object and the second object feature corresponding to the associated object are obtained respectively; Based on the personalized information and the combination method of the target object and the associated object, the first object feature and the second object feature are combined to obtain the recall object information set corresponding to the query information.

8. The interactive information display method according to claim 7, characterized in that, The step of combining the first object features and the second object features based on the personalized information and the combination method of the target object and the associated object to obtain the recall object information set corresponding to the query information includes: Based on the combination method of the target object and the associated object, the first object feature and the second object feature are combined to obtain a feature combination set after the first object feature and the second object feature are combined. Determine the feature combination preference corresponding to the personalized information from the feature combination set; Based on the feature combination preferences and query intent, determine the display weight of each feature combination in the feature combination set; Based on the display weights and feature combination set, a recall object information set corresponding to the query information is generated.

9. The interactive information display method according to claim 1, characterized in that, Also includes: In response to a user's element addition operation, determine the visual element corresponding to the element addition operation; A personalized element library corresponding to the user is generated based on the visual elements.

10. The interactive information display method according to claim 1, characterized in that, Also includes: Retrieve the multimedia materials corresponding to the element splicing operation; Generate personalized materials; Based on the element splicing operation, the personalized materials and image materials are spliced ​​together to obtain personalized spliced ​​multimedia elements.

11. The interactive information display method according to claim 1, characterized in that, Also includes: In response to an embedding operation on the target object, display the embedding object corresponding to the embedding operation; The visual elements corresponding to the target object are embedded into the embedded object.

12. The interactive information display method according to claim 11, characterized in that, After embedding the visual element corresponding to the target object into the embedded object, the method further includes: Dynamically display the embedded object containing the visual elements corresponding to the target object.

13. The interactive information display method according to claim 1, characterized in that, Also includes: Display comparison information between various multimedia contents in the recalled object information set.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the program, it implements the steps of the interactive information display method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the interactive information display method as described in any one of claims 1 to 13.

Images