Information retrieval-based answer generation method and device, equipment and medium

By aggregating multimedia information to generate thematic events and utilizing pre-trained language models, the efficiency and interactivity issues in traditional information acquisition methods are solved, achieving efficient and accurate information acquisition and enhancing user interactivity.

CN118897910APending Publication Date: 2024-11-05TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202410931006.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

Existing methods of information acquisition are inefficient and lack interactivity. Traditional search systems return original web pages that need to be viewed one by one, and the recommended information in apps lacks initiative and interactivity, resulting in low efficiency and a lack of interactivity in information acquisition.

Method used

By aggregating content-related multimedia information to generate topical events, using a pre-trained language model to generate question-and-answer prompts, and combining event indexes and query requests for efficient matching, the target answer is provided.

Benefits of technology

It enables efficient and accurate acquisition of information of interest, enhances the interactivity of information acquisition, reduces the reading of redundant information, and improves the utilization of computing resources and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897910A_ABST
    Figure CN118897910A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an answer generation method and device based on information retrieval, equipment and a medium, and is used for enabling an object to efficiently and accurately obtain interested information and enhancing interactivity. The method comprises the following steps: acquiring a query request input by an object, and acquiring each pre-constructed theme event; each theme event is obtained by aggregating at least one piece of multimedia information related to content; retrieving at least one target theme event of which the event index is matched with the query request from each theme event; each event index is obtained by performing key content extraction on at least one piece of multimedia information associated with the corresponding subject event; the event indexes of the at least one target theme event are combined with the query request, and question and answer prompt information is generated; and inputting the question and answer prompt information into a pre-trained language model, and generating a target answer corresponding to the question and answer prompt information based on the language model. And through event-level retrieval and a large model, the method is efficient, accurate and high in interactivity.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] With the rapid development of mobile Internet, there are more and more ways for objects on mobile Internet to obtain information. In practical applications, the main methods include the following:

[0003] Method 1: Search online information through the search system.

[0004] Specifically, the search system can retrieve and sort and return relevant web pages based on the keywords entered by the object.

[0005] However, when the first method is adopted, the original web page is returned to the object, and the object needs to view and summarize them one by one to extract the required recommended information. This method reduces the efficiency of obtaining recommended information and lacks interactivity.

[0006] Method 2: Develop an application (Application, APP), and the recommendation system recommends information to the object based on the object's interests.

[0007] Specifically, in an APP with a recommendation function, a feed flow is used to present recommended information to an object based on the object's historical browsing preferences.

[0008] However, when the second method is adopted, the subject can only passively receive the recommended information regardless of whether he is interested in the recommended information, and the receiving process lacks interactivity, which reduces the subject's interest in continuing to browse the recommended information and using the APP.

[0009] In summary, how to enable objects to efficiently and accurately obtain information of interest and enhance interactivity is an urgent problem to be solved. Summary of the invention

[0010] The embodiments of the present application provide a method, apparatus, device and medium for generating answers based on information retrieval, so as to enable an object to efficiently and accurately obtain information of interest and enhance interactivity.

[0011] A method for generating an answer based on information retrieval in an embodiment of the present application includes:

[0012] Obtaining a query request input by an object, and obtaining various pre-constructed subject events; wherein each subject event is obtained by aggregating at least one multimedia information related to the content;

[0013] Retrieving at least one target subject event whose event index matches the query request from the subject events; wherein each event index is obtained by extracting key content from the at least one multimedia information associated with the corresponding subject event;

[0014] Combining the event index of each of the at least one target subject event with the query request to generate question-answer prompt information;

[0015] The question and answer prompt information is input into a pre-trained language model, and a target answer corresponding to the question and answer prompt information is generated based on the language model.

[0016] An answer generation device based on information retrieval in an embodiment of the present application includes:

[0017] An acquisition unit is used to acquire a query request input by an object and acquire various pre-constructed theme events; wherein each theme event is obtained by aggregating at least one multimedia information related to the content;

[0018] A retrieval unit, configured to retrieve at least one target subject event whose event index matches the query request from the subject events; wherein each of the event indexes is obtained by extracting key content from the at least one multimedia information associated with the corresponding subject event;

[0019] A merging unit, configured to combine the event index of each of the at least one target subject event with the query request to generate question-answer prompt information;

[0020] The question-and-answer unit is used to input the question-and-answer prompt information into a pre-trained language model, and generate a target answer corresponding to the question-and-answer prompt information based on the language model.

[0021] Optionally, the device further comprises an event generating unit, which is used to perform the following process before the obtaining unit obtains the query request input by the object and obtains each pre-built subject event:

[0022] Input the multimedia information to be aggregated into the aggregation agent, and aggregate at least one multimedia information related to the content to generate a theme event through the aggregation agent; wherein the aggregation result includes: according to the similarity between the multimedia information to be aggregated and the currently existing theme event, aggregating the multimedia information to be aggregated into the currently existing theme event, or generating a new theme event;

[0023] For each theme event, the at least one multimedia information related to the theme event is input into other agents cooperating with the aggregation agent, and key contents of the at least one multimedia information are extracted by the other agents to generate an event index corresponding to the theme event.

[0024] Optionally, the event index includes event description information, event vector and event keyword; the other agents include event generation agent, vector agent and keyword agent;

[0025] Then the event generation unit is specifically used for:

[0026] Inputting the at least one multimedia information related to the theme event into an event generation agent, extracting key content of the at least one multimedia information through the event generation agent to generate event description information corresponding to the theme event;

[0027] The event description information of the theme event is input into the vector agent and the keyword agent respectively, and the event description information is embedded with the vector agent to extract features to generate the event vector of the theme event, and the keyword agent is used to extract keywords from the event description information to generate event keywords of the theme event.

[0028] Optionally, the event index includes an event vector and an event keyword; then the retrieval unit is specifically used to:

[0029] Encoding the query request to obtain a query vector; retrieving a plurality of first topic events according to the similarity between the query vector and the event vectors of the respective topic events; and,

[0030] Segmenting the query request to obtain at least one query word; retrieving a plurality of second subject events by matching the at least one query word with event keywords corresponding to each of the subject events;

[0031] The remaining theme events after deduplication of the plurality of first theme events and the plurality of second theme events are used as the target theme event.

[0032] Optionally, the retrieval unit is specifically used for:

[0033] For each of the query words, searching for candidate topic events containing the query word from the topic events according to the event keywords corresponding to the topic events;

[0034] For each candidate topic event, the relevance of the candidate topic event to the query request is determined according to the inverse document frequency of each query term and the word frequency of each query term in the candidate topic event and the query request; the inverse document frequency of each query term is determined based on the total number of topic events and the number of candidate topic events containing the query term;

[0035] According to the determined relevances, a plurality of second theme events are obtained by screening the theme events.

[0036] Optionally, the event index further includes: event elements; the event vector and event keywords are determined based on the event title and event introduction corresponding to the corresponding theme event; the event title, event introduction and event elements belong to event description information, which are obtained by extracting key content from the at least one multimedia information associated with the corresponding theme event;

[0037] The retrieval unit is also used for:

[0038] Before retrieving multiple second subject events by matching the at least one query word with the event keywords corresponding to each subject event, for each subject event, the event elements corresponding to the subject event are added as keywords to the corresponding event keywords.

[0039] Optionally, the event generating unit is specifically used for:

[0040] By means of the event generation agent, the at least one multimedia information associated with the theme event is encoded respectively to obtain a corresponding information vector; the information vector of the at least one multimedia information is decoded by different decoders to extract the event description information corresponding to the theme event;

[0041] Based on the key content of the at least one multimedia information associated with the theme event, the event generation agent is used to ask questions to the pre-trained language model to generate event description information corresponding to the theme event.

[0042] Optionally, the event generating unit is specifically used for:

[0043] Obtaining the information vector corresponding to the multimedia information to be aggregated and the event vector of the current existing subject event;

[0044] Determining the similarity between the information vector and each event vector;

[0045] If the maximum value among the determined similarities is greater than a preset similarity threshold, the multimedia information to be aggregated is aggregated into the theme event corresponding to the maximum value;

[0046] If the maximum value among the determined similarities is not greater than the preset similarity threshold, the multimedia information to be aggregated is taken as a new theme event.

[0047] Optionally, the device further comprises:

[0048] An updating unit, used for regularly counting multimedia information associated with each subject event;

[0049] For each theme event, the event index corresponding to the theme event is dynamically updated based on the statistically collected multimedia information.

[0050] Optionally, the event index includes at least one event description information; then the merging unit is specifically used to:

[0051] Respectively concatenate various event description information corresponding to each target theme event to obtain context information corresponding to the target theme event;

[0052] The query request is concatenated with the context information corresponding to each target subject event to obtain the question and answer prompt information.

[0053] Optionally, the device further comprises:

[0054] A feedback unit, configured to obtain feedback information of an object for the target answer after the question-answering unit inputs the question-answering prompt information into a pre-trained language model and generates a target answer corresponding to the question-answering prompt information based on the language model;

[0055] Based on the feedback information, adjust at least one of the following:

[0056] The answer generation strategy corresponding to the language model is adjusted, and the retrieval strategy when retrieving the target topic event from the topic events is adjusted.

[0057] Optionally, the device further comprises:

[0058] an elimination unit, configured to determine the heat value of each of the subject events before the retrieval unit retrieves at least one target subject event whose event index matches the query request from the subject events;

[0059] The theme events whose popularity values ​​are not less than the preset popularity threshold among the theme events are combined into a theme event set;

[0060] The retrieval unit is specifically used for:

[0061] At least one target subject event whose event index matches the query request is retrieved from the subject event set.

[0062] Optionally, the elimination unit is specifically used for:

[0063] By analyzing the query logs within a preset historical period, the query frequency and the latest query time of each subject event are counted;

[0064] For each theme event, based on the query frequency and the latest query time corresponding to the theme event, determine the popularity value of the theme event; wherein the popularity value of the new theme event is the preset popularity threshold;

[0065] And, the elimination unit is also used for:

[0066] The popularity value of each of the theme events is updated regularly, and the theme event set is dynamically adjusted based on the updated popularity value.

[0067] Optionally, the elimination unit is further used for:

[0068] When the number of thematic events in the thematic event set reaches a preset number, the thematic events in the thematic event set are sorted according to their popularity values, and at least one thematic event with the lowest popularity value is deleted.

[0069] An electronic device in an embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the above-mentioned information retrieval-based answer generation methods.

[0070] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the above-mentioned information retrieval-based answer generation methods.

[0071] An embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs any step of the above-mentioned information retrieval-based answer generation method.

[0072] The beneficial effects of this application are as follows:

[0073] The embodiments of the present application provide a method, device, electronic device and storage medium for generating answers based on information retrieval. Specifically, in the present application, content-related multimedia information is aggregated into the same theme event in advance, and content-unrelated multimedia information is dispersed into different theme events. By extracting key content from each multimedia information associated with the theme event, an event index corresponding to each theme event is generated in advance. The index scale at the event level is much smaller than the scale of the original multimedia information, which greatly reduces the amount of data that needs to be stored and processed, and significantly reduces the storage and computing overhead of index construction and retrieval.

[0074] On this basis, this application maps the query request of the object to the event level, and queries the target subject event that matches this request through the pre-built event index. The background information of the same subject event can be shared by multiple related queries, which can avoid the problem of repeated calculations caused by repeated queries of different objects for similar content, making the retrieval more efficient and greatly improving the utilization of computing resources. At the same time, the event-level index is naturally diverse, without the need for additional deduplication and diversity breaking up steps, simplifying the retrieval process. Finally, this application uses a large model (i.e., a pre-trained language model) to generate answers based on the search content, which can provide a summary of the key points and multi-angle analysis of the event, so that the object can quickly and accurately grasp the full picture of the event without reading a lot of redundant information, and truly realize human-computer interaction, greatly improving the object experience of traditional passive push or keyword matching retrieval. In summary, this application allows objects to efficiently and accurately obtain information of interest, enhancing interactivity.

[0075] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0077] Figure 1 A schematic diagram of an application scenario in an embodiment of the present application;

[0078] Figure 2 It is an implementation flow chart of an answer generation method based on information retrieval in an embodiment of the present application;

[0079] Figure 3 This is a schematic diagram of a multimedia information aggregation result in an embodiment of the present application;

[0080] Figure 4 A schematic diagram of an event index in an embodiment of the present application;

[0081] Figure 5 A logical schematic diagram of multi-agent collaboration to construct a topic event and event index in an embodiment of the present application;

[0082] Figure 6 A schematic diagram of a new article aggregation process in an embodiment of the present application;

[0083] Figure 7A schematic diagram of another new article aggregation process in an embodiment of the present application;

[0084] Fig. 8A This is a schematic diagram of prompts when using a large model to generate event description information in the first embodiment of the present application;

[0085] Figure 8B It is a schematic diagram of prompts when the second large model is used to generate event description information in the embodiment of the present application;

[0086] Figure 8C It is a schematic diagram of prompts when the third large model is used to generate event description information in the embodiment of the present application;

[0087] Fig. 9 A schematic diagram of an event retrieval process in an embodiment of the present application;

[0088] Fig.10 A schematic diagram of an event retrieval process in an embodiment of the present application;

[0089] Fig.11A A schematic diagram of hierarchical management of subject events in an embodiment of the present application;

[0090] Fig. 11B A schematic diagram of another subject event hierarchical management in an embodiment of the present application;

[0091] Fig.12 A schematic diagram of the interaction logic between a terminal device and a server in an embodiment of the present application;

[0092] Fig.13 A schematic diagram of the structure of an answer generation device based on information retrieval in an embodiment of the present application;

[0093] Fig.14 A schematic diagram of a hardware structure of an electronic device to which an embodiment of the present application is applied;

[0094] Fig.15 The present invention is a schematic diagram of the hardware structure of another electronic device to which the embodiments of the present application are applied. DETAILED DESCRIPTION

[0095] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0096] The following is an introduction to some concepts involved in the embodiments of the present application.

[0097] Thematic events: Aggregated from content-related multimedia information, usually centered around a central topic or hot topic, and containing multiple media forms such as text, images, and videos, aiming to comprehensively and deeply present the multi-dimensional features and dynamic development of specific events.

[0098] Prompt: This term usually refers to a technique or method to guide a model to perform a specific task. In deep learning models, especially language models based on the Transformer architecture (such as the GPT series), prompt refers to a piece of text input that is used to stimulate the model to generate a specific output or complete a specific task. The core idea of ​​the prompt method is to guide the output of the model by designing appropriate text prompts, rather than directly performing large-scale task-specific training on the model. The advantage of this method is that it can reduce the dependence on a large amount of labeled data while improving the flexibility and adaptability of the model.

[0099] Agent: Agent is an agent that receives input from an object, takes corresponding actions and then returns the results of the action. In the embodiments of this application, it refers to a model or algorithm that performs a specific task, such as:

[0100] Aggregation Agent is used to determine whether multimedia information has the same or similar content, and aggregate content-related multimedia information into theme events; Event Generation Agent is used to generate event description information such as event title, event summary, and other event elements; Vector Agent is used to generate event vectors; Keyword Agent is used to generate event keywords.

[0101] Event index: is a data structure specially designed for storing and efficiently retrieving the subject events proposed in the embodiments of this application. It organizes events according to different attributes through specific indexing technology and algorithms, so that objects can quickly locate and access the subject event information that matches the query request.

[0102] Event categories: are broad classifications of themed events, used to group different themed events into larger categories for easy archiving and retrieval. This helps manage and retrieve themed events, making them easier to organize and search. For example, sports events, natural disasters, economic trends, etc. are all common event categories.

[0103] Event tags: are a more detailed classification of the subject events, used to add additional descriptive information to the subject events, and increase the searchability and relevance of the subject events. Event tags are usually keywords or phrases that can help quickly identify specific aspects of the subject events, such as "football match", "earthquake rescue", "economic forum", etc. There can be multiple event tags, providing a multi-dimensional description of the subject event.

[0104] Event title: A title that succinctly expresses the core content of the theme event. It is a high-level summary of the theme event, usually a sentence or phrase, which aims to convey the core content or nature of the theme event concisely and clearly. A good event title should allow people to see the theme of the theme event at a glance, such as "Opening Ceremony of the 2024 Summer Games".

[0105] Event summary: It is a brief description of the subject event, providing more detailed but still concise information than the title, with the aim of allowing readers to quickly understand the key points, background, important participants or impact of the subject event. Event summary usually contains basic information such as when, where, who, and what, for example, "On June 28, 2024, the X1 location successfully held the grand opening ceremony of the 34th X2 Games, attracting hundreds of millions of viewers around the world."

[0106] Event elements: refers to the basic components or key information points that constitute the theme event, usually including but not limited to core details such as time, place, people, causes, and process.

[0107] Event vector: refers to the conversion of the multi-dimensional features of a subject event (such as time, location, participants, event type, etc.) into a high-dimensional numerical vector, which is often used in machine learning and data mining tasks. This conversion allows the subject event to be compared for similarity, clustered or classified in the vector space.

[0108] Event keywords: These are the most representative words or phrases extracted from the description of the subject event or that can summarize the core content of the event. They are the key to event information retrieval and understanding, helping the subject to quickly grasp the main focus of the subject event. Keywords are usually selected based on their frequency of occurrence in the description, uniqueness, and relevance to the event theme.

[0109] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0110] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0111] In the embodiments of the present application, the generation of information vectors, event titles, event introductions, event elements and event vectors, event keywords, query vectors, etc., the generation of target answers, etc., can be achieved through natural language processing and machine learning models.

[0112] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0113] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can enable effective communication between people and computers using natural language. Natural language processing involves natural language, which is the language people use in daily life, and is closely related to linguistic research; it also involves computer science and mathematics, and is an important technology for model training in the field of artificial intelligence. The pre-trained model is developed from the large language model (LLM) in the field of NLP. After fine-tuning, the large language model can be widely used in downstream tasks. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graphs and other technologies.

[0114] For example, in the embodiment of the present application, the title / introduction / element production model is mainly used to process natural language text, wherein the training part involves the technical field of machine learning. In the training part, the corresponding production model is trained by machine learning technology, so that the samples in the training samples pass through the corresponding production model to generate event titles / introductions / elements.

[0115] In addition, prompts can also be designed to generate corresponding event titles / introductions / elements through a pre-training model (PTM).

[0116] Specifically, PTM, also known as cornerstone model or big model, refers to a deep neural network (DNN) with large parameters. It is trained on massive unlabeled data, and uses the function approximation ability of large-parameter DNN to extract common features from the data. After fine tuning, parameter-efficient fine-tuning (PEFT), prompt fine-tuning and other technologies, it is suitable for downstream tasks. Therefore, the pre-trained model can achieve ideal results in few-shot or zero-shot scenarios. According to the data modality processed, PTM can be divided into language models, such as Embeddings from Language Models (ELMO), Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-trained Transformer (GPT), vision models, such as Swin-transformer, Vision Transformer (ViT), Vision Mixture of Experts (V-MOE), speech models, such as VALL-E, multimodal models, such as Vision and Language BERT (ViLBERT), Contrastive Language-Image Pre-training (CLIP), Foundation Models for Learning Cross-Modal Associations (Flamingo), Generalist Agent (Gato), etc., among which multimodal models refer to models that establish feature representations of two or more data modalities. Pre-trained models are important tools for outputting AI-generated content (AIGC), and can also serve as a general interface for connecting multiple specific task models.

[0117] A large language model refers to a natural language generation model that has been pre-trained on a large scale, such as GPT, Hunyuan, etc. These models are pre-trained on a large amount of corpus and have powerful language understanding and generation capabilities. In the embodiment of the present application, event titles / introductions / elements, target answers, etc. can be generated by a large language model.

[0118] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence-generated content, conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0119] The following is a brief description of the design concept in the embodiments of the present application:

[0120] With the rapid development of mobile Internet, there are more and more ways for people on mobile Internet to obtain information. In practical applications, they mainly include the above-mentioned method 1: searching for network information through search system, and method 2: developing APP, and the recommendation system recommends information to the object according to the object's interests.

[0121] Among them, the above method does not support multiple rounds of dialogue and interaction, cannot deeply explore the needs of the object, and returns the original web page, which requires the object to view and summarize one by one, and the efficiency of obtaining information is low. In addition, the current search system directly builds indexes based on the news article (can be referred to as news or articles) level, which will lead to excessively large indexes and low retrieval efficiency. For similar queries of objects, each time, the retrieval, deduplication, sorting and other steps need to be repeated, resulting in a large amount of repeated calculations.

[0122] The second method mentioned above mainly recommends to the object based on the object portrait and interest recommendation model. The traditional news recommendation system adopts the form of feeds stream. The object passively receives information and it is difficult to actively obtain information of interest, which lacks interactivity.

[0123] In addition, current news recommendation systems and search systems face the problem of a large amount of information redundancy and duplication. The same event is usually reported and forwarded by multiple sources, causing the system to store and process a large amount of duplicate content, wasting storage space and computing resources.

[0124] In view of this, the present application proposes an answer generation method, device, equipment and medium based on information retrieval. Specifically, in the present application, content-related multimedia information is aggregated into the same theme event in advance, and content-unrelated multimedia information is dispersed into different theme events. By extracting key content from each multimedia information associated with the theme event, an event index corresponding to each theme event is generated in advance. The index scale at the event level is much smaller than the scale of the original multimedia information, which greatly reduces the amount of data that needs to be stored and processed, and significantly reduces the storage and computing overhead of index construction and retrieval.

[0125] On this basis, this application maps the query request of the object to the event level, and queries the target subject event that matches this request through the pre-built event index. The background information of the same subject event can be shared by multiple related queries, which can avoid the problem of repeated calculations caused by repeated queries of different objects for similar content, making the retrieval more efficient and greatly improving the utilization of computing resources. At the same time, the event-level index is naturally diverse, without the need for additional deduplication and diversity breaking up steps, simplifying the retrieval process. Finally, this application uses a large model (i.e., a pre-trained language model) to generate answers based on the search content, which can provide a summary of the key points and multi-angle analysis of the event, so that the object can quickly and accurately grasp the full picture of the event without reading a lot of redundant information, and truly realize human-computer interaction, greatly improving the object experience of traditional passive push or keyword matching retrieval. In summary, this application allows objects to efficiently and accurately obtain information of interest, enhancing interactivity.

[0126] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0127] like Figure 1 As shown, it is a schematic diagram of an application scenario in an embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120.

[0128] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; the terminal device may be installed with a client related to information retrieval and question-answering, which may be software (such as a browser, information retrieval software, intelligent question-answering software, etc.), or a web page, a small program, etc. The server 120 is a background server corresponding to the software or web page, small program, etc., or a server specifically used to generate answers based on information retrieval, which is not specifically limited in the present application. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0129] It should be noted that the information retrieval-based answer generation method in each embodiment of the present application can be executed by an electronic device, which may be a terminal device 110 or a server 120, that is, the method can be executed by the terminal device 110 or the server 120 alone, or can be executed jointly by the terminal device 110 and the server 120.

[0130] For example, when the system composed of the terminal device 110 and the server 120 is executed together, the terminal device 110 can be installed with intelligent question-and-answer software, and the object can input its own query request through the intelligent question-and-answer software. The terminal device 110 sends the query request to the server 120, and the server 120 obtains the query request input by the object, and obtains each pre-constructed theme event; wherein each theme event is obtained by aggregating at least one multimedia information related to the content; then, the server 120 retrieves at least one target theme event whose event index matches the query request from each theme event; wherein each event index is obtained by extracting key content from at least one multimedia information associated with the corresponding theme event; then, the server 120 combines the event index of at least one target theme event with the query request to generate question-and-answer prompt information; finally, the server 120 inputs the question-and-answer prompt information into the deployed pre-trained language model, generates the target answer corresponding to the question-and-answer prompt information based on the language model, and then returns the target answer to the terminal device 110, which is displayed to the object by the terminal device 110 through the intelligent question-and-answer software.

[0131] The subject can continue to ask questions and answers based on this, or provide certain feedback on the target question and answer, such as rating, liking / disliking, or providing text feedback.

[0132] In an optional implementation, the terminal device 110 and the server 120 may communicate with each other via a communication network.

[0133] In an optional implementation, the communication network is a wired network or a wireless network.

[0134] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0135] In an embodiment of the present application, when there are multiple servers, the multiple servers can form a blockchain, and the servers are nodes on the blockchain; as disclosed in the embodiment of the present application, the answer generation method based on information retrieval, the data involved can be stored on the blockchain, for example, query requests, multimedia information, subject events, event indexes, target answers, feedback information, etc.

[0136] In addition, the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Specifically, the embodiments of the present application can be applied to various business scenarios related to information retrieval in these scenarios, including but not limited to some or all of the following:

[0137] News search, enterprise knowledge management and intelligent customer service, medical and health consulting, legal consulting, academic research support, news and content creation, and financial investment analysis.

[0138] It should be emphasized that in the specific implementation of this application, object-related data is involved, such as the query requests and feedback information listed above. When the above embodiments of this application are applied to specific products or technologies, the object's permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0139] The following describes the information retrieval-based answer generation method provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate the understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0140] See also Figure 2 As shown, it is an implementation flow chart of an answer generation method based on information retrieval in an embodiment of the present application. Taking the server as the execution subject as an example, the specific implementation process of the method is as follows S21-S24:

[0141] S21: Obtaining a query request input by an object, and obtaining pre-constructed subject events; wherein each subject event is obtained by aggregating at least one multimedia information related to the content.

[0142] In an embodiment of the present application, the object can input the query request through the intelligent question-answering software on the terminal device. After the object inputs the query request, the terminal device can send the query request to the server to obtain relevant information or solve the problem.

[0143] Specifically, if the query request contains other information besides text, the server needs to convert it into text form. For example, the object can input the query request by voice input, text input, etc. In the case of voice input by the object, the server can obtain the query request in text form through voice-to-text technology. The object can also input query requests in the form of pictures, videos, etc. These forms of query requests can be converted into text with the help of computer vision technology. For example, after extracting the frames of the video, the objects and texts in the video frames / pictures are identified, the scenes in the video frames / pictures are analyzed, and then the recognition and analysis results are summarized to be converted into text. In addition, if the video contains voice, it is converted into text using voice recognition technology. For another example, the object can also input the query request in the form of a link, then the web page content in the link can be captured, and the title, paragraph, list and other elements in the page can be extracted by parsing the HyperText Markup Language (HTML) code, and organized into an orderly data format, and the web page content can be converted into structured text, etc., which are not specifically limited in this article.

[0144] After the server obtains the query request, it needs to match the query request with the pre-constructed subject events. Therefore, before step S21, the server needs to pre-construct each subject event and establish an event index for each subject event. Specifically, the subject event in the embodiment of the present application is obtained by aggregating content-related multimedia information. In this way, a large amount of multimedia information is pre-aggregated based on whether the content is relevant, and ultimately a smaller number of subject events than the original multimedia information can be obtained. The multimedia information content associated with each subject event is the same or similar, while the multimedia information associated with different subject events is not similar.

[0145] Multimedia information refers to information content that combines multiple media forms, not just text, but also images, audio, video, animation, graphics, etc. These different types of data combined together can provide a richer, multi-dimensional object experience, making information communication more vivid and efficient. Multimedia information is widely used in education, entertainment, advertising, news reporting, social media and other fields, using digital technology and Internet platforms to enable people to receive, create and share information resources containing diverse forms of expression.

[0146] Specifically, multimedia information with the same or similar content belongs to content-related multimedia information. They are related to each other in theme, information or expression, and jointly convey or supplement a central idea or knowledge point. In this application, content-related multimedia information can be aggregated into an event, which is referred to as a theme event in this article.

[0147] After aggregating multimedia information, each subject event will be associated with one or more multimedia information. By extracting key content from the multimedia information associated with the subject event, an event index corresponding to each subject event can be generated. In this way, when subsequently retrieving relevant information of a query request, efficient and fast retrieval at the event level can be performed using the event index as a unit.

[0148] For the sake of simple example, the following uses multimedia information as an example to illustrate the various methods in the embodiments of the present application:

[0149] like Figure 3 As shown, it is a schematic diagram of a multimedia information aggregation result in an embodiment of the present application. Figure 3 12 articles are listed, represented by triangles, and recorded as: Article 1, Article 2, ..., Article 12. The triangles with the same filling pattern indicate related content. Therefore, these 12 articles can be aggregated into 3 theme events, recorded as: Theme Event 1, Theme Event 2, Theme Event 3. The theme event is represented by a five-pointed star, and its filling pattern is the same as the filling pattern of the associated triangle (article). Figure 3 In the figure, article 2, article 6 and article 8 are aggregated to generate topic event 1, article 1, article 3, article 4, article 5 and article 9 are aggregated to generate topic event 2, and article 7, article 10, article 11 and article 12 are aggregated to generate topic event 3.

[0150] In the embodiment of the present application, each event index is used to uniquely identify a subject event, so the event index contains a unique identifier corresponding to the event, recorded as event ID. In addition, the event index also includes various multimedia information associated with the corresponding subject event, such as a list composed of associated articles.

[0151] Furthermore, the event index is obtained by extracting key content from multimedia information associated with the corresponding subject event, and thus also includes one or more extracted summary information, which includes but is not limited to part or all of the following:

[0152] Event category, event label, event title, event introduction (also called summary), event elements (such as time, place, people, etc.), event vector, and event keywords.

[0153] Optionally, the event title, event introduction and event elements belong to the event description information, which are obtained by extracting key content from at least one multimedia information associated with the corresponding theme event. In addition, the event category and event tags can also be extracted in the same way; the event vector and event keywords are: determined at least based on the event title and event introduction corresponding to the corresponding theme event, and can also be generated in combination with event elements, event categories, event tags, etc., which are not specifically limited in this article.

[0154] like Figure 4 As shown, it is a schematic diagram of an event index in an embodiment of the present application. Specifically, Figure 4 Listed Figure 3 The event index corresponding to the subject event 1 in the event index is divided into three levels. The first level contains the event ID and the list of related articles (such as article 2, article 6 and article 8); the second level is the key content extracted from the related articles: event category, event label, event title, event introduction, event elements; the third level is the event vector and event keywords further generated on the basis of the second level. Among them, the event keyword can be in the form of a set, and the event keyword set contains one or more event keywords.

[0155] In an embodiment of the present application, content-related multimedia information is aggregated into theme events in advance, and an event index corresponding to each theme event is generated in advance. The index scale at the event level is much smaller than the scale of the original multimedia information, which greatly reduces the amount of data that needs to be stored and processed, and significantly reduces the storage and computing overhead of index construction and retrieval.

[0156] For example, if there are 50 news articles reporting the same topic event, the traditional method needs to index these 50 news articles separately, while this application only needs to aggregate them into one topic event and create an event index. Assuming that 1,000 subjects all ask questions related to the topic event, the traditional method needs to perform 1,000 searches to obtain 50 articles, sort the 50 articles, and break up the diversity, while this application only needs to perform 1,000 searches, and obtain one topic event each time, which greatly reduces the amount of calculation.

[0157] In addition, compared with the content generation system based on large models in related technologies, in related technologies, news web pages (i.e. multimedia information) are indexed and related news is retrieved based on object input. In this method, the content of news web pages from different sources is highly repeated, the cost of indexing and retrieval is very high, and the retrieval results need to be deduplicated and diversified, otherwise the object experience will be poor. When facing similar query requests for different objects, the steps of retrieval, dispersion (or deduplication) must be repeated each time, which has low computational efficiency.

[0158] Optionally, in the present application, multiple agents may collaborate to construct theme events and event indexes, mainly including aggregation agents, event generation agents, vector agents and keyword agents, wherein the aggregation agent is used to aggregate content-related multimedia information into theme events, the event generation agent is used to generate event description information such as event titles, event profiles and event elements, the vector agent is used to generate event vectors, and the keyword agent is used to generate event keywords.

[0159] Therefore, before step S21, each subject event and the corresponding event index may be generated in the following manner:

[0160] First, the multimedia information to be aggregated is input into the aggregation agent, and at least one multimedia information related to the content is aggregated by the aggregation agent to generate a theme event; wherein the aggregation result includes: according to the similarity between the multimedia information to be aggregated and the currently existing theme event, the multimedia information to be aggregated is aggregated into the currently existing theme event, or a new theme event is generated; the specific aggregation method is as follows, which will not be repeated here.

[0161] Next, for each theme event, at least one multimedia information related to the theme event is input to other agents cooperating with the aggregation agent, and key contents of the multimedia information can be extracted by other agents to generate an event index corresponding to the theme event.

[0162] In the embodiments of the present application, the event index may include one or more different information. Generally speaking, it may be obtained through model encoding and decoding, may be obtained by asking questions to a pre-trained language model, or may be implemented through algorithms such as term frequency-inverse document frequency (TF-IDF) and text ranking (TextRank), or topic modeling techniques. Any agent for extracting key text content or extracting embedded vectors is applicable to the embodiments of the present application, and this document does not make any specific limitations.

[0163] In the following, the event index includes event description information, event vector and event keyword as an example, which is equivalent to that other agents cooperating with the aggregation agent include: event generation agent, vector agent and keyword agent. Therefore, the event index is extracted as follows:

[0164] First, at least one multimedia information related to the theme event is input into the event generation agent, and the key content of at least one multimedia information is extracted through the event generation agent to generate event description information corresponding to the theme event; the event description information in this article includes but is not limited to: event title, event introduction, event elements, etc.

[0165] Furthermore, the event description information of the theme event is input into the vector agent and the keyword agent respectively, and the event description information is embedded and feature extracted by the vector agent to generate the event vector of the theme event, and the event description information is keyword extracted by the keyword agent to generate the event keyword of the theme event.

[0166] It should be noted that, before step S21, several subject events can be pre-constructed through the collaboration between the above-mentioned agents, and corresponding event indexes can be generated.

[0167] On this basis, as time goes by, new multimedia information will be released to the network. In this case, it is also necessary to call the aggregation agent repeatedly in real time or regularly to aggregate the new multimedia information, that is, the newly released multimedia information, that is, the multimedia information to be aggregated. If it is aggregated into a new theme event, the event generation agent, vector agent and keyword agent can be called to generate the event index of the new theme event; if it is aggregated into an existing theme event, the event generation agent, vector agent and keyword agent can be called in real time or regularly to update the event index of the existing theme event.

[0168] Combine the following Figure 5 A brief description of the collaboration logic between multiple agents:

[0169] like Figure 5 As shown, it is a logical schematic diagram of multi-agent collaboration to construct subject events and event indexes in an embodiment of the present application. The working principle of each agent is described in detail below.

[0170] (I) Aggregation Agent.

[0171] The input of the aggregation agent is new multimedia information (i.e. multimedia information that has not yet participated in the aggregation). It outputs the aggregation result by retrieving the current topic events.

[0172] An optional implementation is to aggregate multimedia information to generate a theme event in the following manner:

[0173] First, the information vector corresponding to the multimedia information to be aggregated (such as new multimedia information) and the event vector of the current subject event are obtained; and the similarity between the information vector and the event vector of each current subject event is calculated respectively.

[0174] Note that in order to measure the similarity between the query vector calculated in the retrieval stage and the event vector of each topic event, the similarity between the information vector and the event vector (that is, the multimedia information to be aggregated and the currently existing topic event) can be recorded as the first similarity, and the similarity between the query vector calculated in the retrieval stage and the event vector of each topic event (that is, the query request and the topic event) can be recorded as the second similarity.

[0175] Among them, the information vector of multimedia information is mainly determined by a series of technologies that extract features from raw data and convert them into numerical representations. Different types of multimedia data (such as text, images, audio, and video) have different feature extraction methods. For example, text information vectorization can be achieved through the Bag of Words (BoW) model, word embedding (such as doc2vec, Word2Vec, GloVe, etc.), BERT, transformer and other pre-trained model embedding, etc.; image information vectorization can be achieved through CNN feature extraction, deep learning feature descriptors (such as SIFT, SURF) or image hashing, etc.; audio information vectorization can be achieved through Mel-frequency cepstral coefficients (MFCCs), Wavelet transform, deep learning models, etc.; video information vectorization can be achieved through frame-level feature extraction, 3D CNN, etc.

[0176] When processing multimedia information containing multiple types of information, the above-mentioned technologies can be combined to generate comprehensive multimodal information vectors through cross-modal fusion technologies (such as early fusion and late fusion), such as news articles with pictures and texts, videos with audio commentary, etc., to convert multimedia information into a machine-processable numerical form, which is convenient for subsequent processing, such as aggregation tasks.

[0177] The method of determining the event vector is mainly described in the vector agent part, and will not be repeated here.

[0178] In the embodiment of the present application, there are many methods for calculating the similarity between two vectors, including but not limited to any of the following:

[0179] (1) Cosine similarity: The similarity between two vectors is evaluated by calculating the cosine value of the angle between them. The cosine value ranges from -1 to 1. The closer the value is to 1, the more similar the two vectors are. The closer the value is to 0, the more orthogonal (dissimilar) they are, while -1 means they are completely opposite. The following uses cosine similarity as an example.

[0180] (2) Euclidean distance: usually used to measure differences rather than similarity. In the present embodiment, the similarity can be indirectly expressed by taking its reciprocal or using the reciprocal of its square. The smaller the Euclidean distance, the more similar the two vectors are.

[0181] (3) Manhattan distance: Another method for measuring the difference between vectors. Similar to the Euclidean distance, in the embodiment of the present application, it can be converted into a similarity measure by taking its reciprocal.

[0182] (4) Jaccard similarity: It is mainly applied to set data. In the embodiment of the present application, it can be applied by considering the vector as a set in the feature space. It measures the ratio of the intersection size of the non-zero elements in two vectors to the union size.

[0183] (5) Pearson Correlation Coefficient: It measures the degree of linear correlation between two vectors, ranging from -1 to 1. Values ​​close to 1 indicate strong positive correlation, -1 indicates strong negative correlation, and values ​​close to 0 indicate no obvious linear relationship.

[0184] It should be noted that the above-mentioned methods of determining information vectors and calculating the similarity between vectors are just simple examples. Other methods are also applicable to the embodiments of the present application and are not listed one by one here.

[0185] After determining the similarity between the new multimedia information and the existing topic events through the vector, the maximum value among the determined similarities needs to be selected, such as the maximum similarity, and then the maximum similarity is compared with the preset similarity threshold, and the final aggregation result is determined according to the comparison result. The specific process is as follows:

[0186] If the determined maximum similarity is greater than a preset similarity threshold, the new multimedia information is aggregated into the subject event corresponding to the maximum similarity;

[0187] If the determined maximum similarity is not greater than a preset similarity threshold, the new multimedia information is regarded as a new topic event.

[0188] In the above implementation, the aggregation agent aggregates the relevant multimedia information into theme events in advance, which greatly reduces redundant information and thus greatly reduces the amount of data that needs to be stored and processed. The index scale at the event level is usually much smaller than the scale of the original news, which can significantly reduce the storage and computing overhead of index construction and retrieval. Especially in the scenario of massive news data, the advantages of reducing costs and increasing efficiency will be more obvious.

[0189] The polymerization process is briefly described below with reference to the accompanying drawings:

[0190] like Figure 6 As shown, it is a schematic diagram of a new article aggregation process in an embodiment of the present application.

[0191] Taking multimedia information as an example, the aggregation agent is responsible for aggregating similar articles into topic events to avoid duplication. The aggregation results are divided into two types: (1) new articles are aggregated into existing topic events, and (2) new articles are not aggregated -> new topic events are generated. Specifically, the aggregation agent determines whether a new article should be aggregated into an existing topic event. The workflow is as follows:

[0192] Input: New article (the content of the new article can be recorded as D i ), output: aggregation result y i ∈{0,1}, where 0 means no aggregation (i.e. the new article is treated as a new topic event) and 1 means aggregation (i.e. the new article is aggregated into the existing topic event).

[0193] Step 1: Retrieve the current topic events.

[0194] Step 2: Generate information vectors d for each new article i , the event vector {e1,e2,...,e n}.

[0195] For example, through doc2vec, or BERT, or transformer, etc., to generate the information vector d of the new article i .

[0196] Step 3: Calculate the new article d i With each existing theme event j The similarity of ij :

[0197]

[0198] Specifically, here cosine similarity is taken as an example, and other methods of calculating similarity between vectors are also applicable, such as the Euclidean distance mentioned above, etc., which will not be described here one by one.

[0199] Step 4: Find the existing topic event that is most similar to the new article j* :

[0200]

[0201] Step 5: If the maximum similarity s ij* If the similarity is greater than the preset threshold τ, then it is aggregated, otherwise it is not aggregated:

[0202]

[0203] Among them, in Step 1, if this is the first aggregation and there is no existing topic event, you can skip the intermediate step and directly determine y i = 0, that is, the new article is directly regarded as a new theme event. Then, in the subsequent aggregation, the theme event can be regarded as an existing theme event. With the multiple executions of the above workflow, more and more theme events can be aggregated.

[0204] like Figure 7 As shown, it is a schematic diagram of another new article aggregation process in an embodiment of the present application. Figure 7 is Figure 3 Based on the example, it is specifically shown that: for new article 13, it can be aggregated into topic event 1, for new article 14, it can be aggregated into topic event 3, and for new article 15, it cannot be aggregated into the existing topic event, so it is recorded as a new topic event as topic event 4.

[0205] In the embodiment of the present application, each time new multimedia information is discovered, it can be aggregated in the above manner. Therefore, for a certain theme event, as the number of aggregations increases, the number of multimedia information aggregated to the theme event can also increase continuously. Correspondingly, part or all of the content in the event index corresponding to the theme event also needs to be updated accordingly (except for the id). The update method can be specifically divided into the following two types:

[0206] Update method 1: Regular background updates.

[0207] An optional implementation is to regularly count the multimedia information associated with each theme event; after each count, for each theme event, dynamically update the event index corresponding to the theme event based on the counted multimedia information.

[0208] For example, this article does not specify the specific period of regular statistics, and can be set according to actual needs, such as 1 hour, one day, one week, etc.

[0209] Taking a week as an example, assuming that for a theme event, its currently associated multimedia information are D1 and D2, and the corresponding event index is also determined based on D1 and D2. If related multimedia information D3 and D4 are released successively within a week, they will not be updated when released, but the event index of the theme event will be re-determined based on D1, D2, D3 and D4 after one week.

[0210] In the above implementation, multimedia information associated with the subject events is regularly counted and event indexes are dynamically optimized to ensure content novelty and diversity, improve search accuracy and object experience, gain insight into object preferences, achieve personalized recommendations, efficiently utilize resources, and enhance platform attractiveness and object engagement.

[0211] Update method 2: Real-time update.

[0212] Assume that for a theme event, the currently associated multimedia information is D1 and D2, and the corresponding event index is also determined based on D1 and D2. One day later, the related multimedia information D3 is released, and the event index of the theme event is re-determined based on D1, D2, D3 and D4. Two days later, the related multimedia information D4 is released again, and the event index of the theme event is re-determined based on D1, D2, D3 and D4.

[0213] Similarly, each time a new multimedia information is released, the aggregation of the new multimedia information may be triggered periodically or in real time, which is not specifically limited here.

[0214] On the basis of the above, the event title, event introduction, event elements, etc. corresponding to the theme event can be extracted based on the key contents of each related multimedia information corresponding to the theme event. The specific process is as follows:

[0215] (2) Event generation agent.

[0216] In an embodiment of the present application, the input of the event generation agent can be new multimedia information that cannot be aggregated into existing theme events. The new multimedia information is used as a new theme event. After passing through the event generation agent, the key information of the new theme event can be refined to generate an event title, event introduction, and other event elements (such as time, place, people, etc.).

[0217] Still taking the article as an example, when the aggregation agent determines that the new article should not be aggregated, the event generation agent is responsible for generating a new topic event for it. Its workflow is as follows:

[0218] Input: New article content D i , output: event title t i , event profile i , other event elements m i .

[0219] Specifically, in the Agent, a title / introduction / element production model can be trained, such as an encoder-decoder model. The main structure of the model is an encoder and a decoder, and then the trained model is used to generate relevant event key information. The specific process is as follows:

[0220] Step 1: D i Encoded as a latent vector h i :

[0221] h i=Encoder(D i )

[0222] For example, through BERT, or transformer, etc., to generate the hidden vector h of the new article i .

[0223] It should be noted that the vector h here i The same as the d mentioned in the above aggregation agent i Both represent article D i The corresponding vectors can be the same or different, which depends on the model and will not be included here. Step 2: Generate event title t through the decoder i :

[0224] t i =Decoder t (h i )

[0225] Step 3: Generate event profile a through the decoder i :

[0226] a i =Decoder a (h i )

[0227] The decoder is used in both Step 2 and Step 3, but the decoder parameters are different and are trained separately for different tasks, which is equivalent to having a separate decoder for generating event title t, as shown in the above Decoder t , generating event profile a has a separate decoder, such as the above Decoder a .

[0228] In addition, when generating event elements m, there can also be an independent decoder, such as Decoder m , then m i =Decoder m (h i ). Alternatively, the event elements can be generated through the metadata extraction module as follows:

[0229] Step 4: Generate event element m through metadata extraction module i :

[0230] m i =MetaExtractor(D i )

[0231] Among them, the event element m iIt may include attributes such as the time, place, and people involved of the event.

[0232] For example, first, i Perform data preprocessing, including text cleaning and syntactic analysis. For example, when cleaning text, remove irrelevant characters and punctuation marks and convert them to a unified format, such as converting all to lowercase and segmenting words. When performing syntactic analysis, use technologies such as part-of-speech tagging and named entity recognition (NER) to provide structural information for subsequent steps. Then, on this basis, identify information such as time, place, and people. Specifically, regular expression matching and time expression parsers (such as SUTime, Dateutil, etc.) can be used to identify the time in the text; geographic named entity recognition technology (GNER) can be used to identify the names of geographical locations in the text, including cities, countries, landmarks, etc. Through named entity recognition technology, especially the recognition of human names, the mentioned human entities are extracted from the text. This may include public figures, historical figures, or any individuals related to the event.

[0233] In the embodiment of the present application, event elements can be used as keyword information for retrieval in the RAG step. In addition, structured information such as event elements can also be directly input into the big model to assist the big model in generating better content and making the conversation smoother.

[0234] In addition to the above-listed model extraction methods, you can also design prompts to request LLM to generate the corresponding event title, event introduction, and other event elements. There can be many kinds of prompts here, such as: given the event content xxx, generate event title, generate event introduction, generate event elements, etc. The following is a simple example:

[0235] like Fig. 8A As shown, it is a schematic diagram of prompts for the first method of using a large model to generate event description information in an embodiment of the present application. The specific prompts are: 1. What event does this article report? Please summarize it in one sentence. 2. When did this event happen? Where? What people or organizations are involved? 3. Please describe the course of the event in concise language. 4. What type of event can this event be classified as? 5. How large is the scope of impact of this event? 6. Who are the main participants in this event? 7. What caused this event to occur? 8. What is the final outcome of this event?

[0236] like Figure 8BAs shown, it is a schematic diagram of prompts for generating event description information using a large model in the second embodiment of the present application. The specific prompts are: 1. Draft an attractive news headline. 2. Write a concise news summary, including the time, place, people and process of the event. 3. Analyze the key elements of the news, such as the type of event, scope of impact, main characters, cause and result of the event.

[0237] like Figure 8C As shown, it is a schematic diagram of prompts for generating event description information using a large model in the third embodiment of the present application. The specific prompts are: 1. Generate a concise and accurate event title (no more than xx characters). 2. Summarize the event introduction, including time, place, people, and event process (no more than xxx characters). 3. List the key elements of the event, including: event type, scope of impact, main characters, event cause, and event result.

[0238] It should be noted that the above is a process of generating a corresponding new theme event for a new multimedia information based on an event generation agent. Of course, for a theme event associated with multiple multimedia information, the corresponding event title, event summary, and other event elements can also be generated in the above manner.

[0239] That is, the input of the event generation agent can also be the various multimedia information associated with the aggregated existing theme events. After the event generation agent, the key information of the existing theme events can be refined to generate event titles, event introductions, and other event elements.

[0240] For example, by encoding at least one multimedia information associated with the subject event, a corresponding information vector is obtained. In this case, each multimedia information corresponds to an h i ; Decode the information vector of at least one multimedia message through different decoders to extract the event description information corresponding to the theme event, such as the event title, event introduction and event elements, etc. Here, before decoding, the h corresponding to each multimedia message can be pre-set i Concatenate into a long vector or matrix.

[0241] In addition, it can also be generated by designing prompts, that is, based on the key content of each multimedia information associated with the theme event, questions are asked to the pre-trained language model to generate event description information corresponding to the theme event, such as event title, event introduction and event elements, etc. The specific method is the same as above and will not be repeated here.

[0242] In addition, it should be noted that as time goes by, the multimedia information associated with a theme event may continue to increase. When new multimedia information is added to the aggregation, it may not be enough to regenerate the event title, event summary, event elements, etc. At this time, it can be updated regularly through the background, such as by the background scheduled task, regularly querying all multimedia information under the theme event, and then regenerating it regularly.

[0243] In summary, the extraction of event titles, event introductions, other event elements, etc. in the embodiments of the present application can be generated by training an encoder-decoder model, or all can be generated by a large model.

[0244] Optionally, for a theme event, based on the extracted event title and event introduction (which may also be combined with event elements, etc.), an event vector and an event keyword set may be generated in the following manner:

[0245] (III) Vector Agent.

[0246] The input of the vector agent is the event description information (such as the event title t i 、Event Introduction i The output is a vector representation of the subject event, such as the event vector e i , used for subsequent vector retrieval.

[0247] Specifically, event vectors can use word embedding technology (such as Word2Vec, BERT embedding) to process text information, combined with geographic location encoding, time encoding and other methods to form a mathematical representation that can express event characteristics:

[0248] e i =Embedder(t i ,a i )

[0249] Among them, Embedder can be a pre-trained language model such as BERT, RoBERTa (Robustly OptimizedBERT Approach), llama (Leveraging LAnguage Models Amplified), etc. Specifically, pre-trained language models can provide powerful language understanding and generation capabilities. Through fine-tuning, they can be optimized for specific tasks, thereby showing excellent performance in practical applications.

[0250] (iv) Keyword: Agent.

[0251] The input of the keyword Agent is also the event description information (such as event title t i 、Event Introduction ietc., the output is the keyword of the theme event, that is, the event keyword, which can be represented as the event keyword set K i , for subsequent rapid retrieval, such as inverted index.

[0252] Specifically, the calculation formula of K i is as follows:

[0253] K i = KeywordExtractor(t i , a i )

[0254] Among them, KeywordExtractor can be implemented based on algorithms such as Term Frequency-Inverse Document Frequency (TF-IDF), TextRank, etc., or topic modeling techniques. In addition, keywords can also be generated by organizing prompt requests to large models.

[0255] Specifically, before extracting keywords based on the above algorithms, the event title and event summary can be merged into a single document, and then a tokenization tool (such as NLTK, jieba, etc.) is used to split the document into words or phrases, and common stop words (such as "de", "he", "zai", etc.) are removed. Furthermore, stemming (such as the Porter Stemming algorithm) or lemmatization (such as the WordNetLemmatizer of NLTK) is applied to normalize the lexical form, reduce lexical variants, and lay the foundation for subsequent keyword extraction.

[0256] In the embodiments of the present application, TF-IDF helps to identify words that are prominent in the current document article but not very common globally, and these words are often closely related to the unique content of the document article. TextRank further screens out more critical words in the document article structure by analyzing the interdependence and context association between words, which helps to extract a more coherent keyword set that reflects the central idea. By extracting keywords in this combined way, it is possible to maintain the close relevance of the keywords to the text content while also taking into account the semantic coherence and context importance between the keywords, thus more comprehensively and accurately summarizing the core information of the text.

[0257] In addition, it should be noted that over time, the multimedia information associated with a theme event may continue to increase. When new multimedia information is added to the aggregation, it may not be possible to regenerate event vectors, event keywords, etc. in time. At this time, it can be updated regularly through the background, such as by a timed task in the background, regularly querying all multimedia information under the theme event, and then regularly regenerating.

[0258] In the embodiment of the present application, through the collaboration of the above Agents, the construction of event index can be completed efficiently and flexibly. The present application aggregates content-related multimedia information into theme events, so that the query request of the object can be mapped to the event level, improving the retrieval efficiency. At the same time, the background information of the same theme event can be shared by multiple related queries, which can avoid the problem of repeated calculation caused by repeated queries of different objects on similar content, and greatly improve the utilization of computing resources. This can significantly reduce server pressure and response delay in large-scale online service scenarios.

[0259] On this basis, this application adopts the Retrieval-Augmented Generation (RAG) method to perform subsequent steps. The RAG method is a natural language processing technology that combines information retrieval (Retrieval) and text generation (Generation). This method aims to enhance the generation ability of the language model by retrieving relevant information from the index, thereby breaking through the knowledge bottleneck of the large model and enhancing the generation ability.

[0260] Specifically, the RAG method of the present application is divided into a retrieval phase and a generation phase, wherein the retrieval phase corresponds to S22, and the generation phase corresponds to S23 and S24, which are described below respectively:

[0261] S22: Retrieving at least one target subject event whose event index matches the query request from each subject event; wherein each event index is obtained by extracting key content from at least one multimedia information associated with the corresponding subject event.

[0262] Specifically, the event index contains one or more summary information obtained by extracting key content from multimedia information associated with the corresponding subject event. Based on this, one or more subject events matching the query request can be retrieved from the pre-constructed subject events, which are referred to as target subject events. On this basis, the target subject event and the query request can be combined to generate the target answer corresponding to the query request by asking questions to the large model.

[0263] In the retrieval stage, the most relevant k target topic events can be retrieved according to the query request q of the object and the event index of each topic event. There are many specific retrieval methods, such as vector retrieval, inverted retrieval, and a combination of vector retrieval and inverted retrieval, taking into account both efficiency and effect.

[0264] An optional implementation is that the event index includes an event vector and an event keyword, and step S22 is implemented in the following manner: Fig. 9, which is a schematic diagram of an event retrieval process in an embodiment of the present application, including the following steps S91 to S93:

[0265] S91: Encode the query request to obtain a query vector; retrieve and obtain a plurality of first theme events according to the similarity between the query vector and the event vectors of each theme event.

[0266] Specifically, step S91 is the vector search method mentioned in the embodiment of the present application, and the process of the method is as follows:

[0267] Step 1: Encode the query request q into a query vector q:

[0268] q=Embedder(q)

[0269] Specifically, Embedder can be a pre-trained language model such as BERT, RoBERTa, llama, etc., or it can be a simple word embedding model such as Word2Vec, etc. This article does not make specific limitations.

[0270] Step 2: Calculate the query vector q and each event vector e i The similarity r i :

[0271]

[0272] Specifically, here cosine similarity is taken as an example, and other methods of calculating similarity between vectors are also applicable, such as the Euclidean distance mentioned above, etc., which will not be described here one by one.

[0273] Step 3: Select the k1 topic events with the highest similarity This is the first topic event in this article.

[0274] In the embodiments of the present application, vector retrieval can significantly improve the accuracy and efficiency of information retrieval by virtue of its understanding of semantics, context adaptability, multimodal support and personalization capabilities.

[0275] S92: Segment the query request to obtain at least one query word; retrieve and obtain multiple second subject events by matching the at least one query word with event keywords corresponding to each subject event.

[0276] Specifically, step S92 is the keyword reverse search method mentioned in the embodiment of the present application. First, the query request q needs to be segmented to obtain a query word set Q. Then, each query word w in the query word set Q is matched with the event keyword corresponding to each subject event to retrieve the most matching subject events, which are recorded as second subject events.

[0277] Specifically, the event keywords mentioned in this step can be determined based on the event title and event introduction corresponding to the corresponding theme event, and can be obtained by extracting the above-mentioned keyword Agent. The extraction method is shown in the above-mentioned embodiment, and the repeated parts will not be repeated.

[0278] Alternatively, the event keywords mentioned in this step can also be determined based on the event title, event introduction and event elements corresponding to the corresponding theme event, and are specifically obtained through the above-mentioned keyword Agent extraction. The extraction method is shown in the above embodiment, and the repeated parts will not be repeated.

[0279] In addition, the above event generation agent can also extract event elements, which can also be used as another event keyword to retrieve candidate theme events in S92. That is, before S92, for each theme event, the event element corresponding to the theme event can also be used as a keyword and added to the corresponding event keyword set.

[0280] For example, for a theme event, an event keyword set can be extracted based on the event title and event introduction of the event through the keyword Agent and recorded as K1. In S92, candidate theme events can be directly retrieved based on K1.

[0281] It is also possible to extract an event keyword set based on the event title, event introduction and event elements of the event through the keyword Agent and record it as K2. In S92, candidate topic events can also be retrieved based on K2.

[0282] You can also add event elements directly as keywords to K1 to obtain K3, and then retrieve candidate topic events based on K3.

[0283] In the above implementation, incorporating time, place, person, etc. as search keywords can accurately locate information in the context of time and space, improve search relevance, meet the object's needs for regionality, timeliness, person relationships, etc., optimize personalized recommendations, and enhance situational understanding and object experience.

[0284] Regardless of which of the above situations the event keyword is, S92 can be implemented based on the following methods. The specific implementation methods of S92 are introduced below:

[0285] In the embodiment of the present application, keyword matching can be performed based on the idea of ​​TF-IDF. An optional implementation method is to implement step S92, including S921 to S923, in the following manner:

[0286] S921: For each query word, according to the event keywords corresponding to each subject event, search for candidate subject events containing the query word from each subject event.

[0287] Specifically, for each query word w in the query word set Q, according to the event keyword set corresponding to each topic event, the topic event containing w is searched in the inverted index and recorded as the candidate topic event to form the candidate set E corresponding to the query word w. w .

[0288] Then, the relevance between the candidate topic event and the query request can be calculated using the TF-IDF algorithm:

[0289] S922: For each candidate topic event, determine the relevance of the candidate topic event to the query request based on the inverse document frequency (IDF) of each query term and the word frequency of each query term in the candidate topic event and the query request; the inverse document frequency of each query term is determined based on the total number of topic events and the number of candidate topic events containing the query term.

[0290] For example, for each candidate topic event e∈E w , calculate its relevance f(e,q) to the query request q:

[0291]

[0292] Where IDF(w) is the inverse document frequency of query word w, TF(w,e) and TF(w,q) are the frequency of query word w in candidate topic event e and query request q respectively. IDF(w) = log(total number of topic events / number of candidate topic events containing query word w).

[0293] S923: Filter and obtain a plurality of second theme events from each theme event according to the determined relevance.

[0294] Specifically, select the k2 candidate topic events with the highest relevance This is the second theme event in this article.

[0295] In the embodiment of the present application, through keyword matching, the subject events containing specific words can be quickly located, which improves the efficiency and directness of information retrieval, simplifies the object query process, and is the basis for achieving accurate information acquisition. In addition, the keyword matching is specifically performed by an inverted index. Through the inverted index, even in the face of massive subject events, the keyword query can be completed in a short time, greatly improving the search efficiency.

[0296] It should be noted that the implementation order of the above steps S91 and S92 is not specifically limited in this article, and can be executed in parallel or in series. When executed in series, S91 can be executed first or S92 can be executed first. In order to improve the retrieval efficiency, parallel execution can be used for retrieval in actual applications.

[0297] S93: The remaining theme events after removing the duplication of the plurality of first theme events and the plurality of second theme events are used as the target theme event.

[0298] In step S93, the search results of steps S91 and S92 need to be merged, such as taking the union of the results of vector search and inverted search, to obtain the final search result R:

[0299] R = {e i1 ,e i2 ,...,e ik1}∪{e j1 ,e j2 ,...,e jk2}

[0300] It should be noted that k=k1+k2 topic events are usually taken as the search results (ie, the target topic events in this article). However, if there are repeated topic events in the first topic event and the second topic event, the duplicates need to be removed and the removed topic events are used as the search results.

[0301] like Fig.10 As shown, it is a schematic diagram of an event retrieval process in an embodiment of the present application. For example, there are many existing theme events, such as theme event 1, theme event 2, theme event 3, theme event 4, ..., theme event 15, theme event 16... After the object inputs the query request, the above-mentioned vector retrieval and inverted index process can be executed. For example, four first theme events are obtained through vector retrieval, namely theme event 2, theme event 8, theme event 11 and theme event 15, and four second theme events are obtained through inverted index, namely theme event 3, theme event 8, theme event 10 and theme event 15, among which theme event 8 and theme event 15 are repeatedly retrieved, so the final retrieval results are: theme event 2, theme event 3, theme event 8, theme event 10, theme event 11 and theme event 15.

[0302] It should be noted that Fig.10 This is just a simple example. In actual applications, there may be more subject events, such as hundreds, thousands, or tens of thousands. This article does not make specific limitations.

[0303] In addition to the search methods listed above, you can also search through intelligent semantic understanding, topic model matching, time and category filtering, etc., which will not be listed here one by one.

[0304] The RAG method of the present application cleverly combines vector retrieval and keyword inverted retrieval, taking into account the two dimensions of semantic similarity and keyword matching, as well as the real-time and accuracy of retrieval. The hybrid retrieval mode of vector retrieval and keyword inverted retrieval can achieve accurate semantic matching and personalized search while maintaining efficient query speed, greatly improving the efficiency and quality of complex information retrieval, and can quickly and accurately find subject events that match query requests. In addition, through the pre-built event index, the retrieval of a large amount of duplicate content is avoided, making the retrieval more efficient. At the same time, the event-level index is naturally diverse, without the need for additional deduplication and diversity breaking up steps, simplifying the retrieval process.

[0305] Furthermore, the present application may also introduce a Least Recently Used (LRU) mechanism to perform hierarchical management of subject events, placing popular subject events into a cache and the remaining subject events into regular storage. For performance considerations, when used online, most query requests only retrieve the cache to improve resource utilization efficiency and query performance.

[0306] An optional implementation is that before step S22, the popularity value of each theme event can also be determined, and the theme events whose popularity values ​​are not lower than the preset popularity threshold among the theme events are formed into a theme event set and stored in a cache, and the theme events whose popularity values ​​are lower than the preset popularity threshold among the theme events are placed in regular storage to achieve hierarchical management of theme events.

[0307] The preset heat threshold can be flexibly set according to experience, actual needs, etc. In addition, the setting of the threshold is also related to the scoring mechanism of the heat value, which is not specifically limited in this article. For example, it can be set to 10,000 or 20,000.

[0308] Thus, in step S22, at least one target subject event whose event index matches the query request can be directly retrieved from the subject event set stored in the cache. For the specific retrieval method, refer to the above embodiment, and the repeated parts will not be repeated.

[0309] like Fig.11A As shown, it is a schematic diagram of a hierarchical management of subject events in an embodiment of the present application. Fig.11A Six topic events are listed, recorded as topic events 1 to 6. Assuming that the heat values ​​of topic events 2, 3, and 5 are not lower than the preset heat threshold, they are divided into cache, and the remaining three topic events are stored conventionally. When searching based on the query request input by the object later, you can only query the cache or query the cache first, which is not specifically limited in this article.

[0310] In the above implementation, some low-heat thematic events are filtered out by sorting by heat value, so as to retrieve thematic events with high heat value, which can quickly focus on the current hot topics of the public, improve the relevance and attractiveness of the content, optimize resource allocation, increase the object participation and satisfaction, and at the same time assist in precision marketing and personalized recommendations, ultimately enhancing the platform's competitiveness and audience stickiness.

[0311] In the embodiment of the present application, considering that the popularity of the subject event will change over time and is not a fixed value, an optional implementation method is to determine and update the popularity value of the subject event in the following manner:

[0312] First, regularly analyze the query logs within a preset historical period to count the query frequency and latest query time of each theme event; then, for each theme event, determine the popularity value of the theme event based on the query frequency and latest query time corresponding to the theme event.

[0313] In this implementation, by analyzing historical query logs, the query frequencies and latest query times of different subject events are counted, and a heat value is set for each subject event by comprehensively considering the query frequency and time factors.

[0314] Specifically, the popularity value reflects the degree of attention paid to a topic event, and it is positively correlated with the query frequency, that is, the higher the query frequency, the higher the popularity value. Moreover, the popularity value is not constant, but will gradually decay over time. Even if a topic event is extremely popular at first, as time goes by, if the attention decreases, its popularity value will also decrease. This mechanism ensures that the popularity ranking can dynamically reflect the current focus of public interest.

[0315] In addition, in order to ensure that a new theme event can be retrieved in the embodiment of the present application, the heat value of the new theme event can be set to a preset heat threshold.

[0316] For example, if there are only 10,000 or 100,000 empty positions in the cache, and there are many subject events, you can sort them by heat value and put the subject events with high heat values ​​into the cache. However, this may cause problems. The heat value of new subject events (such as new articles) is 0 or very low, and they cannot enter the cache and cannot be retrieved. Therefore, it is necessary to set the initial heat value of the new subject event to a higher value to ensure that it can enter the cache. For example, set the initial heat value to the threshold that can just enter the cache, that is, the preset heat threshold in this article.

[0317] In the above implementation, the theme event heat value is set based on the comprehensive query frequency and time decay, which can effectively reflect the real-time attention and content timeliness, ensure the immediate presentation of hot spots, and avoid outdated information occupying resources; and the object behavior directly affects the heat, promotes the object's participation and platform interaction, and helps to enhance interactivity; through high-frequency queries, the object's interest is directly reflected, and the heat of old events is automatically lowered in combination with time factors, the content sorting and resource allocation are optimized, the object experience and system efficiency are improved, and content diversity is promoted; in addition, a higher initial heat value is set for new theme events, which provides more exposure opportunities for new topics, and provides data support for trend analysis and personalized recommendations, which is conducive to achieving accurate information push and enhancing platform interactivity and competitiveness.

[0318] In addition, in the embodiment of the present application, the heat value of the subject event can be updated in real time or regularly, thereby dynamically adjusting the cache content. Considering that real-time update of heat value can quickly respond to dynamic changes, it may increase the burden and complexity of the system, facing resource consumption and stability challenges; regular update reduces system pressure by reducing the update frequency, ensures stability, and although it sacrifices a certain degree of timeliness, it can better reflect long-term trends.

[0319] The following is a brief description of the example of regularly updating the popularity value of each theme event and dynamically adjusting the theme event set based on the updated popularity value:

[0320] For example, the query frequency and the latest query time of each theme event are re-counted every other day; then, for each theme event, the popularity value of the theme event is re-determined based on the query frequency and the latest query time corresponding to the theme event.

[0321] Furthermore, after the theme event heat values ​​are updated, some theme event heat values ​​may decrease, while some theme event heat values ​​may increase. Therefore, the theme event set needs to be adjusted dynamically according to this.

[0322] like Fig. 11B As shown, it is a schematic diagram of another type of subject event hierarchical management in an embodiment of the present application. Fig. 11B Indicated in Fig.11A On this basis, the 6 topic events after the heat value is updated, and a new topic event (i.e., topic event 7) appears. After the heat values ​​of these topic events change, the heat value of topic event 1 exceeds the preset heat threshold and needs to be put into the cache. As a new topic event, the corresponding initial heat value of topic event 7 is the preset heat threshold and can also be put into the cache. The heat value of topic event 2 decreases and is lower than the preset heat threshold, so it needs to be removed from the cache for regular storage.

[0323] In the above implementation, the dynamic update of the heat value can reflect the popularity and current attention of the subject event in real time, which is conducive to ensuring that the object sees the popular and timely information first, thereby increasing the stickiness of the object; in addition, the cache is automatically adjusted based on the dynamic heat value, which improves the efficiency of the system and reduces the display of uninteresting information; at the same time, the solution can effectively promote content diversification, avoid long-term monopoly of resources by hot subject events, give new subject events exposure opportunities, and maintain the vitality of the platform.

[0324] In addition, in the embodiment of the present application, the heat value of the subject event will change dynamically, so the subject event set will also be dynamically updated. In the present application, the subject event set is stored in a cache. Therefore, when the cache reaches the capacity limit, the LRU cache elimination strategy can be started to eliminate the subject event with the lowest heat value. An optional implementation is as follows:

[0325] When the number of thematic events in the thematic event set reaches a preset number, the thematic events in the thematic event set are sorted according to their heat values, and at least one thematic event with the lowest heat value is deleted.

[0326] The preset number reflects the capacity limit of the cache, which can be flexibly set based on experience or actual needs, or determined based on system resources, and is not specifically limited in this article.

[0327] Specifically, according to the heat value sorting of each subject event in the subject event set, one or more subject events with the lowest heat value are deleted, that is, the subject event that has been accessed least recently is eliminated first.

[0328] For example, the cache in the embodiment of the present application maintains an ordered data structure (usually a linked list or a combined hash table) to ensure that the most recently accessed subject event is always at the front of the linked list, while the subject event that has not been accessed for a long time gradually moves to the end of the linked list. When a new subject event needs to be added but the cache is full, the subject event at the end of the linked list is removed. Assuming that the upper limit of the cache capacity is 10,000, if 1 subject event needs to be added, then a subject event at the end of the linked list is removed, and if 2 subject events need to be added, then two subject events at the end of the linked list are removed; or, instead of removing according to demand, when the upper limit of the cache capacity reaches 10,000, the 10 subject events at the end of the linked list are removed by default, and so on.

[0329] In the above implementation, the LRU mechanism dynamically adjusts the cache content and automatically eliminates low-heat events according to the heat value, effectively balancing the relationship between storage space and access efficiency, ensuring the freshness and relevance of the content, efficiently utilizing cache resources, and improving the cache hit rate. In addition, the mechanism can also more accurately reflect the access popularity of the subject event, improve the object experience, and realize the automation and optimization of dynamic content management.

[0330] The above mainly describes the retrieval stage. In the embodiment of the present application, the task of the generation stage is to generate the target answer a of this query based on the retrieval result R and the object query request q. The specific process is as follows:

[0331] S23: Combining the event index of at least one target subject event with the query request to generate question and answer prompt information.

[0332] There are many specific combinations. Optionally, the event index includes at least one event description information; the event description information includes but is not limited to: event title, event introduction, event elements; then step S23 can be implemented in the following manner, including steps S231 to S232 ( Figure 2 not shown):

[0333] S231: Various event description information corresponding to each target theme event is spliced ​​respectively to obtain context information corresponding to the target theme event.

[0334] If the event description information of a certain subject event only includes one type, such as an event title, an event introduction, or event elements, then the context information of the subject event is the event description information.

[0335] If there are multiple event description information for a certain theme event, the multiple event description information can be spliced ​​together to obtain the context information of the theme event.

[0336] For example, the event description information includes the event title and event introduction. In this step, for each target subject event, that is, each target subject event e in the search result R i , then the event title corresponding to the event needs to be i and event profile i Spliced ​​into context information c i :

[0337] c i =t i ⊕a i ,i=1,2,...,k.

[0338] Among them, ⊕ represents the concatenation operation.

[0339] For another example, the event description information includes the event title, event introduction and event elements. In this step, for each target subject event, that is, each target subject event e in the search result R i , then the event title corresponding to the event needs to be i 、Event Introduction i and event element m i Spliced ​​into context information c i :

[0340] c i =t i ⊕a i ⊕m i ,i=1,2,...,k.

[0341] S232: Concatenate the query request with the context information corresponding to each target subject event to obtain question and answer prompt information.

[0342] After step S231, each target subject event retrieved corresponds to a context information. In step S232, the query request q and all context {c1, c2, ..., c k} are concatenated into a question-answer prompt, denoted as prompt p:

[0343] p=q⊕c1⊕c2⊕…⊕c k .

[0344] In the above implementation, the event content obtained by vector search and inverted search is organized into prompts, which may include event titles, introductions, keywords, questions (i.e., query requests) input by the object, and other information. On this basis, answers are given through a large model. This method can highly condense and intuitively display relevant information, greatly improving the object interaction experience. It not only speeds up the speed of information screening and understanding, but also promotes deeper interactive exploration through precise matching and context-rich prompts, ensures the relevance and comprehensiveness of query results, and provides objects with a more intelligent and personalized retrieval and information service experience.

[0345] On this basis, step S24 can be executed:

[0346] S24: Input the question and answer prompt information into a pre-trained language model, and generate a target answer corresponding to the question and answer prompt information based on the language model.

[0347] Specifically, the organized question-answer prompt information prompt p is input into the pre-trained language model (such as Hunyuan, GPT, etc.). The large model generates an introduction to relevant hot news based on the event background information in the prompt, which is the target answer in this article, denoted as a:

[0348] a=LanguageModel(p).

[0349] In the embodiment of the present application, the powerful natural language generation capability of the large model is used to generate high-quality answers based on the retrieved event background, provide a summary of the key points of the event, and provide a summary of the key points of the event and a multi-angle analysis, so that the subject can quickly and accurately grasp the full picture of the event without having to read a lot of redundant information. The multi-round dialogue capability allows the subject to conduct in-depth inquiries on the details of interest, truly realizing human-computer interaction, and greatly improving the subject experience of traditional passive push or keyword matching retrieval.

[0350] In addition, the present application may also introduce an object feedback mechanism to dynamically adjust the retrieval and / or generation strategy based on the object's evaluation of the target answer. An optional implementation is:

[0351] After the question and answer prompt information is input into a pre-trained language model and a target answer corresponding to the question and answer prompt information is generated based on the language model, feedback information of the subject regarding the target answer is obtained; and then at least one of the following items is adjusted based on the feedback information:

[0352] Adjust the answer generation strategy corresponding to the language model, and adjust the retrieval strategy when retrieving the target topic event from each topic event.

[0353] Specifically, the present application designs an object feedback interface to allow objects to rate, like / dislike, or provide text feedback on the target answers generated by the system, aiming to enhance the interactivity and self-optimization capabilities of the system.

[0354] Among them, this application allows the subject to score the target answer provided by the system, which can be specifically rated on a five-point scale, a ten-point scale or a star rating, etc., to intuitively reflect the subject's satisfaction. Accordingly, the subject feedback interface can receive and process the rating data submitted by the subject. This application also allows the subject to express approval or dissatisfaction with the answer by clicking the "like" or "dislike" button.

[0355] In addition to the quantitative ratings and thumbs up / downs listed above, this application can also provide a text input box for the subject to provide specific feedback or suggestions, so as to collect deeper subject needs and problem details, which is crucial for the continuous improvement of algorithms and content quality. Accordingly, the subject feedback interface needs to support the collection, storage and subsequent analysis and processing of text data.

[0356] In addition, this application also allows the subject to provide feedback on the target answer through voice feedback, visual selection feedback, etc. Among them, visual selection feedback refers to providing graphical options for the subject to choose, such as using emoticons to express satisfaction, or selecting the most appropriate one from preset answer options, simplifying the feedback process.

[0357] In addition, AI can proactively ask the subject whether the problem has been solved or request more information to optimize the answer through multiple rounds of dialogue interaction; or, by analyzing the subject's voice characteristics such as tone and speed during the interaction or emotional vocabulary in the text, automatically evaluate the subject's satisfaction with the answer and emotional response. Alternatively, by monitoring the subject's usage habits, such as repeatedly asking the same question, quickly skipping answers, frequently switching queries, etc., the subject's acceptance of the answer can be indirectly inferred, etc.

[0358] After obtaining the above feedback information of the object, the retrieval strategy and / or generation strategy may be adjusted.

[0359] The adjustment of the search strategy mainly refers to the adjustment of the search strategy when searching for the target subject event from various subject events, including but not limited to some or all of the following:

[0360] (1) Adjust the number of event recalls.

[0361] For example, if the subject frequently scrolls through the pages to find a satisfactory answer or content, it indicates that the current number of recalled content may be insufficient. The system can gradually increase the number of recalled content, such as increasing the number of results displayed from 5 to 10, to increase the probability of the subject finding a satisfactory answer.

[0362] (2) Adjust diversity rules.

[0363] Assuming that the subject frequently indicates that the content is too similar to the target answer, the system can dynamically increase the strength of the diversity rule, such as expanding from recommending closely related topics to content that is less relevant but still within the subject's interest range, to provide more diverse information.

[0364] For example, the system gradually enhances diversity rules: first, it broadens the keyword matching range to include synonyms and related concepts; then, it lowers the similarity threshold to include broader related content; next, it introduces more meta-models and data sources; finally, it implements the object interest circle expansion strategy to recommend marginal interest content until the exploration mode is activated and cross-border novel information is pushed to ensure rich and colorful content.

[0365] Adjustments to the language model generation strategy include but are not limited to some or all of the following:

[0366] (a) Adjust the cutoff length.

[0367] For example, if the subject gives feedback that the answer is too long and the key points are not highlighted, the system can shorten the text truncation length before entering the large model to ensure that the summary or answer is more refined and directly presents the core information. Conversely, if the subject wants to know more details, the truncation length can be appropriately increased to include more information.

[0368] (2) Adjust prompts.

[0369] For example, when the subject gives feedback that "the question is not understood accurately", the system needs to optimize the way of asking questions (prompt). For example, the original prompt is "explain quantum mechanics", and according to the feedback, it can be changed to a more specific or more guiding prompt, such as "briefly explain the basic principles of quantum mechanics for people who are not majoring in physics", to improve the relevance and comprehensibility of the answer.

[0370] (3) Adjust model parameters or algorithm logic.

[0371] For example, if the subject's feedback often involves insufficient knowledge in a specific field, the system can increase the weight of the data in that field, or fine-tune the model parameters to make its performance in that field more professional; or fine-tune the parameters in the model, such as the exploration factor, to increase the possibility of the model exploring different answer paths.

[0372] In the above implementation, the object feedback interface not only promotes the interaction between the object and the system, but also provides a valuable self-optimization path for the system, ultimately promoting the continuous improvement of service quality. By continuously collecting and analyzing object feedback, the system can dynamically adjust these strategies, optimize the object experience, and improve satisfaction and interaction efficiency.

[0373] See also Fig.12 As shown in FIG. 1 , it is a schematic diagram of the interaction logic between a terminal device and a server in an embodiment of the present application. The terminal device may be installed with intelligent question-and-answer software, such as a hot information assistant. The subject may input his query request through the intelligent question-and-answer software, and the terminal device sends the query request to the server. The server obtains the query request input by the subject and obtains each pre-built subject event. Then, the server retrieves the target subject event whose event index matches the query request from each subject event, such as Fig.12 It means that after searching and removing duplicates by means of vector retrieval and inverted index, 6 target topic events are obtained; then, the server combines the event index of each of the 6 target topic events with the query request to generate question and answer prompt information; finally, the server 120 inputs the question and answer prompt information into the deployed large model (that is, the language model mentioned above), generates the corresponding target answer, and then returns the target answer to the terminal device, which displays it to the object through the intelligent question and answer software.

[0374] The subject can also continue to ask questions and answers based on this, or provide certain feedback on the target question and answer, such as rating, liking / disliking, or providing text feedback.

[0375] It should be noted that the hot information assistant of this application adopts a multi-round dialogue format. Based on the search content, the large model directly generates answers to questions and summarizes the key points of events, which greatly improves the information acquisition experience of the subject.

[0376] In an embodiment of the present application, the aggregated theme events can be used as the basic unit of content operation, and the automatically generated metadata such as event titles, introductions, keywords, etc. can be directly used in operational scenarios such as content push, hot spot discovery, and trend analysis to improve operational efficiency. For example, in a content push scenario, a news client or social media platform can use these precise metadata to instantly push highly relevant news or articles to an object, increasing the object's click-through rate and engagement. For hot spot discovery, the algorithm can quickly identify and aggregate content with similar themes to form hot topics, helping the editorial team to efficiently grasp public opinion trends and plan hot spot-related reports or activities. In the field of trend analysis, automatically extracted metadata can help big data analysis. By detecting keyword frequency, topic popularity changes, etc., it can gain insight into changes in object interests and social dynamics, and provide data support for content strategy adjustments and future content planning.

[0377] In addition, object portraits and behavior analysis based on topic events can provide better support for precise object operations and personalized recommendations. This fine division makes content marketing more targeted, such as pushing content that different object groups are more likely to be interested in, thereby improving object satisfaction and retention. Behavioral analysis further reveals the object's response to specific types of content, optimizes the personalized recommendation algorithm, ensures that the content received by each object is both personalized and high-quality, and drives the growth of object participation and conversion rate.

[0378] Based on the same inventive concept, the embodiment of the present application also provides an answer generation device based on information retrieval. Fig.13 As shown, it is a schematic diagram of the structure of the answer generation device 1300 based on information retrieval, which may include:

[0379] The acquisition unit 1301 is used to acquire the query request input by the object and acquire each pre-constructed theme event; wherein each theme event is obtained by aggregating at least one multimedia information related to the content;

[0380] The retrieval unit 1302 is used to retrieve at least one target subject event whose event index matches the query request from the subject events; wherein each event index is obtained by extracting key content from the at least one multimedia information associated with the corresponding subject event;

[0381] A merging unit 1303, configured to combine the event index of each of the at least one target subject event with the query request to generate question and answer prompt information;

[0382] The question-and-answer unit 1304 is used to input the question-and-answer prompt information into a pre-trained language model, and generate a target answer corresponding to the question-and-answer prompt information based on the language model.

[0383] Optionally, the device further includes an event generating unit 1305, which is used to perform the following process before the obtaining unit 1301 obtains the query request input by the object and obtains each pre-built subject event:

[0384] Input the multimedia information to be aggregated into the aggregation agent, and aggregate at least one multimedia information related to the content to generate a theme event through the aggregation agent; wherein the aggregation result includes: according to the similarity between the multimedia information to be aggregated and the currently existing theme event, aggregating the multimedia information to be aggregated into the currently existing theme event, or generating a new theme event;

[0385] For each theme event, the at least one multimedia information related to the theme event is input into other agents cooperating with the aggregation agent, and key contents of the at least one multimedia information are extracted by the other agents to generate an event index corresponding to the theme event.

[0386] Optionally, the event index includes event description information, event vector and event keyword; the other agents include event generation agent, vector agent and keyword agent;

[0387] Then the event generating unit 1305 is specifically used for:

[0388] Inputting the at least one multimedia information related to the theme event into an event generation agent, extracting key content of the at least one multimedia information through the event generation agent to generate event description information corresponding to the theme event;

[0389] The event description information of the theme event is input into the vector agent and the keyword agent respectively, and the event description information is embedded with the vector agent to extract features to generate the event vector of the theme event, and the keyword agent is used to extract keywords from the event description information to generate event keywords of the theme event.

[0390] Optionally, the event index includes an event vector and an event keyword; then the retrieval unit 1302 is specifically used to:

[0391] Encoding the query request to obtain a query vector; retrieving a plurality of first topic events according to the similarity between the query vector and the event vectors of the respective topic events; and,

[0392] Segmenting the query request to obtain at least one query word; retrieving a plurality of second subject events by matching the at least one query word with event keywords corresponding to each of the subject events;

[0393] The remaining theme events after deduplication of the plurality of first theme events and the plurality of second theme events are used as the target theme event.

[0394] Optionally, the retrieval unit 1302 is specifically configured to:

[0395] For each of the query words, searching for candidate topic events containing the query word from the topic events according to the event keywords corresponding to the topic events;

[0396] For each candidate topic event, the relevance of the candidate topic event to the query request is determined according to the inverse document frequency of each query term and the word frequency of each query term in the candidate topic event and the query request; the inverse document frequency of each query term is determined based on the total number of topic events and the number of candidate topic events containing the query term;

[0397] According to the determined relevances, a plurality of second theme events are obtained by screening the theme events.

[0398] Optionally, the event index further includes: event elements; the event vector and event keywords are determined based on the event title and event introduction corresponding to the corresponding theme event; the event title, event introduction and event elements belong to event description information, which are obtained by extracting key content from the at least one multimedia information associated with the corresponding theme event;

[0399] The retrieval unit 1302 is also used for:

[0400] Before retrieving multiple second subject events by matching the at least one query word with the event keywords corresponding to each subject event, for each subject event, the event elements corresponding to the subject event are added as keywords to the corresponding event keywords.

[0401] Optionally, the event generating unit 1305 is specifically used for:

[0402] By means of the event generation agent, the at least one multimedia information associated with the theme event is encoded respectively to obtain a corresponding information vector; the information vector of the at least one multimedia information is decoded by different decoders to extract the event description information corresponding to the theme event;

[0403] Based on the key content of the at least one multimedia information associated with the theme event, the event generation agent is used to ask questions to the pre-trained language model to generate event description information corresponding to the theme event.

[0404] Optionally, the event generating unit 1305 is specifically used for:

[0405] Obtaining the information vector corresponding to the multimedia information to be aggregated and the event vector of the current existing subject event;

[0406] Determining the similarity between the information vector and each event vector;

[0407] If the maximum value among the determined similarities is greater than a preset similarity threshold, the new multimedia information is aggregated into the theme event corresponding to the maximum value;

[0408] If the maximum value among the determined similarities is not greater than the preset similarity threshold, the new multimedia information is regarded as a new theme event.

[0409] Optionally, the device further comprises:

[0410] An updating unit 1306, configured to periodically collect statistics on multimedia information associated with each theme event;

[0411] For each theme event, the event index corresponding to the theme event is dynamically updated based on the statistically collected multimedia information.

[0412] Optionally, the event index includes at least one event description information; then the merging unit 1303 is specifically used to:

[0413] Respectively concatenate various event description information corresponding to each target theme event to obtain context information corresponding to the target theme event;

[0414] The query request is concatenated with the context information corresponding to each target subject event to obtain the question and answer prompt information.

[0415] Optionally, the device further comprises:

[0416] A feedback unit 1307 is configured to obtain feedback information of the object for the target answer after the question and answer unit 1304 inputs the question and answer prompt information into a pre-trained language model and generates a target answer corresponding to the question and answer prompt information based on the language model;

[0417] Based on the feedback information, adjust at least one of the following:

[0418] The answer generation strategy corresponding to the language model is adjusted, and the retrieval strategy when retrieving the target topic event from the topic events is adjusted.

[0419] Optionally, the device further comprises:

[0420] An elimination unit 1308 is configured to determine the heat value of each of the subject events before the retrieval unit 1302 retrieves at least one target subject event whose event index matches the query request from the subject events;

[0421] The theme events whose popularity values ​​are not less than the preset popularity threshold among the theme events are combined into a theme event set;

[0422] The retrieval unit 1302 is specifically used for:

[0423] At least one target subject event whose event index matches the query request is retrieved from the subject event set.

[0424] Optionally, the elimination unit 1308 is specifically configured to:

[0425] By analyzing the query logs within a preset historical period, the query frequency and the latest query time of each subject event are counted;

[0426] For each theme event, based on the query frequency and the latest query time corresponding to the theme event, determine the popularity value of the theme event; wherein the popularity value of the new theme event is the preset popularity threshold;

[0427] And, the elimination unit 1308 is further used for:

[0428] The popularity value of each of the theme events is updated regularly, and the theme event set is dynamically adjusted based on the updated popularity value.

[0429] Optionally, the elimination unit 1308 is further configured to:

[0430] When the number of thematic events in the thematic event set reaches a preset number, the thematic events in the thematic event set are sorted according to their popularity values, and at least one thematic event with the lowest popularity value is deleted.

[0431] Specifically, in the present application, content-related multimedia information is pre-aggregated into the same theme event, and content-unrelated multimedia information will be dispersed into different theme events. By extracting key content from each multimedia information associated with the theme event, an event index corresponding to each theme event is pre-generated. The index scale at the event level is much smaller than the scale of the original multimedia information, which greatly reduces the amount of data that needs to be stored and processed, and significantly reduces the storage and computing overhead of index construction and retrieval.

[0432] On this basis, this application maps the query request of the object to the event level, and queries the target subject event that matches this request through the pre-built event index. The background information of the same subject event can be shared by multiple related queries, which can avoid the problem of repeated calculations caused by repeated queries of different objects for similar content, making the retrieval more efficient and greatly improving the utilization of computing resources. At the same time, the event-level index is naturally diverse, without the need for additional deduplication and diversity breaking up steps, simplifying the retrieval process. Finally, this application uses a large model (i.e., a pre-trained language model) to generate answers based on the search content, which can provide a summary of the key points and multi-angle analysis of the event, so that the object can quickly and accurately grasp the full picture of the event without reading a lot of redundant information, and truly realize human-computer interaction, greatly improving the object experience of traditional passive push or keyword matching retrieval. In summary, this application allows objects to efficiently and accurately obtain information of interest, enhancing interactivity.

[0433] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0434] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0435] After introducing the information retrieval-based answer generation method and device according to an exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0436] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0437] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Fig.14 As shown, it includes a memory 1401 , a communication module 1403 and one or more processors 1402 .

[0438] The memory 1401 is used to store computer programs executed by the processor 1402. The memory 1401 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0439] The memory 1401 may be a volatile memory, such as a random-access memory (RAM); the memory 1401 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1401 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1401 may be a combination of the above memories.

[0440] The processor 1402 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1402 is used to implement the above-mentioned information retrieval-based answer generation method when calling the computer program stored in the memory 1401 .

[0441] The communication module 1403 is used to communicate with terminal devices and other servers.

[0442] The specific connection medium between the memory 1401, the communication module 1403 and the processor 1402 is not limited in the embodiment of the present application. Fig.14 In the embodiment, the memory 1401 and the processor 1402 are connected via a bus 1404. Fig.14 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.14 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or one type of bus.

[0443] The memory 1401 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the information retrieval-based answer generation method of the embodiment of the present application. The processor 1402 is used to execute the above-mentioned information retrieval-based answer generation method, such as Figure 2 shown.

[0444] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. 1 is a terminal device 110 shown in FIG. 1 . In this embodiment, the structure of the electronic device can be as follows: Fig.15 As shown, it includes: a communication component 1510, a memory 1520, a display unit 1530, a camera 1540, a sensor 1550, an audio circuit 1560, a Bluetooth module 1570, a processor 1580 and other components.

[0445] The communication component 1510 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the user to send and receive information through the WiFi module.

[0446] The memory 1520 can be used to store software programs and data. The processor 1580 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 1520. The memory 1520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1520 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 1520 can store an operating system and various application programs, and can also store a computer program that executes the answer generation method based on information retrieval in the embodiment of the present application.

[0447] The display unit 1530 may also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1530 may include a display screen 1532 disposed on the front of the terminal device 110. The display screen 1532 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 1530 may be used to display the user interface of the client in the embodiment of the present application, etc.

[0448] The display unit 1530 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device 110. Specifically, the display unit 1530 may include a touch screen 1531 arranged on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0449] The touch screen 1531 can be covered on the display screen 1532, or the touch screen 1531 and the display screen 1532 can be integrated to realize the input and output functions of the terminal device 110, and the integrated display screen can be referred to as a touch display screen. In this application, the display unit 1530 can display the application program and the corresponding operation steps.

[0450] The camera 1540 can be used to capture still images, and the user can publish the images taken by the camera 1540 through the application. The camera 1540 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the processor 1580 to convert it into a digital image signal.

[0451] The terminal device may further include at least one sensor 1550, such as an acceleration sensor 1551, a distance sensor 1552, a fingerprint sensor 1553, and a temperature sensor 1554. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0452] The audio circuit 1560, the speaker 1561, and the microphone 1562 can provide an audio interface between the user and the terminal device 110. The audio circuit 1560 can transmit the electrical signal converted from the received audio data to the speaker 1561, which is converted into a sound signal for output. The terminal device 110 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1560 and converted into audio data, and then the audio data is output to the communication component 1510 to be sent to, for example, another terminal device 110, or the audio data is output to the memory 1520 for further processing.

[0453] The Bluetooth module 1570 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1570 to exchange data.

[0454] The processor 1580 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1520 and calling data stored in the memory 1520. In some embodiments, the processor 1580 may include one or more processing units; the processor 1580 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, the user interface and the application program, and the baseband processor mainly processes wireless communication. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1580. In the present application, the processor 1580 can run the operating system, the application program, the user interface display and the touch response, as well as the answer generation method based on information retrieval in the embodiment of the present application. In addition, the processor 1580 is coupled to the display unit 1530.

[0455] In some possible implementations, various aspects of the information retrieval-based answer generation method provided in the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the information retrieval-based answer generation method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device may execute the following steps: Figure 2 Follow the steps shown in .

[0456] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0457] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.

[0458] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0459] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0460] The computer program for performing the operation of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program can be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user electronic device through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0461] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0462] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0463] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0464] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0465] These computer program commands may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the commands stored in the computer readable memory produce an article of manufacture comprising a command device, the command device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0466] These computer program commands can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the commands executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0467] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0468] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating an answer based on information retrieval, characterized in that: The method comprises: Obtaining a query request input by an object, and obtaining various pre-constructed subject events; wherein each subject event is obtained by aggregating at least one multimedia information related to the content; Retrieving at least one target subject event whose event index matches the query request from the subject events; wherein each event index is obtained by extracting key content from the at least one multimedia information associated with the corresponding subject event; Combining the event index of each of the at least one target subject event with the query request to generate question-answer prompt information; The question and answer prompt information is input into a pre-trained language model, and a target answer corresponding to the question and answer prompt information is generated based on the language model.

2. The method according to claim 1, characterized in that Before obtaining the query request input by the object and obtaining each pre-built subject event, it also includes: Input the multimedia information to be aggregated into the aggregation agent, and aggregate at least one multimedia information related to the content to generate a theme event through the aggregation agent; wherein the aggregation result includes: according to the similarity between the multimedia information to be aggregated and the currently existing theme event, aggregating the multimedia information to be aggregated into the currently existing theme event, or generating a new theme event; For each theme event, the at least one multimedia information related to the theme event is input into other agents cooperating with the aggregation agent, and key contents of the at least one multimedia information are extracted by the other agents to generate an event index corresponding to the theme event.

3. The method according to claim 2, characterized in that The event index includes event description information, event vector and event keywords; the other agents include event generation agent, vector agent and keyword agent; Then, inputting the at least one multimedia information related to the theme event into other agents cooperating with the aggregation agent, and extracting key content of the at least one multimedia information through the other agents to generate an event index corresponding to the theme event, including: Inputting the at least one multimedia information related to the theme event into an event generation agent, extracting key content of the at least one multimedia information through the event generation agent to generate event description information corresponding to the theme event; The event description information of the theme event is input into the vector agent and the keyword agent respectively, and the event description information is embedded with the vector agent to extract features to generate the event vector of the theme event, and the keyword agent is used to extract keywords from the event description information to generate event keywords of the theme event.

4. The method according to claim 1, characterized in that The event index includes an event vector and an event keyword; Then, retrieving at least one target subject event whose event index matches the query request from the subject events includes: Encoding the query request to obtain a query vector; retrieving a plurality of first topic events according to the similarity between the query vector and the event vectors of the respective topic events; and, Segmenting the query request to obtain at least one query word; retrieving a plurality of second subject events by matching the at least one query word with event keywords corresponding to each of the subject events; The remaining theme events after deduplication of the plurality of first theme events and the plurality of second theme events are used as the target theme event.

5. The method according to claim 4, characterized in that The retrieving a plurality of second subject events by matching the at least one query word with the event keywords corresponding to each of the subject events includes: For each of the query words, searching for candidate topic events containing the query word from the topic events according to the event keywords corresponding to the topic events; For each candidate topic event, the relevance of the candidate topic event to the query request is determined according to the inverse document frequency of each query term and the word frequency of each query term in the candidate topic event and the query request; the inverse document frequency of each query term is determined based on the total number of topic events and the number of candidate topic events containing the query term; According to the determined relevances, a plurality of second theme events are obtained by screening the theme events.

6. The method according to claim 4, characterized in that The event index also includes: event elements; the event vector and event keywords are determined based on the event title and event introduction corresponding to the corresponding theme event; the event title, event introduction and event elements belong to event description information, which are obtained by extracting key content from the at least one multimedia information associated with the corresponding theme event; Before retrieving a plurality of second subject events by matching the at least one query word with the event keywords corresponding to each of the subject events, the method further includes: For each theme event, the event elements corresponding to the theme event are used as keywords and added to the corresponding event keywords.

7. The method according to claim 3, characterized in that The extracting key content of the at least one multimedia information by the event generation Agent to generate event description information corresponding to the subject event includes: By means of the event generation agent, the at least one multimedia information associated with the theme event is encoded respectively to obtain a corresponding information vector; the information vector of the at least one multimedia information is decoded by different decoders to extract the event description information corresponding to the theme event; Based on the key content of the at least one multimedia information associated with the theme event, the event generation agent is used to ask questions to the pre-trained language model to generate event description information corresponding to the theme event.

8. The method according to claim 1, characterized in that The method further comprises: Regularly count the multimedia information associated with each theme event; For each theme event, the event index corresponding to the theme event is dynamically updated based on the statistically collected multimedia information.

9. The method according to claim 1, characterized in that The event index includes at least one event description information; Then, combining the event index of each of the at least one target subject event with the query request to generate question-answer prompt information includes: Respectively concatenate various event description information corresponding to each target theme event to obtain context information corresponding to the target theme event; The query request is concatenated with the context information corresponding to each target subject event to obtain the question and answer prompt information.

10. The method according to any one of claims 1 to 9, characterized in that: After inputting the question and answer prompt information into a pre-trained language model and generating a target answer corresponding to the question and answer prompt information based on the language model, the method further includes: Obtaining feedback information from the subject regarding the target answer; Based on the feedback information, adjust at least one of the following: The answer generation strategy corresponding to the language model is adjusted, and the retrieval strategy when retrieving the target topic event from the topic events is adjusted.

11. The method according to any one of claims 1 to 9, characterized in that: Before retrieving at least one target subject event whose event index matches the query request from the subject events, the method further includes: Determine the heat value of each of the subject events; The theme events whose popularity values ​​are not less than the preset popularity threshold among the theme events are combined into a theme event set; The retrieving at least one target subject event whose event index matches the query request from the subject events includes: At least one target subject event whose event index matches the query request is retrieved from the subject event set.

12. The method according to claim 11, characterized in that Determining the heat value of each of the subject events includes: By analyzing the query logs within a preset historical period, the query frequency and the latest query time of each subject event are counted; For each theme event, based on the query frequency and the latest query time corresponding to the theme event, determine the popularity value of the theme event; wherein the popularity value of the new theme event is the preset popularity threshold; And, the method further comprises: The popularity value of each of the theme events is updated regularly, and the theme event set is dynamically adjusted based on the updated popularity value.

13. The method according to claim 11, characterized in that The method further comprises: When the number of thematic events in the thematic event set reaches a preset number, the thematic events in the thematic event set are sorted according to their popularity values, and at least one thematic event with the lowest popularity value is deleted.

14. An answer generation device based on information retrieval, characterized in that: include: An acquisition unit is used to acquire a query request input by an object and acquire various pre-constructed theme events; wherein each theme event is obtained by aggregating at least one multimedia information related to the content; A retrieval unit, configured to retrieve at least one target subject event whose event index matches the query request from the subject events; wherein each of the event indexes is obtained by extracting key content from the at least one multimedia information associated with the corresponding subject event; A merging unit, configured to combine the event index of each of the at least one target subject event with the query request to generate question-answer prompt information; The question-and-answer unit is used to input the question-and-answer prompt information into a pre-trained language model, and generate a target answer corresponding to the question-and-answer prompt information based on the language model.

15. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The method comprises a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the methods of claims 1 to 13.

17. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any one of the methods described in claims 1 to 13.

Citation Information

Cited By

  • Dialogue model training method and device, equipment and medium

    CN119578558A

  • Document retrieval control method and system and server

    CN121327210A

  • Document retrieval control methods, systems and servers

    CN121327210B

  • Standardized data management system based on RAG and large model warehouse

    CN121615792A

  • A standardized data governance system based on RAG and large model warehouse

    CN121615792B