Mining model training method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to the technical field of artificial intelligence, and provides a mining model training method and device, electronic equipment and a storage medium which are used for improving venation mining efficiency and quality. The method comprises the following steps: training an initial mining model based on a first sample set to obtain an intermediate mining model; the model corresponds to a plurality of tasks, and each task represents a sub-process of the event venation mining process; the first sample set comprises a first sample and a sample label associated with each task; inputting at least one second sample associated with the task into the intermediate mining model to obtain a candidate result set output for each second sample; screening each candidate result contained in each obtained candidate result set according to a preset evaluation standard to construct a second sample set; the second sample set comprises each second sample and the corresponding positive candidate result and negative candidate result; and training the intermediate mining model based on the second sample set to obtain a target mining model.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, in particular to the field of artificial intelligence technology, and provides a training method, device, electronic device and storage medium for a mining model. Background Art

[0002] Event context mining technology aims to organize events belonging to the same hot topic in a complete, accurate and orderly manner according to a certain logical relationship (such as temporal relationship, cause and effect relationship, etc.), helping the subject to quickly sort out the ins and outs of the event and understand the cause and effect of the event, thereby greatly saving the cost of information acquisition and improving the subject experience.

[0003] Traditional event context mining techniques are limited by the model's capacity and representational capabilities, typically only completing certain steps in the intermediate chain and requiring significant manual intervention, such as setting similarity thresholds and designing templates and schemas. This introduces a significant amount of prior bias (i.e., data constraints or structural assumptions based on manual experience) into the event context system, making truly automated mining difficult.

[0004] In addition, each of the above specific steps often requires a personalized customized model to complete, and personalized customized models usually lack the ability to dynamically adapt to different data and scenarios. Therefore, when the model capabilities are limited, once the model candidate results of one or several specific steps are deviated or misjudged, it is very easy for errors to propagate and accumulate, ultimately leading to poor quality of the generated event context, such as logical confusion, missing information, or unclear priorities.

[0005] Therefore, how to improve the efficiency and output quality of event context mining is an urgent issue that needs to be addressed. Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, electronic device, and storage medium for training a mining model to improve the efficiency of context mining and the quality of context.

[0007] The present invention provides a method for training a mining model, including:

[0008] An initial mining model is trained based on a pre-built first sample set to obtain an intermediate mining model; the initial mining model corresponds to multiple tasks, each task representing a sub-process in the event context mining process; the first sample set includes: a first sample and a sample label associated with the task content of each task;

[0009] Inputting a second sample associated with the task content of at least one of the tasks into the intermediate mining model to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining;

[0010] According to a preset evaluation criterion, each candidate result included in each obtained candidate result set is screened to construct a second sample set; the second sample set includes: each second sample, and a positive candidate result and a negative candidate result corresponding to each second sample; the degree of match between the positive candidate result and the negative candidate result and the preset evaluation criterion meets a preset matching condition;

[0011] The intermediate mining model is trained based on the second sample set to obtain a target mining model.

[0012] An embodiment of the present application provides a training device for a mining model, comprising:

[0013] A first training unit is configured to train an initial mining model based on a pre-constructed first sample set to obtain an intermediate mining model; the initial mining model corresponds to a plurality of tasks, each task representing a sub-process in the event context mining process; the first sample set includes: a first sample and a sample label associated with the task content of each task;

[0014] A sample construction unit is configured to input a second sample associated with the task content of at least one of the tasks into the intermediate mining model to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining; the candidate results contained in each obtained candidate result set are screened according to a preset evaluation standard to construct a second sample set; the second sample set includes: each second sample, and a positive candidate result and a negative candidate result corresponding to each second sample; the degree of match between the positive candidate result and the negative candidate result and the preset evaluation standard meets a preset matching condition;

[0015] The second training unit is configured to train the intermediate mining model based on the second sample set to obtain a target mining model.

[0016] An embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the above-mentioned mining model training methods.

[0017] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the above-mentioned mining model training methods.

[0018] An embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of the above-mentioned mining model training methods.

[0019] The beneficial effects of this application are as follows:

[0020] The embodiments of the present application provide a method, apparatus, electronic device, and storage medium for training a mining model. Specifically, the present application proposes a multi-task mining model for the event context mining process. A single mining model can implement the entire event context mining process without the need for customized models for each intermediate link or specific step. Furthermore, the present application improves the model's generalization ability and decision-making accuracy in multiple tasks through a two-stage iterative optimization mechanism.

[0021] First, in the first training phase, the initial mining model undergoes supervised multi-task joint training based on a pre-built, labeled first sample set, enabling the model to initially grasp the processing capabilities of each sub-process within the event context. Because each task corresponds to different training samples and labels, the model learns common feature representations across tasks during this phase, enhancing its ability to understand and express the semantics of the input content. This multi-task joint training approach not only improves the model's versatility but also reduces overall error caused by bias in the model for individual tasks.

[0022] In the second phase, at least one second sample related to the task is introduced and fed into the intermediate mining model to obtain a corresponding set of candidate results. Subsequently, the candidate results are screened according to pre-set evaluation criteria to construct a second sample set containing both positive and negative candidate results. This mechanism helps the model further distinguish between high-quality and low-quality outputs, strengthening its judgment in ambiguous or unclear situations. For example, by evaluating logical plausibility and information accuracy, it can effectively filter out unreasonable candidate results, ensuring that the final generated event context is clear and complete.

[0023] Finally, the intermediate mining model is retrained based on the second sample set to form the final target mining model. This coarse-to-fine training strategy not only improves the accuracy of the model on various tasks, but also enhances its robustness and adaptability in practical applications. In this way, the present application overcomes the limitations of traditional methods that require a lot of manual intervention and personalized customization of models, realizes efficient and automated processing of event context mining, and significantly improves the quality and reliability of event context.

[0024] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0026] Figure 1 This is an optional schematic diagram of an application scenario in an embodiment of the present application;

[0027] Figure 2 A flowchart of an implementation method of a mining model training method provided in an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of a first sample set in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a task instruction for a topic generation task in an embodiment of the present application;

[0030] Figure 5A A schematic diagram of a candidate result set in an embodiment of the present application;

[0031] Figure 5B is a schematic diagram of a second sample set in an embodiment of the present application;

[0032] Figure 6 This is a logic block diagram of a model training in an embodiment of the present application;

[0033] Figure 7 This is a topic merging logic block diagram in an embodiment of the present application;

[0034] Figure 8 A link diagram of a context mining system in an embodiment of the present application;

[0035] Figure 9A schematic diagram of an information retrieval process in an embodiment of the present application;

[0036] Figure 10A This is a topic search diagram in an embodiment of the present application;

[0037] Figure 10B This is a schematic diagram of a context search in an embodiment of the present application;

[0038] Figure 11 This is a schematic diagram of an event sequence in an embodiment of the present application;

[0039] Figure 12 This is a logic block diagram of interaction between a terminal device and a server in an embodiment of the present application;

[0040] Figure 13 A schematic diagram of the structure of a training device for a mining model according to an embodiment of the present application;

[0041] Figure 14 A schematic diagram of the hardware structure of an electronic device to which an embodiment of the present application is applied;

[0042] Figure 15 A schematic diagram of the hardware structure of another electronic device to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0043] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of the technical solutions of this application, but not all of them. Based on the embodiments described in this application document, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the technical solutions of this application.

[0044] The following is an introduction to some concepts involved in the embodiments of this application.

[0045] 1. Mining Model: The mining model in this application refers to a multi-task model that can uniformly handle multiple sub-processes within the event context mining process. Each task corresponds to a major step in event context construction, such as topic generation, event generation, and event topic attachment. During training, the model gradually establishes the ability to jointly represent the characteristics of each task. After training, the model can complete the automated processing of multiple sub-processes within a unified architecture, effectively reducing the reliance on manual rules and independent models in traditional methods, and improving overall mining efficiency and structured output capabilities.

[0046] 2. Generation tasks: These tasks involve automatically generating semantically clear and clearly structured topics or event content based on input information (e.g., topic information sources, event information sources, etc.). In event context mining, generation tasks are primarily responsible for extracting high-level semantic units from raw data, such as generating initial topics and extracting event descriptions, providing the foundation for subsequent structured processing.

[0047] 3. Decision-making tasks: These involve cognitive reasoning tasks such as judging, matching, and screening existing candidate content. In this application, decision-making tasks are primarily used to determine which topic an event should be assigned to, establishing semantic associations between event nodes and topics, and gradually building a complete event context structure. This type of task typically relies on the model's understanding of context and logical reasoning capabilities to ensure the accuracy and structure of the event context.

[0048] 4. Event context: This refers to a hierarchical information representation based on semantic understanding and structured integration of multiple related events, organized according to chronological order, causal relationships, or other logical relationships. This structure systematically presents the overall process of an event from its occurrence, development, and evolution, helping users quickly grasp the main content of an event and its internal connections. Event context not only includes basic event information (such as time, location, and subject), but also emphasizes the connections and evolutionary paths between events. It is a crucial component of the event knowledge system.

[0049] 5. Event Information Sources: These refer to the data sources used to support event identification and generation, including but not limited to news reports, social media updates, log records, online forum discussions, and video platform content. These information sources are highly timely and factual, reflecting the occurrence, development, and changing trends of events in the real or virtual world. They provide the raw corpus for generating event nodes in mining models and serve as an important input for constructing event context.

[0050] 6. Event: refers to the description of a situation or behavior in the real world or virtual world, usually including elements such as subject, action, time, and place. It is used to reflect the state change or interaction process in a specific context and is the basic unit for information organization and event context construction.

[0051] During event context mining, events can originate from a variety of information sources, including news reports, social media, and log records. After structured processing, these events are transformed into standardized event representations. Multiple related events can be organized according to logical relationships, such as chronological and causal relationships, to form a clearly structured event context, helping users understand the development paths and internal connections of complex events.

[0052] 7. Contextual Events: A contextual event is a broader, higher-level event, typically providing a more comprehensive background or larger context. A contextual event is a specific manifestation or subsequent development of the contextual event, focusing on specific details or instances. Contextual events are interconnected by inclusion, inheritance, or evolution. This hierarchical structure helps build a more logical and structured event context, enhancing users' understanding of the relationship between the overall and specific aspects of an event.

[0053] In the context of an event, a superordinate event refers to a macro-event with a higher level of abstraction and a broader scope, typically used to describe the overall context or key theme of a particular type of event. Subordinate events, on the other hand, are micro-events that occur specifically within that theme and have temporal or causal connections, serving as a concretization and refinement of the superordinate event. For example, "a tech company's annual new product launch" could be considered a superordinate event, while specific activities like "launching a new smartphone" or "announcement of a smart wearable device upgrade plan" would be considered subordinate events.

[0054] 8. Event Node: This is the basic unit of an event context, representing the structured information of a specific event. Each event node typically contains attribute information such as the event timestamp, subject object, key description, and sentiment. It connects to other nodes through semantic or temporal relationships, forming a complete event evolution path.

[0055] In the embodiments of the present application, events are converted into event nodes after standardization, becoming the smallest data unit that can be traced and analyzed in the event context. The design of event nodes is intended to improve the readability and structured degree of event information, making it easier to perform subsequent operations such as sorting, classification, and causal reasoning. Through the organization and linking of event nodes, it is possible to achieve a graphical presentation of event knowledge, enhance the logical clarity and information integrity of the event context, and provide users with more efficient information retrieval and decision support services.

[0056] 9. Topic: A semantically aggregated unit centered around a key theme or focal point, used to organize and categorize related events and information. A topic typically encompasses multiple event nodes and integrates these events into a coherent information set through semantic clustering and contextual association. In event context mining, topics serve as a bridge between the past and the future, serving as both the fundamental unit for event classification and an important basis for modeling event evolution paths. By identifying and maintaining topics, hierarchical organization and dynamic updating of event information can be achieved.

[0057] 10. Topic Information Source: refers to the original data source used to generate and update topics, including but not limited to trending words, news reports, social media updates, forum discussions, professional articles, and other resources. It can reflect some focus or the main content of a specific field, and provide a semantic basis and background support for topic modeling and event context construction.

[0058] 11. Preset evaluation criteria: refers to a set of quantifiable or determinable indicators used to measure the quality of the output results of the mining model, which aims to screen high-quality candidate results and construct a second sample set to support the iterative optimization of the model. The criteria include but are not limited to dimensions such as logical rationality, information accuracy, semantic relevance, and concise expression. For example, in event generation tasks, evaluation criteria may include event completeness (whether it contains key elements such as subject, action, and time), time accuracy (whether the time of event occurrence is consistent with the facts), subject recognition accuracy, semantic coherence, etc.; in topic generation tasks, they may include topic generalization (whether relevant events are accurately covered), keyword matching, differentiation from existing topics, semantic consistency, etc. By setting up a multi-dimensional evaluation system, it is possible to effectively identify positive candidate results with high confidence and low-quality negative candidate results, thereby improving the pertinence of model training and the quality of the final output event context.

[0059] 12. Event Topic Mounting: This refers to assigning events (or structured event nodes) to corresponding topics based on their semantic characteristics, forming a logically coherent event context. By analyzing the relevance of events to existing topics, the system automatically determines and mounts event nodes, ensuring that each topic covers the most relevant event set. This improves the hierarchy and readability of information organization and supports more efficient event monitoring.

[0060] The following is a brief introduction to the design concept of the embodiment of this application:

[0061] Numerous hot topics occur daily. With the development of internet technology, news resources related to these events have become widely disseminated and accumulated, and internet users generally prefer to quickly learn about current events through these hot topics. Against the backdrop of rapidly increasing amounts of information, efficiently sorting through this vast amount of information to understand the ins and outs of events and identifying their developmental trajectory and key milestones has become a key challenge in improving information acquisition efficiency and user experience.

[0062] To address this challenge, event context mining technology has emerged. Event context refers to integrating relevant information based on a thorough understanding of an event to extract a structured information presentation that reflects its causes, developments, and outcomes. Through event context, users can quickly grasp the overall evolution of a hot topic and its inherent logical relationships, significantly improving the efficiency and depth of information processing.

[0063] This technology aims to structure multiple events related to the same hot topic according to chronological order, causal relationships, and other logical relationships, helping users fully understand the development paths of events and their interrelationships. In the era of information explosion, event context mining is crucial for improving information integration capabilities and assisting decision-making and analysis.

[0064] However, traditional event context mining technology is limited by model capacity and representation capabilities, typically only able to complete certain specific steps in the process. It also relies heavily on manual intervention, such as setting similarity thresholds and template design, which introduces a significant amount of prior bias into the system and makes full automation difficult. Furthermore, each step often requires customized models, lacking the ability to adapt to diverse data and scenarios. With limited model capabilities, local errors can easily propagate and accumulate, resulting in confusing event context logic, missing information, or unclear priorities, impacting the quality of the final output.

[0065] Furthermore, current event context mining methods based on large language models (LLMs) typically rely on vector retrieval to obtain candidate events, which are then fed into the LLM via prompts at one time, with the model generating event context end-to-end. This method employs a black-box generation mechanism, lacking structured modeling of the event reasoning process and explicit expression of intermediate states. This results in poor interpretability of the mining process and is prone to hallucinations, resulting in confusion in causal and temporal logic. The resulting event context lacks hierarchy, making it difficult for users to understand and utilize it. Furthermore, this approach cannot effectively handle dynamic data streams. As events evolve, new developments are difficult to capture and integrate in a timely manner, limiting the system's ability to continuously update and optimize the event evolution path.

[0066] Therefore, how to achieve an efficient, explainable, and structured reasoning-capable mining method while ensuring the integrity and accuracy of the event context has become a key issue that needs to be urgently addressed.

[0067] In view of this, the present application proposes a training method, device, electronic device and storage medium for a mining model. For a multi-task mining model for event context mining processes, first, in the first stage, the initial mining model is supervised multi-task joint training based on a pre-constructed labeled first sample set, so that the model can initially grasp the processing capabilities of each sub-process in the event context; then, in the second stage, a second sample set is constructed by introducing positive and negative samples related to at least one task selected based on evaluation criteria, and the model is further fine-tuned to enhance its ability to recognize and output high-quality results.

[0068] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0069] like Figure 1 , which is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes a terminal device 110 and a server 120.

[0070] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; a client related to event context mining can be installed on the terminal device, which can be software (such as a browser, social software, etc.), or a web page, applet, etc. The server 120 is a background server corresponding to the software or web page, applet, etc., or a server specifically used for event context mining, which is not specifically limited in this application. The server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.

[0071] It should be noted that the training method of the mining model in each embodiment of the present application can be executed by an electronic device, which can be a terminal device 110 or a server 120, that is, the method can be executed by the terminal device 110 or the server 120 alone, or can be executed jointly by the terminal device 110 and the server 120.

[0072] For example, when the system is jointly executed by the terminal device 110 and the server 120, the phased training method proposed in this application can be used to train the initial mining model, obtain the target mining model, and deploy it on the server 120. During the model application phase, the server 120 can automatically mine and store event contexts by performing tasks such as topic generation, topic merging, event generation, event topic mounting, key event node identification, and event node deduplication. A user can trigger a context search request through the terminal device 110. Subsequently, the terminal device 110 interacts with the server 120 to obtain the event context search results and display them to the user. The specific implementation process will be described below and will not be repeated here.

[0073] In an optional implementation, the terminal device 110 and the server 120 may communicate via a communication network.

[0074] In an optional implementation, the communication network is a wired network or a wireless network.

[0075] It should be noted that Figure 1 The examples shown are just for illustration. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of this application.

[0076] In an embodiment of the present application, when there are multiple servers, the multiple servers can be combined into a blockchain, and the servers are nodes on the blockchain; as disclosed in the embodiment of the present application, the training method of the mining model, the relevant data involved can be saved on the blockchain, for example, the first sample set, the second sample set, model-related parameters, event context-related data, etc.

[0077] Furthermore, the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. For example, in smart transportation, it can be used to structure modeling of the "evolution path of urban traffic congestion"; in assisted driving, it can be used to monitor and construct the context of the "development process of accidents on specific road sections"; in cloud platform log analysis, it can be used to identify and organize the "evolution chain of abnormal system behavior"; and in artificial intelligence information flow analysis, it can automatically sort out the development process of hot topics and related events by modeling the information flow paths of multi-source data.

[0078] The following will further illustrate the application of the embodiments of the present application in actual business with reference to several typical application scenarios:

[0079] (1) Search engine.

[0080] In information distribution products like search engines, community content platforms, and short video recommendations, users often want to quickly understand the background and evolution of a hot topic. This application's model and event context mining method can automatically process multi-source information, such as real-time news and social media content, extract initial topics, and associate related event nodes to construct a clearly structured and time-series event context map.

[0081] This event context can be displayed in the form of a timeline, causal chain, etc. on the search results page or thematic page, helping users to grasp the full picture of the event at a glance and significantly improving information acquisition efficiency. At the same time, it also helps the platform achieve content aggregation and in-depth understanding, enhancing user stickiness and user experience.

[0082] (2) Recommendation system

[0083] In personalized recommendation systems, event context can be used as contextual features to improve the relevance and interpretability of recommended content. Through the model and event context mining method provided in this application, historical event data can be mined offline to construct a complete event development path and match it with user interest tags. The system can then simultaneously display the event background, key nodes, and latest developments on the push content landing page.

[0084] For example, when recommending news related to a certain subject to users, the cause, discussion process and current status of the related events can be displayed through the event context, so that users can obtain more comprehensive information support while consuming content, thereby enhancing the value and credibility of the recommended content.

[0085] (3) News reports.

[0086] In the fields of news editing and content management, event context mining technology can effectively improve editing efficiency and reporting quality. The model and event context mining method provided in this application can automatically extract hot topics and related event nodes, generate a preliminary event development map, and provide structured reference materials for editors.

[0087] For example, during major sporting events, information such as the progress of the event, key player performances, and important game milestones can be automatically organized into a clearly structured document that journalists can quickly access and write in-depth reports. This not only improves the timeliness of news production, but also enhances the professionalism and systematic nature of the content.

[0088] (4) Knowledge graph and intelligent question-answering system.

[0089] In knowledge graph construction and intelligent question-answering systems, event context can serve as an important component of structured knowledge. The model and event context mining method provided in this application can extract and organize multi-dimensional information about historical events, incorporating event nodes and their relationships into the knowledge graph to form a dynamic knowledge system with time-evolving characteristics.

[0090] For example, when answering questions such as "The development history of a certain technology company", the company's key journey from establishment, financing, product launch to market expansion can be displayed based on the event context, allowing users to obtain more comprehensive and well-organized information feedback, and improving the depth of understanding and expression ability of the question-answering system.

[0091] It should be noted that the several application scenarios listed above are just simple examples, and other application scenarios are also applicable to the embodiments of the present application and are not specifically limited here.

[0092] It is also important to emphasize that the specific implementations of this application involve user-related data, such as user context search requests, comments, and social platform discussions. When the above embodiments of this application are applied to specific products or technologies, the subject's permission or consent must be obtained, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0093] The following describes the training method of the mining model provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0094] The following first describes the training process of the multi-task mining model proposed in this application:

[0095] See Figure 2 As shown, it is an implementation flow chart of a training method for a mining model provided by an embodiment of the present application. Taking the server as the execution subject as an example, the specific implementation process of the method is as follows S21 to S24:

[0096] S21: Train the initial mining model based on the pre-built first sample set to obtain an intermediate mining model; the initial mining model corresponds to multiple tasks, each task represents a sub-process in the event context mining process; the first sample set includes: the first sample and sample label associated with the task content of each task.

[0097] In an embodiment of the present application, the first sample set is a structured training data set pre-constructed based on multiple tasks in the event context mining process (also referred to as multiple subtasks involved in event context mining), which is used to support multi-task joint training of the initial mining model. The sample set includes samples related to the task content corresponding to the multiple tasks (referred to as first samples) and sample labels. Here, the first sample refers to the input information required for each task, and the sample label is the target output or annotation information corresponding to the input information.

[0098] The data of the first sample set mentioned above may come from multiple public channels, including but not limited to some or all of the following: news websites, social media platforms, industry databases, public service notification platforms, news aggregation applications (Application, APP), etc.

[0099] Specifically, raw data can be obtained from the channels listed above or other channels through crawlers, application programming interfaces (APIs), or third-party data services, and then classified and organized according to the requirements of the event context mining task. Subsequently, the raw data is cleaned and preprocessed, including but not limited to removing noisy text, unifying time formats, normalizing entities (such as standardizing different representations of the same event), and deduplication. Then, based on the goals of each task, corresponding sample labels are generated using manual or semi-automatic labeling.

[0100] A combination of manual and semi-automatic labeling ensures label accuracy and consistency. For example, for tasks with complex semantics and high judgment standards, professional labelers perform each item according to unified labeling standards to ensure high-quality output of key information. For tasks that are more structured or rely on model-based initial screening, semi-automatic labeling based on rule engines, keyword matching, or pre-trained models is used, combined with a manual review mechanism to improve labeling efficiency and data availability.

[0101] The sample labels generated in the above way have good task adaptability and semantic consistency, which can effectively support the model learning process under multi-task joint training and improve the mining model's ability to understand various semantic relationships and structural logic in the event context.

[0102] In summary, the constructed first sample set includes first samples and sample labels corresponding to multiple tasks. Each task has at least one first sample, usually multiple, and each first sample corresponds to a sample label.

[0103] To more clearly define the types of tasks involved in the event context mining process and their roles in the process, the present embodiment further provides the following method:

[0104] An optional implementation is that the multiple tasks include at least one generation task and at least one decision task related to event context mining; wherein the event context is composed of a topic and at least one event node associated with the topic, and each event can be used as an event node for constructing the event context.

[0105] Specifically, an event context consists of several topics, each of which contains related event nodes. These event nodes can be either raw event descriptions or standardized event representations that have undergone deduplication, merging, and classification. This task-division mechanism enables hierarchical organization and dynamic updating of event knowledge.

[0106] Optionally, the generation task is used to generate topics and events based on input information; the decision task is used to attribute events to corresponding topics to build event context. In the specific implementation process, multiple tasks include but are not limited to the following forms:

[0107] Generation tasks include but are not limited to topic generation tasks, topic merging tasks, event generation tasks, etc., which are used to generate new semantic units based on input information (such as news headlines, historical events, topics, etc.).

[0108] Decision-making tasks include but are not limited to event topic mounting tasks, key event node identification tasks, event node deduplication tasks, etc., which are used to classify, filter and organize generated or extracted events to build a structured event context.

[0109] The following examples primarily illustrate that generation tasks include topic generation, topic merging, and event generation, while decision-making tasks include at least event topic mounting and may also include key event node identification and event node deduplication. It should be emphasized that the task divisions listed herein are for illustrative purposes only and do not constitute a limitation on the scope of protection of this application.

[0110] In practical applications, other related tasks can be introduced based on specific business needs, such as event timeline sorting (used to build a temporal structure for event development), event impact analysis (used to assess the impact of events), and topic popularity prediction (used to predict future trends in topics). By introducing these extended tasks, the ability to structure event context can be further enriched, improving the system's modeling accuracy and semantic understanding of complex event evolution processes, thereby enhancing the overall intelligence level and adaptability of the mining system to application scenarios.

[0111] By clearly defining the various task types involved in event context mining, this implementation provides a clearer functional division and logical structure for the entire event context modeling process. This approach helps improve the model's task understanding and output consistency under complex multi-task joint training, thereby enhancing the completeness, accuracy, and interpretability of the resulting event context.

[0112] Next, we briefly list the first samples and sample labels corresponding to several tasks:

[0113] a) Topic Generation Task: The first sample can be raw input text (i.e., topic information source), including trending keywords, related hot search content, and temporal context. Sample labels are generated topic names or abstract expressions, such as "X celebrity's new drama starts filming." This task can be performed by professional annotators to summarize representative initial topics from hot information, thereby training the model to identify semantic focus and generate concise topics.

[0114] b) Topic Merging Task: The first sample can be the text representation of a newly generated topic or several historically similar topics from an existing topic set. The sample label is the merged superordinate topic, such as "Evolution of Artificial Intelligence Technology Applications." Through this task, the model learns to identify semantic connections between topics and achieves aggregation and optimization of the topic system.

[0115] c) Event Generation Task: The first sample can be the title, body, or summary of a news article (i.e., the event information source). The sample label is a standardized event description, such as "A certain company released a new generation of artificial intelligence chips." For example, annotators extract key information from the event and generate a structured event description to improve the model's ability to extract and normalize event information.

[0116] d) Event topic assignment task: The first sample can include the event description text and the text representation of several candidate topics from a pre-existing topic set. The sample label is the number or name of the successfully matched topic, such as "Technology Industry Innovation Dynamics." Through this task, the model learns to accurately assign events to corresponding topics and improve the organizational structure of event context.

[0117] e) Key event node identification task: The first sample can include the event description text and contextual information about the topic to which it belongs. Sample labels can be binary classification results (e.g., critical or not) or multi-class classification results (e.g., strong / weak / non-critical). For example, domain experts can annotate event types with significant impact, such as "significant progress," to help the model identify high-value event nodes.

[0118] f) Event Node Deduplication Task: The first sample can include the text descriptions of two event nodes and their associated topic information. The sample label is the duplication determination result or semantic similarity score. Based on the initial screening of the semantic model, combined with manual confirmation and labeling, the model is equipped to identify and eliminate redundant events, improving the accuracy and clarity of the event context.

[0119] Through the sample construction method under the above tasks, the first sample set can comprehensively cover the key links in the event context mining process, ensuring that the initial mining model learns rich semantic representation and structural modeling capabilities during the training phase, thus laying a solid foundation for subsequent iterative optimization based on the second sample set.

[0120] Moreover, in the first stage, through multi-task joint training of the first sample set, the mining model can effectively learn the semantic associations and logical structures between each link in the event context, laying a solid foundation for subsequent iterative optimization of the model, improving its generalization ability and modeling accuracy in complex scenarios, and at the same time possessing cross-task generalization and reasoning capabilities.

[0121] The following is a brief description of the initial mining model:

[0122] In an embodiment of the present application, the initial mining model can be a randomly initialized model, or a pre-trained language model, such as an LLM, including but not limited to a Chat Generative Pre-trained Transformer (ChatGPT), DeepSeek, etc., or can be constructed based on a preset deep neural network structure, such as an encoder-decoder model using a transformer architecture, or a pre-trained language model based on a bidirectional encoder representation from transformer (BERT), a text-to-text transfer transformer model (T5), etc. for multi-task adaptation. Its parameters can be randomly initialized (such as by random initialization using a standard normal distribution) or partially derived from parameter migration of an existing pre-trained model.

[0123] On this basis, multi-task joint training is carried out based on the pre-constructed first sample set to gradually learn the semantic representation and structured modeling capabilities of each task in the event context mining process, and finally obtain an intermediate mining model with preliminary generalization capabilities.

[0124] Considering the collaborative modeling requirements of multiple tasks involved in the event context mining task, and the first sample set containing content and annotation information related to each task, in order to improve the modeling ability and generalization performance of the initial mining model for multiple tasks, the embodiment of the present application adopts a multi-task learning method to iteratively train the initial mining model based on the first sample set. In each round of training, the following steps S211 to S212 are performed ( Figure 2 not shown):

[0125] S211: Input the first sample selected from the first sample set in this round into the current initial mining model, and obtain the predicted probability distribution output by the current initial mining model guided by the task instructions related to the first sample.

[0126] Among them, the current initial mining model in the first round is the initial initial mining model; the current initial mining model in other rounds is the initial mining model output in the previous round.

[0127] During each round of training, one or more first samples are randomly selected from the first sample set as the input data for the current batch. This sample selection can be done by random sampling to ensure that the model is fully exposed to a variety of task samples during training, avoiding learning bias and improving the model's generalization and robustness across different tasks.

[0128] The number of samples can be set based on hardware resources and training stability. For example, mini-batch sampling (e.g., selecting 4 to 16 samples per iteration) can be used to balance training efficiency and model convergence. This sampling strategy helps the model better capture common features and differential expressions across tasks, thereby improving overall training effectiveness and the accuracy of event context modeling.

[0129] Optionally, in order to make multi-task joint training more flexible and efficient, the samples used in each round of training can be selected according to different strategies. The following is a detailed description of how to select samples in each round in S211:

[0130] In the embodiment of the present application, considering that the event context mining task involves the collaborative modeling requirements of multiple tasks, in order to improve the modeling capability and generalization performance of the initial mining model for multiple tasks, when training the initial mining model based on the pre-constructed first sample set, the samples selected in each round can adopt the following strategies:

[0131] Sample selection strategy 1: single-task rotation strategy.

[0132] In each training round, only the first sample related to a specific task is selected for training. For example, in the first round, samples related to the topic generation task are selected, in the second round, samples related to the topic merging task are selected, in the third round, samples related to the event generation task are selected, in the fourth round, samples related to the event topic attachment task are selected, and so on. This strategy helps the model gradually optimize on each task and ensures that each task receives balanced learning opportunities.

[0133] Sample selection strategy 2: multi-task hybrid strategy.

[0134] Each training round simultaneously selects first samples related to multiple different tasks for training. For example, a single training round can include samples from topic generation as well as samples from multiple tasks, such as topic merging, event generation, and event topic attachment. This allows the model to learn feature representations for multiple tasks in a single iteration, enhancing its cross-task knowledge transfer capabilities.

[0135] like Figure 3 As shown, it is a schematic diagram of a first sample set in an embodiment of the present application. Figure 3The first sample set shown can be divided into multiple sample subsets according to the task category corresponding to the sample, such as topic generation sample subset, topic merging sample subset, event generation sample subset, event topic mounting sample subset, key event node identification sample subset and event node deduplication sample subset, where different patterns represent the first samples corresponding to different tasks, and the sample label corresponding to each first sample is Figure 3 When a multi-task hybrid strategy is used to select samples, the first sample of a batch selected may include samples corresponding to multiple tasks, such as Figure 3 As shown in 30.

[0136] Sample selection strategy three: dynamic sampling strategy.

[0137] Based on the current training results of each task (such as the trend of the loss function or the performance on the validation set), the task allocation ratio for each round of sample selection is dynamically adjusted. For example, if the loss value of a task decreases slowly, the proportion of samples from that task is increased in the next few rounds of training to accelerate convergence. Conversely, for tasks that perform well, the proportion of samples is appropriately reduced. This approach helps balance the learning progress of each task and ensures improved overall model performance.

[0138] Sample selection strategy four: task priority strategy.

[0139] Prioritize tasks based on their importance and complexity, and select samples in this order for each training round. For example, focus on training the more complex event generation task first, and once it reaches a certain level of performance, move on to other, relatively simpler tasks. This ensures that critical tasks are fully optimized first, thereby improving the robustness of the overall system.

[0140] Specifically, in each round of training, the server can select appropriate first samples and input them into the current initial mining model based on one or more of the above strategies. For example, when adopting a multi-task hybrid strategy, the system randomly extracts the first samples of several different tasks from the first sample set to form a mixed batch input model, and uses the corresponding task instructions as a guide to obtain the predicted probability distribution of the model output. Through diversified sample selection methods, the model can effectively learn the semantic associations and logical structures between tasks during the training process, further improving its generalization ability and modeling accuracy in complex scenarios.

[0141] By flexibly applying these sample selection strategies, the learning progress of each task can be better balanced during multi-task joint training, ensuring that the model can comprehensively and effectively master various skills in the event context mining process.

[0142] After selecting several first samples in each round, each selected first sample can be input into the current initial mining model respectively, and guided by the task instructions related to the first sample, the initial mining model performs corresponding task semantic understanding and structured output generation, thereby obtaining the predicted probability distribution of the current initial mining model output.

[0143] Specifically, in this step, the server inputs the text with task instruction prompts into the current initial mining model according to the current training task type (such as topic generation, topic merging, event generation, etc.), and the model outputs the corresponding target sequence and its probability distribution based on the input samples and task instructions.

[0144] Task instructions are a set of structured textual information used to clarify task objectives and guide model behavior during model training or inference. They tell the model what type of processing to perform, including which semantic elements to recognize, what logical order to follow, and what format to output. They serve as a bridge between user intent and model response.

[0145] In order to effectively guide the model to learn the semantic representation and logical structure of each task in event context mining during the training process, when performing the operation as described in S211, a supervised fine-tuning (SFT) process driven by task instructions is actually being implemented. This means that on the basis of the pre-trained language model, high-quality supervised data with labels is used to train the model according to the requirements of the specific task to improve its performance on the specific task. This process usually guides the model to learn task-related semantic understanding and generation capabilities in the form of input instruction-output pairs, and is widely used in instruction adaptation and function customization of large models. This training method forms the basis for subsequent multi-task joint optimization.

[0146] Taking the LLM as an example, in the SFT phase, multi-instruction fine-tuning is a training strategy for the LLM. This strategy uses a series of different types of instructions or task descriptions to guide the model's learning process. This strategy aims to enhance the model's generalization and task adaptability through diverse and detailed task instructions. This approach is not limited to a single type of input and output pattern, but rather, by introducing a variety of different task instruction sets, enables the model to understand and handle a wider range of task types and scenarios.

[0147] The following are simple examples of the corresponding task instructions for the tasks of topic generation, topic merging, event generation, event topic mounting, key event node identification, and event node deduplication:

[0148] (1) Topic generation task.

[0149] Taking the topic generation task as an example, the task instructions used by SFT are as follows: Figure 4 As shown, including:

[0150] You are tasked with generating topics based on news headlines. Before doing so, please familiarize yourself with the following two important concepts:

[0151] Concept 1: Superordinate Events

[0152] A superordinate event refers to an event of a wider scope and higher level, which usually provides a more comprehensive background or larger context, while a subordinate event is the specific manifestation or subsequent development of the superordinate event, focusing on certain details or instances.

[0153] Concept 2: Determiners

[0154] Qualifiers are important information that limits and supplements an event, including key information such as time, place, person, reason, and quantity. It's important to accurately identify the subject of the event and avoid mistaking a noun that merely serves as a modifier for the subject. To complete the task, follow these steps:

[0155] Step 1: Determine whether it is an event

[0156] Read the following news headline and first determine whether it describes a specific factual event. If the headline does not refer to any actual event, directly output "non-factual event" and end the task.

[0157] Step 2: Determine the event status

[0158] If the headline describes an event, further analyze its status: whether it is an event that has not yet occurred, is uncertain, or is false. For such events, the generated event phrase should clearly indicate whether it has not yet occurred, is uncertain, or is a rumor. Ensure that the generated content is consistent with the facts expressed in the headline.

[0159] Step 3: Extract hypernyms and qualifiers

[0160] Combine the concepts of "superordinate events" and "qualifiers" above to extract superordinate events and important qualifiers from the title. Ensure that the extraction process is clear and accurate.

[0161] It should be noted that the superordinate events and important qualifiers extracted in step 3 are used to generate topics.

[0162] (2) Topic merging task.

[0163] Taking the topic merging task as an example, the task instructions used by SFT are as follows:

[0164] You will be tasked with determining whether a newly generated topic should be merged with a historical topic. Before doing so, please understand the following two key concepts:

[0165] Concept 1: Semantic Similarity

[0166] Semantic similarity means that two topics are highly consistent in terms of the main meaning expressed, the subjects involved, and the background information. Even if the expressions are different, they should be regarded as belonging to the same topic category.

[0167] Concept 2: Topic-level abstraction

[0168] Topic-level abstraction refers to promoting multiple subordinate topics to more general superordinate topics through merging operations, thereby reducing redundancy and enhancing the structure of the topic system.

[0169] To complete a task, follow these steps:

[0170] Step 1: Read the input

[0171] The input includes the current newly generated topic and several historical topics. Please analyze their semantic expressions one by one.

[0172] Step 2: Determine semantic similarity

[0173] To determine whether a new topic has a high degree of semantic similarity with any historical topic, comparison can be made from dimensions such as subject, time, place, and behavior.

[0174] Step 3: Perform topic merging

[0175] If there are highly similar historical topics, they are merged into a more abstract and broader topic; if there is no match, the new topic is kept unchanged.

[0176] (3) Event generation task.

[0177] Taking the event generation task as an example, the task instructions used by SFT are as follows:

[0178] You are tasked with generating standardized event descriptions based on news content. Before doing so, please understand the following two key elements:

[0179] Concept 1: Event Structuring Elements

[0180] Event descriptions must include basic elements such as subject, action, time, and place to ensure the completeness and accuracy of event expression.

[0181] Concept 2: Normalized Expression of Events

[0182] When describing an event, avoid using subjective modifiers, emotional vocabulary, or uncertain tones, and use objective, concise, and unified language to describe it.

[0183] To complete a task, follow these steps:

[0184] Step 1: Identify the subject and behavior of the event

[0185] Extract the subject of the event and its specific behavior from the news text.

[0186] Step 2: Extract time and location information

[0187] Clarify the time node and geographical space scope of the event.

[0188] Step 3: Generate standardized event expressions

[0189] Integrate the above information into a concise event description, such as: "A company releases a new generation of artificial intelligence chips."

[0190] (4) Event topic mounting task.

[0191] Taking the event topic mounting task as an example, the task instructions used by SFT are as follows:

[0192] You are responsible for attaching events to appropriate topics. Before doing so, please understand the following two main principles:

[0193] Concept 1: Semantic Relevance

[0194] The semantic relevance between events and topics determines their affiliation, which is mainly judged based on the subject of the event, the participants, the background of the event, etc.

[0195] Concept 2: Topic-Level Adaptability

[0196] When attaching an event, you need to consider the degree of match between the depth of its content and the level of the topic, and avoid directly classifying detailed events into overly macro topics.

[0197] To complete a task, follow these steps:

[0198] Step 1: Understand the event content

[0199] Read the input event description to understand its theme and background information.

[0200] Step 2: Retrieve candidate topics

[0201] The system provides several potentially relevant historical topics for selection.

[0202] Step 3: Execute event mounting

[0203] Based on semantic relevance and topic hierarchy, events are accurately assigned to the most appropriate existing topics.

[0204] (5) Key event node identification task.

[0205] Taking the key event node identification task as an example, the task instructions used by SFT are as follows:

[0206] You are responsible for determining whether an event is a critical event node. Before doing so, please understand the following two key definitions:

[0207] Concept 1: Key event nodes

[0208] Key event nodes refer to developments that are of great significance to the development of the topic, such as the parties speaking out, major turning points, and the official announcement of the handling results.

[0209] Concept 2: Non-critical event nodes

[0210] Non-critical event nodes usually refer to repeated reports, background introductions, irrelevant comments, etc., which do not have the ability to promote the development of the topic.

[0211] To complete a task, follow these steps:

[0212] Step 1: Read the event description

[0213] Clarify the specific content and nature of the incident.

[0214] Step 2: Determine the event type

[0215] Analyze whether the incident falls into the category of key developments, such as whether there are substantial changes, intervention by authoritative parties, etc.

[0216] Step 3: Output the judgment result

[0217] If it is a key node, output "yes"; otherwise, output "no".

[0218] It should be noted that this task instruction is based on two-classification as an example, of course three-classification or other multi-classification can also be feasible.

[0219] (6) Event node deduplication task.

[0220] Taking the event node deduplication task as an example, the task instructions used by SFT are as follows:

[0221] You are responsible for determining whether two events are duplicate event nodes. Before doing so, please understand the following two basic concepts:

[0222] Concept 1: Event Semantic Consistency

[0223] Event semantic consistency means that two events are highly consistent in terms of subject, behavior, time, location, etc., and should be regarded as the same event even if the expressions are different.

[0224] Concept 2: Diversity of Event Expression

[0225] The same event may be expressed in many different ways depending on the source, but the main facts remain the same. The ability to recognize this diversity is required.

[0226] To complete a task, follow these steps:

[0227] Step 1: Read the two event descriptions

[0228] Compare key information such as the subject, behavior, time, and place of the two.

[0229] Step 2: Determine semantic consistency

[0230] Analyze whether the two describe the same event and whether there are substantial differences.

[0231] Step 3: Output the judgment result

[0232] If it is a repeated event, output "yes"; otherwise output "no".

[0233] It should be noted that the task instructions listed above simply emphasize the concept definition, judgment criteria, execution process and output format. Of course, in addition to this, the task instructions can also include other auxiliary content according to actual needs, such as task background description, context dependency prompts, language style requirements, common error avoidance suggestions, multi-task collaboration prompts, etc., to enhance the model's understanding of the task objectives and the execution effect. This is not specifically limited in the embodiments of the present application, and can be flexibly designed and adjusted according to different task requirements.

[0234] In practical applications, the corresponding task instruction template can be dynamically matched based on the task category corresponding to the selected first sample. These task instructions can then be concatenated with the input sample and fed into the model for training. For example, when selecting the first sample using a multi-task hybrid strategy, the six task instructions mentioned above can be organized into a unified set of SFT instruction templates to guide the multi-task joint training process of the initial mining model, helping to improve the model's understanding of different tasks and its generalization performance.

[0235] S212: According to the difference between the predicted probability distribution and the true probability distribution of the corresponding sample label, the parameters of the current initial mining model are adjusted to obtain the initial mining model output in this round.

[0236] In the context of event context mining, the predicted probability distribution of the mining model output in this application refers to the model's confidence distribution for each possible output outcome for the current input sample. In other words, it is the probability value assigned by the model to all possible output options based on the input information and task objectives, indicating the model's judgment tendency towards different outcomes.

[0237] In the embodiments of the present application, the form of expression of the predicted probability distribution varies depending on the task type. For generation tasks (such as topic generation, event generation, etc.), the mining model can adopt a sequence generation method to calculate a probability distribution on the entire vocabulary for the output token at each decoding position (one decoding position corresponds to one time step), indicating the most likely output word at that position. By combining the maximum probability words of each time step, such as using greedy search or beam search, a complete generation result can be obtained. Among them, the vocabulary refers to the set of all discrete tokens that the model can recognize and output during the training and reasoning process, and its content can be derived from the vocabulary space defined by the tokenizer used by the model, including but not limited to common words, special symbols and some sub-word units, which will not be repeated here.

[0238] For decision-making tasks (such as event topic mapping, event node deduplication, and key node identification), the mining model can output a fixed-dimensional probability vector in the form of a classifier, corresponding to each category (such as "yes / no," "strong / weak / non-key node," etc.). This distribution reflects the model's confidence that the current input belongs to a certain category. During training, it can be used to compare with the true label, calculate the loss function, and guide model parameter updates.

[0239] Therefore, whether for generation or decision-making tasks, in event context mining, the predicted probability distribution not only influences the final output selection but also forms the primary basis for model training. By analyzing these distributions, we can also assess the model's uncertainty for certain tasks, thereby assisting in the design of subsequent optimization strategies.

[0240] Correspondingly, the true label refers to the standard answer for the current input sample, which is manually annotated or predefined by the system. It is used to reflect the correct output result of the sample under the corresponding task. It is the main training signal in supervised learning and is used to guide the model to optimize its predictive ability.

[0241] In the generation task, the true label is represented as a sequence of tokens arranged in order, where each token corresponds to a one-hot encoding in the vocabulary, representing the correct output word at that position.

[0242] In decision-making tasks, the true label is usually expressed as a specific category number, such as 0 or 1 in a binary classification task, and 0, 1, 2, etc. in a multi-classification task, corresponding to different semantic categories.

[0243] Specifically, in this step, the server compares the predicted probability distribution output by the model with the true labels, calculates the corresponding loss function, and uses algorithms such as backpropagation to drive the update of model parameters, improving its modeling accuracy and generalization capabilities in event context mining tasks. During the update process, optimizers (such as AdamW), learning rate scheduling strategies, and gradient clipping mechanisms can also be introduced to improve training stability and convergence speed.

[0244] Among them, the loss function calculated during the training process can be a cross-entropy (CE) loss. In the SFT stage, the present application uses a multi-instruction fine-tuning method to train a single initial mining model on all NLP tasks, which can be a single large language model. The training loss function is shown in the following formula 1, which is a calculation formula for a cross-entropy loss in an embodiment of the present application:

[0245]

[0246] Where N represents the total number of all tokens involved in a batch (for example, a batch involves 5 samples, and the output of each sample contains 5 tokens, then N is 25). i Indicates the sample label corresponding to the i-th token in these N tokens, Indicates the predicted probability corresponding to the i-th token among these N tokens.

[0247] Optionally, the server can also use other types of loss functions to measure the difference between the predicted probability distribution and the true label, such as Mean Squared Error (MSE) loss, Mean Absolute Error (MAE), Huber loss, etc., which are not detailed here. These loss functions can be selected and applied according to specific task requirements.

[0248] In an embodiment of the present application, when adjusting the parameters of the initial mining model, the model parameters refer to the learnable variables that constitute the network structure of the initial mining model and affect its input-output mapping relationship. Specifically, the model parameters may include but are not limited to: the weight matrix and bias term in the neural network layer, the query (Q), key (K), value (V) projection parameters in the attention mechanism, the scaling factor and offset factor of the normalization layer, etc. These parameters are continuously updated through the backpropagation algorithm during the training process, so that the model can better fit the training data and improve the modeling ability and generalization performance in the event context mining task.

[0249] Optionally, some other parameters besides the model parameters can be adjusted during the training process, such as the learning rate configuration parameters, which are used to control the update step size of the parameters of each part of the model during the gradient descent process. Different learning rate strategies can be set according to different tasks or parameter types, such as the basic learning rate, learning rate decay factor, number of warm-up steps, etc.; optimizer state parameters, such as the momentum, variance and other intermediate state information in the AdamW optimizer; task adaptation head parameters, such as the classification / generation head part used for different tasks, can also be trained as adjustable parameters.

[0250] The above implementation method uses an iterative training method to perform multi-task joint training on the initial mining model. In each round of training, the first sample is input into the current model, and the predicted probability distribution is generated guided by the task instructions, thereby gradually optimizing the model's performance on each task of event context mining. This method can effectively improve the model's ability to learn multi-task semantic representations and enhance its cross-task generalization performance and structural modeling capabilities. At the same time, through multi-task mixing or rotation training strategies, it helps the model establish correlations between tasks in a complex semantic space, thereby improving the accuracy and robustness of the overall event context construction.

[0251] Optionally, considering that large-scale pre-trained language models have large parameters and high tuning and storage costs in the process of multi-task joint training, in order to improve training efficiency, maintain model flexibility, and reduce computing resource consumption, the embodiment of the present application can also introduce a low-rank adaptation (LoRA) mechanism when tuning the parameters of the initial mining model, and only incrementally update some key parameters of the model instead of retraining all parameters.

[0252] That is, when executing S212, an optional implementation method is as follows:

[0253] According to the difference between the initial prediction probability and the corresponding sample label, the LoRA method is used to adjust the parameters of the current initial mining model.

[0254] Specifically, the LoRA method introduces a lightweight trainable adapter module next to the original frozen model weight matrix (referred to as the original weight matrix) through low-rank decomposition, thereby realizing incremental adjustment of model parameters and significantly reducing the computing resources and storage overhead required during the fine-tuning process.

[0255] For example, the original weight matrix W is expanded to:

[0256] W'=W+ΔW=W+A·B T (Formula 2);

[0257] Where W represents the fixed weight matrix in the original model, such as the query / key / value projection matrix in the self-attention module in the Transformer layer, the fully connected weights of the feed-forward neural network (FFN) layer, etc.; ΔW is the low-rank incremental matrix, which represents the incremental update to the original weight matrix and is constructed by the low-rank matrices A and B, where A is the low-rank adapter matrix (left factor) and B is the low-rank adapter matrix (right factor); W' is the adjusted weight matrix, which is obtained by adding W and ΔW.

[0258] During training, the original weight matrix W is frozen and not updated, and only the low-rank matrices A and B are optimized. This efficient parameter fine-tuning method not only preserves the knowledge capabilities of the original model but also significantly reduces training costs. It is particularly suitable for event context mining tasks in multi-task joint training scenarios.

[0259] Optionally, when the LoRA method is used to adjust the model parameters, multiple related parameters can also be adjusted to improve the adaptation effect, including but not limited to: low-rank adapter matrix (such as A and B matrices in LoRA), which is used to control the incremental adjustment of model weights; rank size (r), which controls the dimension of the low-rank matrix and affects the flexibility and computational overhead of model adjustment; the insertion position of the adaptation module, such as choosing to insert the LoRA module in the self-attention module or FFN layer of the Transformer to achieve differentiated adaptation of different model components.

[0260] By properly setting these parameters, we can preserve the model's general capabilities while making it more efficiently adaptable to the specific needs of event context mining tasks. This approach is particularly suitable for the rapid migration and continuous optimization of large-scale pre-trained models across multiple tasks and scenarios, and has good engineering applicability and scalability.

[0261] This implementation introduces a low-rank adaptive mechanism during parameter adjustment, updating only the low-rank incremental matrix in the model rather than retraining all parameters. This approach significantly reduces computational resource consumption and storage overhead during training, improves the efficiency and flexibility of model fine-tuning, and enhances the model's generalization capabilities in multi-task scenarios. It is particularly suitable for complex application scenarios such as event context mining, which involve multiple tasks and require frequent model adaptation.

[0262] Through the training process described in S211 to S212 above, the embodiment of the present application can effectively realize multi-task joint training of the initial mining model, so that it can gradually have the ability to understand and model each link of the event context, thereby providing high-quality basic model support for subsequent model iterative optimization.

[0263] In the embodiment of the present application, the iterative training process of the initial mining model can be terminated according to a preset stopping condition to ensure model convergence and avoid overfitting. Among them, the initial mining model output after the last execution of S212 is the intermediate mining model obtained by training.

[0264] Optionally, stop model training when any of the following conditions are met:

[0265] (1) The model's evaluation indicators (such as accuracy and F1 value) on the validation set have not shown significant improvement over multiple training cycles;

[0266] (2) The training loss value tends to be stable and lower than the preset threshold;

[0267] (3) Reach the pre-set maximum number of training rounds.

[0268] It should be noted that the several stopping conditions listed above are just simple examples. The stopping conditions of other model training are also applicable to the embodiments of this application and will not be repeated here. By setting a reasonable training termination mechanism, the training cost can be effectively controlled and the generalization ability of the model can be improved.

[0269] After completing the multi-task joint training of the initial mining model and obtaining an intermediate mining model with preliminary modeling capabilities, in order to further improve the output quality and discrimination ability of the model on specific tasks, a variety of candidate results can be generated based on the intermediate mining model, as shown in the following S22:

[0270] S22: Inputting the second samples associated with the task content of at least one task into the intermediate mining model respectively to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining.

[0271] Among them, the second sample is also training data related to the event context mining task, and its form is similar to the first sample.

[0272] Optionally, the second sample can be part of the first samples in the first sample set. For example, the first samples associated with some tasks that require further intensive training can be re-screened as second samples to participate in the subsequent training process to enhance the generalization ability of the model on such tasks.

[0273] In addition, the second samples can also be new samples outside the first sample set. The sources of these new samples are similar to those of the first samples mentioned above and will not be repeated here. They are used to expand the distribution of training data and improve the model's adaptability to unseen samples and overall performance.

[0274] Specifically, the task corresponding to the second sample may be any one or more of the multiple tasks corresponding to the event context mining process.

[0275] In step S22, a second sample associated with the task content of at least one task is input into the intermediate mining model to obtain a candidate result set for each second sample. This process not only verifies the model's performance on specific tasks but also distinguishes between different types of tasks: generation tasks and decision-making tasks. Generation tasks require the model to flexibly produce diverse output results based on input information, while decision-making tasks focus on classifying or judging the input data and outputting a probability vector of fixed dimensions.

[0276] Optionally, considering that most decision tasks are classification tasks, the model only needs to output a single category number among the candidates, so the ideal alignment effect can be obtained through SFT training of high-quality data; for generation tasks, the model needs to generate high-quality text that is preferred by humans based on a single or multiple inputs; and decision tasks usually rely on the structured information output by the generation task as a basis, so when the mining model is trained in the second stage, the task corresponding to the second sample used can be at least one generation task, such as a topic generation task, a topic merging task, an event generation task, etc.

[0277] The generation task can more comprehensively evaluate the language understanding and expression ability of the model. By generating multiple semantically related but differently expressed candidate results for diverse inputs, it can not only test the creativity and flexibility of the model, but also provide a rich data basis for further screening of representative samples. In addition, the output diversity of the generation task helps to discover potential knowledge points and patterns, which is especially important for event context mining in complex scenarios. Therefore, in the embodiments of the present application, the importance of the generation task is particularly emphasized, and it is used as the main task type of the second sample to enhance the overall generalization ability and adaptability of the model.

[0278] In this way, the model's performance in generation tasks can be strengthened based on the screening results, and its judgment accuracy and generalization ability in subsequent decision-making tasks can be improved, thereby enhancing the coherence and stability of the entire event context mining process.

[0279] The following is a brief description of the generation of the candidate result set:

[0280] In S22, for each second sample, the intermediate mining model can output a variety of candidate results by switching different generation strategies under the same input conditions, thereby constructing a candidate result set with rich expression forms. Each candidate result set includes multiple candidate results generated by different generation strategies for the same second sample.

[0281] Here are a few simple generation strategies:

[0282] (1) Greedy search: In each decoding step, the token with the highest probability is selected as the output, and the complete sequence is gradually constructed. This method is computationally efficient, but it is prone to falling into local optimality and lacks diversity in the generated results.

[0283] (2) Beam search: Maintains a fixed-width set of candidate paths, expands all possible high-probability words at each step, and retains the optimal path. Compared with greedy search, it is more global and can generate more coherent and higher-quality text.

[0284] (3) Sampling Decoding: Randomly sample the next token according to the probability distribution of the model output, allowing low-probability words to be selected, thereby introducing a certain degree of randomness and creativity and improving the diversity of the generated results.

[0285] (4) Top-k Sampling: At each step, only the k words with the highest probability are sampled, limiting the range of candidate words to ensure language rationality while retaining a certain degree of randomness to avoid completely deterministic output.

[0286] (5) Top-p Sampling (Nucleus Sampling): Instead of fixing the number of candidate words, the minimum set of words whose cumulative probability exceeds the threshold p is selected for sampling, dynamically adapting to the vocabulary distribution in different contexts, and further improving the generation diversity and controllability.

[0287] (6) Temperature Scaling: This is done by adjusting the "temperature" parameter of the probability distribution of the softmax output to control the randomness of the output. A high temperature makes the distribution flatter and increases randomness; a low temperature makes the distribution sharper and increases determinism.

[0288] It should be noted that the several generation strategies listed above are just simple examples. In addition, other generation strategies are also applicable to the embodiments of the present application and will not be described in detail here.

[0289] like Figure 5A As shown, it is a schematic diagram of a candidate result set in an embodiment of the present application. Figure 5A It means that a second sample (such as a white smiley face) is input into the intermediate mining model, and different candidate results are output through different generation strategies, such as black smiley face 1 (corresponding to generation strategy 1), black smiley face 2 (corresponding to generation strategy 2), black smiley face 3 (corresponding to generation strategy 3), black smiley face 4 (corresponding to generation strategy 4), black smiley face 5 (corresponding to generation strategy 5),... These candidate results constitute the candidate result set of the second sample.

[0290] The following is a brief example of candidate results:

[0291] In topic generation tasks, the intermediate mining model can generate multiple semantically related but distinct topic descriptions. For example, given the news headline "Tech company releases next-generation autonomous driving system," the model can generate candidate topic descriptions such as "Autonomous driving system undergoes a complete upgrade," "Tech company launches next-generation autonomous driving technology," "Intelligent driving sees another major advancement," and "Next-generation autonomous driving platform officially unveiled."

[0292] In the topic merging task, the intermediate mining model can generate a more general fusion topic description based on multiple existing input topics. For example, it can merge "heavy rains in a certain place caused disasters" and "heavy rains in many places caused floods" into "heavy rains have frequently occurred in many places recently, and the flood prevention situation is severe", "some areas have encountered extreme precipitation, forming regional floods", "heavy rains in many provinces and cities for several days have caused damage to infrastructure", etc.

[0293] In the event generation task, the intermediate mining model can generate multiple event phrases with different expressions but consistent semantics based on the news title or text content. For example, "A company releases a new generation of AI chips" can be generated as: "A new generation of artificial intelligence chips is officially launched", "A company launches a high-performance AI processor", "The country's first AI acceleration chip is successfully launched", "A technology company releases a new AI computing power platform", "AI chip technology is upgraded again, and performance is significantly improved", "A brand releases AI chips for intelligent driving", "A new generation of AI chips are unveiled, supporting edge computing applications", etc.

[0294] In this way, a diverse candidate result set corresponding to each second sample can be obtained, which can be used for subsequent screening of representative samples and construction of a more representative and discriminative training data set.

[0295] After obtaining a diverse set of candidate results for the second sample output associated with the task content of at least one task through the intermediate mining model, these candidates need to be screened according to preset evaluation criteria to ensure the quality of the final training data. This process allows high-quality positive candidates and representative negative candidates that best meet the task requirements to be selected from the numerous candidate results, providing a solid data foundation for subsequent model optimization, as described below:

[0296] S23: According to the preset evaluation criteria, each candidate result contained in each obtained candidate result set is screened respectively to construct a second sample set; the second sample set includes: each second sample, and the positive candidate result and negative candidate result corresponding to each second sample; the matching degree between the positive candidate result and the negative candidate result and the preset evaluation criteria meets the preset matching degree condition.

[0297] In order to ensure that high-quality training samples are screened out from the candidate result set so as to construct an effective second sample set, the embodiment of the present application screens the candidate results according to the preset evaluation criteria in step S23, which is also called labeling, that is, labeling the positive and negative results from the candidate result set. Among them, the preset evaluation criteria are the reference standards when labeling the candidate results, which are used to measure the quality of the output results of the intermediate mining model. Specifically, it is a set of quantifiable or determinate indicators, which may include but are not limited to: logical rationality, information accuracy, simplicity and fluency, illusion problems, relevance, information completeness and other dimensions. These dimensions can be set based on task-specific quality requirements, business needs, etc., and can also be set based on the subjective preferences of the target user group or specific application scenarios, such as style tendencies, word usage habits, emotional colors, etc. The main purpose is to make the generated results closer to the values, aesthetic orientations or communication habits reflected in the human preference labels.

[0298] These evaluation criteria can not only be used for automated match calculations, but also serve as the primary basis for human preference annotation. In practical applications, manual or semi-automatic auxiliary annotation of candidate results, or even model-generated preference prediction results in specific scenarios, can further verify and supplement the deficiencies of automated evaluation, improving the accuracy and task suitability of screening results.

[0299] Optionally, you can use specific numerical indicators to measure the degree of match between candidate results and preset evaluation criteria (referred to as matching degree), thereby systematically evaluating and comparing the quality of different samples to screen positive and negative candidate results. The following briefly lists several matching degree calculation methods:

[0300] Calculation method 1: Preference Score.

[0301] Each candidate is scored based on manual annotation or automated evaluation tools, reflecting its performance against pre-set evaluation criteria, such as information accuracy, language fluency, style, and emotional tone. For example, if a majority of annotators believe that candidate A meets the "information completeness" requirement more than candidate B, then candidate A will receive a higher preference score. This method is suitable for quantitative evaluation of content preference criteria.

[0302] Calculation method 2: statistical significance test.

[0303] Methods such as t-tests and chi-square tests are used to determine whether the difference between two candidate results under a specific evaluation criterion (such as language fluency, conciseness, and information completeness) is statistically significant, thereby preventing the screening results from being affected by chance. This method, as a supplement to other evaluation methods, helps to enhance the reliability of evaluation conclusions and is suitable for verifying quality standards and content preference criteria.

[0304] Calculation method three: machine learning indicators.

[0305] For generation tasks (such as topic generation, topic merging, event generation, etc.), some generation results can be converted into quantifiable classification problems by constructing binary or multi-valued judgment labels (for example, whether it contains key information, whether it is grammatically correct, whether it is logical, etc.), so as to use indicators such as accuracy, recall rate, F1 value, etc. to perform local quality assessment on specific dimensions. For example, by judging whether the generated text covers the key entities in the input, its performance in terms of information completeness, factual consistency, etc. can be evaluated. This method is suitable for the evaluation of structured attributes in quality standards. Among them, the binary or multi-valued judgment labels are constructed according to the preset evaluation criteria.

[0306] Calculation method 4: natural language generation evaluation indicators.

[0307] In generation tasks, metrics such as the Bilingual Evaluation Understudy (BLEU), the Recall-Oriented Unigram Generator Evaluator (ROUGE), and the Metric for Evaluation of Translation and Other Regenerated Output (METEOR) can be used to measure the similarity between the generated results and the reference text in terms of preset evaluation criteria, such as "semantic relevance, information completeness, and language coherence."

[0308] For example, for each candidate result in each candidate result set, the degree of match between it and the true label or expected target can be calculated based on the preset evaluation criteria in the embodiments of this application. This step can be achieved through an automated scoring algorithm, such as using the BLEU score to evaluate text similarity, or measuring the confidence of the generated task based on the probability value output by the model, etc., which will not be described in detail here.

[0309] It should be noted that the several matching degree calculation methods listed above are only simple examples. In addition, other matching degree calculation methods are also applicable to the embodiments of the present application. In addition, these matching degree calculation methods can be used individually or in combination, that is, multiple calculation methods are used to jointly evaluate the degree of match between each candidate result and the preset evaluation criteria, and through weighted fusion, averaging or other integration strategies, a more comprehensive and robust comprehensive score is obtained, thereby improving the accuracy and robustness of sample screening.

[0310] For each candidate result, its corresponding matching degree can be calculated using any one or more of the above methods. This allows us to filter out those within each candidate result set that highly match the pre-set evaluation criteria and accurately reflect the task objectives. These results typically have high matching scores and can serve as high-quality training samples, which we refer to as positive candidates in this article.

[0311] Furthermore, we identify and select candidate results that clearly do not meet the evaluation criteria and use them as counterexamples to enhance the robustness and generalization ability of the model. These results may be marked as negative due to low matching scores, misclassification, or other quality issues, which are referred to as negative candidate results in this paper.

[0312] Finally, the second examples in each candidate result set are combined with their corresponding positive and negative candidate results to form a complete second sample set. This sample set contains not only the original input second examples but also the filtered positive and negative candidate results, providing rich and diverse training data for subsequent model fine-tuning.

[0313] Optionally, the process also supports feedback and iterative optimization, that is, based on the preliminary screening results, analyze the problems and deficiencies in the screening process, adjust the evaluation criteria or optimize the screening algorithm, and then perform multiple rounds of iterative optimization until a satisfactory screening effect is achieved.

[0314] like Figure 5B As shown, it is a schematic diagram of a second sample set in an embodiment of the present application. Figure 5B The process related to S22 and S23 is illustrated, where different patterns represent second samples corresponding to different tasks. For example, samples from the topic generation sample subset, topic merging sample subset, and event generation sample subset are used as second samples and input into the intermediate mining model to obtain the candidate result set corresponding to each second sample, such as Figure 5B 51, 52, 53, etc. Based on the preset evaluation criteria, one positive candidate result and one negative candidate result can be screened out in each candidate result set. Each second sample and its corresponding positive candidate result and negative candidate result can form a triplet, such as shown in 54, which is a triplet of the example of the present application, wherein the positive candidate result is black smiley face 1 and the negative candidate result is black smiley face 4; for example, shown in 55 is another triplet of the example of the present application, wherein the positive candidate result is black five-pointed star 1 and the negative candidate result is black five-pointed star 2; for example, shown in 56 is another triplet of the example of the present application, wherein the positive candidate result is black sun 3 and the negative candidate result is black sun 5.

[0315] Through the above steps, S23 not only improves the quality of the candidate result set, but also enhances the adaptability and flexibility of the model in complex application scenarios. This is especially important for event context mining scenarios involving multiple tasks and requiring frequent model adaptation.

[0316] Among them, when screening high-quality positive candidate results and low-quality negative candidate results, in order to further optimize the screening process and ensure that the selected samples can more accurately reflect the expected standards, a method based on positive and negative matching degree thresholds is as follows:

[0317] Optionally, each candidate result included in a candidate result set is: a diversity of candidate results generated based on different generation strategies for the same second sample; the preset matching condition includes: a positive matching threshold and a negative matching threshold, and the positive matching threshold is greater than the negative matching threshold; then:

[0318] The positive candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample whose matching degree with the preset evaluation criteria is higher than the positive matching degree threshold;

[0319] The negative candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample, whose matching degree with the preset evaluation criterion is lower than the negative matching degree threshold.

[0320] Specifically, for each second sample, a diverse set of candidate results must first be generated based on different generation strategies (such as greedy search, beam search, Top-k sampling, temperature adjustment, etc.). This diversity is reflected not only in the differences in the output content in terms of vocabulary, sentence structure, and semantics, but also in the various generation logics and expressions reflected behind it. For example, greedy search tends to generate highly deterministic and well-structured results, while sampling strategies (such as Top-k or temperature adjustment) may introduce greater randomness and creativity, generating more diverse or divergent expressions; beam search strikes a balance between stability and diversity. By introducing a diverse set of candidate results, the possible output space of the task can be more comprehensively covered, improving the model's adaptability to different semantic expressions or language styles. At the same time, it also provides a richer selection basis for the screening of high-quality positive candidate results and representative negative candidate results in subsequent steps, helping to enhance the representativeness and discriminative power of the training data.

[0321] Then, based on the positive match threshold and the negative match threshold, we filter out one positive candidate result above the positive match threshold and one negative candidate result below the negative match threshold, respectively. During this process, we must ensure the accuracy and consistency of the match calculation method to ensure the effectiveness of the screening results.

[0322] In the embodiments of the present application, the positive match threshold and the negative match threshold can be set in a variety of ways according to actual needs. For example, it can be a pre-set fixed threshold, based on an empirical value obtained through statistical analysis of historical data; it can also be dynamically determined through a sorting mechanism, such as sorting the candidate results according to the match score, selecting the first several proportions of results as positive candidate results (such as the one ranked first), and the last several proportions of results as negative candidate results (such as the one ranked last); it can also be adaptively adjusted based on feedback information during the model training process, such as dynamically updating the threshold based on the performance of the validation set to improve the screening effect.

[0323] Furthermore, different threshold setting strategies can be employed for different task types. For example, topic generation tasks can focus on "information completeness" and "language fluency," setting higher thresholds to ensure good generalization of generated topics. Topic merging tasks should emphasize "semantic consistency" and "logical rationality," dynamically determining the boundaries between positive and negative candidate results through a ranking mechanism. Event generation tasks, on the other hand, focus on "information accuracy" and "relevance," using BLEU or ROUGE metrics to assist in threshold setting. This approach allows for more targeted evaluation of various tasks, enabling more flexible adaptation to evaluation needs and improving the accuracy and effectiveness of the second sample set construction.

[0324] By setting positive and negative match thresholds, the above implementation effectively screens out positive candidates that are highly consistent with the pre-set evaluation criteria and negative candidates that clearly don't, thereby ensuring the quality and diversity of the second sample set. This approach not only improves the effectiveness of model training but also enhances the model's generalization and robustness in complex scenarios.

[0325] Optionally, considering the importance of screening high-quality candidate result sets, in order to ensure that each candidate result included in the second sample set can accurately reflect the requirements of the preset evaluation criteria, this application also proposes a method of applying the evaluation criteria based on the above, by introducing quality standards and content preference standards set for candidate results, thereby making the screening process more accurate and helping to build a higher quality second sample set.

[0326] In an optional implementation, the preset evaluation criteria include: a quality standard set for the candidate results, and a content preference standard set for the candidate results.

[0327] Quality standards refer to the task-specific data requirements that candidate results must meet during the generation process, used to measure whether they meet the objectives of the specific task. These standards typically have clear functional orientations and structural constraints, and require specific screening rules to be set based on the task type. These can be determined through automated means or explicit rules. For example, quality standards include but are not limited to some or all of the following:

[0328] a. Topic generation task: Candidates must include key event arguments from the hot list words, intuitively suggest follow-up developments, and possess a certain degree of abstraction and generalization.

[0329] b. Topic merging task: The generated topic must be more abstract and top-level than the multiple input topics, reflecting the integration logic of different topics and covering events under all sub-topics, avoiding duplication or irrelevant content.

[0330] c. Event generation task: The generated event phrases must be as concise and accurate as possible, avoiding redundancy while ensuring factual consistency.

[0331] It should be noted that the several quality standards listed above are just simple examples. In addition, other quality standards are also applicable to the embodiments of this application. For example, in the event generation task, the candidate results may be required to maintain consistency between the event subject and behavior to avoid confusion in the subject-verb-object structure or semantic deviation; at the same time, the output format may be further required to be unified and the structure clear. For example, event phrases should uniformly adopt a verb-object structure or a subject-predicate structure to improve the standardization of the overall sample set and the efficiency of model learning.

[0332] Content preference criteria refer to the language normative requirements that candidate results should meet in terms of language expression, including but not limited to language fluency, grammatical correctness, logical consistency, information completeness, etc. This is a relatively common language quality assessment dimension that reflects whether the candidate results are closer to the aesthetic orientation, expression habits, and communication style of human language use. For example, content preference criteria include but are not limited to some or all of the following:

[0333] i. Logical rationality: Whether the generated content conforms to the logical relationship implied in the title (such as time sequence, cause and effect, etc.).

[0334] ii. Information accuracy: Whether the output fully expresses the key information in the title (such as location, time, people, etc.).

[0335] iii. Conciseness and fluency: whether the topic or event phrases are concise and whether the sentences are fluent.

[0336] iv. Troubleshooting for hallucinations: When annotating, focus on checking whether there is fictitious information in the generated content (such as incorrect details not mentioned in the title).

[0337] It should be noted that the several content preference standards listed above are just simple examples. In addition, other content preference standards are also applicable to the embodiments of the present application, such as: tone adaptability (whether the generated content conforms to the tone style of the target scenario, such as formal, colloquial, humorous, etc.), or expression diversity (avoiding repeated use of the same sentence patterns or vocabulary to improve language richness), etc., which can all be used as references for measuring whether the candidate results are in line with the user's language preferences.

[0338] In general, the quality standards in the embodiments of the present application are used to determine whether the candidate results meet the functional requirements and language correctness requirements under the task objectives, and the content preference standards are used to determine whether the candidate results are closer to human expression habits, aesthetic orientations or subjective preferences.

[0339] For example, when annotators manually annotate positive and negative candidate results, the quality standard is used to guide the annotation, while the content preference standard is a reference standard for manual annotation. An optional manual annotation method is:

[0340] To ensure that the content generated by the model better meets user needs, we first need to build a high-quality second sample set, also known as a preference feedback dataset or preference comparison dataset. For the three tasks requiring reinforcement learning alignment: topic generation, topic merging, and event generation, the specific construction process is as follows:

[0341] First, the model trained in the SFT stage (i.e., the intermediate mining model) is used to generate multiple candidate results for each input. For example, for the topic generation task, multiple possible topic phrases need to be sampled from the model output, and the event generation task requires sampling multiple event phrases. Moreover, each set of candidate results must have a certain degree of diversity, covering different generation logics and expressions.

[0342] Subsequently, a professional annotation team will compare the candidate results pairwise based on the content preference criteria and annotate the output that is more in line with human preferences to form a triple (x, y w ,y l ), where x represents the input content, i.e., the second sample; y w represents the positive candidate result that wins in the candidate result set corresponding to x, y l Indicates the negative candidate result that failed in the candidate result set corresponding to x.

[0343] Throughout the entire annotation process, quality standards are used to guide the annotation behavior of annotators and ensure that the candidate results meet the basic functional requirements of the task, while content preference standards serve as a reference for manual judgment and are used to measure the pros and cons of candidate results in terms of expression style, logic, and readability.

[0344] Taking the topic merging task as an example, the data examples that have been annotated by human preferences are as follows:

[0345] Input (topics to be merged): Huang AB deletes official announcement, Huang C officially announces his girlfriend, Huang AB makes their relationship public, Huang AB proposes.

[0346] Winning new topic: Huang AB romance;

[0347] New topic of defeat: Huang AB proposed to his girlfriend and officially announced her marriage.

[0348] Specifically, based on the above input, the intermediate mining model outputs multiple merged new topics as follows:

[0349] (1) Huang AB’s romance: It covers key information such as “Huang AB’s public romance” and “Huang AB’s marriage proposal”, while avoiding repeated mention of specific events, and is highly abstract and general.

[0350] (2) Huang AB proposed and officially announced his girlfriend: Mixing “Huang AB proposed” with “Huang C officially announced his girlfriend” resulted in confusing information and lack of logical coherence.

[0351] (3) Huang AB’s relationship and proposal: It contains the information of “Huang AB’s public relationship” and “Huang AB’s proposal”, but the expression is lengthy and not concise enough, and the core theme is not fully extracted.

[0352] (4) Huang AB’s public relationship and proposal: Similar to “Huang AB’s relationship and proposal”, but the wording is a bit cumbersome and fails to reflect a higher level of abstraction and generality.

[0353] (5) Events related to the relationship between Huang AB and Huang C: The article mentioned “Huang C officially announced his girlfriend”, but failed to highlight the core events of the main character “Huang AB”, thus deviating from the main content focus.

[0354] The winning new topic was "Huang AB Love," which best met the content preference criteria. It concisely and clearly covered all information related to "Huang AB" relationships, while maintaining logical consistency and completeness. It not only possessed a high degree of abstraction and generalization, but also avoided unnecessary cluttering with details, making the presentation more concise and intuitive.

[0355] The losing new topic was "Huang AB proposed and officially announced his girlfriend." This topic had obvious logical errors and information confusion. It forcibly combined two unrelated events, "Huang AB proposed" and "Huang C officially announced his girlfriend," resulting in output content that was inconsistent with the facts and lacked logical rationality. Therefore, it was judged as the worst candidate result.

[0356] In this way, the second stage of training can also be reinforcement learning with human feedback (RLHF) training, which means using preference data from human feedback and using a reinforcement learning algorithm to train the model.

[0357] The above examples demonstrate how to filter out the best (positive candidates) and worst (negative candidates) topics from multiple merged candidate topics. This comparison helps the model learn how to generate more reasonable, concise, and human-friendly merged topics.

[0358] When executing S23, an optional automatic marking method is as follows S231 to S233 ( Figure 2 not shown):

[0359] S231: For each candidate result set, a candidate result in the candidate result set that meets the quality standard and has a matching degree with the content preference standard higher than a positive matching degree threshold is selected as a positive candidate result corresponding to the second sample in the candidate result set.

[0360] In this step, for each candidate result set, the server can screen out a result that meets the quality standards and has a matching degree higher than the preset positive matching degree threshold under the content preference standard from the candidate results generated by multiple models, as the positive candidate result corresponding to the candidate result set. Among them, the quality standard determines whether the candidate result is qualified. Only the candidate results that pass the quality standard screening will be further sorted and optimized according to the content preference standard to select the high-quality output that is closest to the user's preference, that is, the content preference standard is used to further screen out more popular results from multiple qualified results. The two complement each other, not only ensuring the technical availability of the candidate results, but also improving their acceptability and user experience in practical applications.

[0361] S232: For each candidate result set, a candidate result in the candidate result set whose matching degree with the content preference standard is lower than the negative matching degree threshold is selected as a negative candidate result corresponding to the second sample in the candidate result set.

[0362] Among them, this article does not make specific restrictions on whether the negative candidate results meet the quality standards. For example, the negative candidate results can also meet the quality standards, that is, a candidate result in a candidate result set that meets the quality standards and has a matching degree with the content preference standard lower than the negative matching degree threshold is used as a candidate result set to match the negative candidate result corresponding to the second sample.

[0363] That is, in this step, for each candidate result set, the server selects a result from multiple candidate results whose matching degree with the content preference standard is lower than the preset negative matching degree threshold as the negative candidate result corresponding to the candidate result set. This candidate result may or may not meet the quality standard, but is significantly inferior to other results in terms of content preference standards such as expression style, semantic coherence or user acceptance. If only the preference dimension is considered, the results that meet the quality standard but have the lowest preference matching degree can be selected as negative samples to enhance the validity and comparability of the training data; if it is necessary to improve the model's ability to recognize low-quality outputs, some candidate results that do not fully meet the quality standard can also be introduced as negative candidate results.

[0364] S233: Combining the second samples corresponding to each candidate result set, and the corresponding positive candidate results and negative candidate results to form a second sample set.

[0365] In this step, the input samples, positive candidate results and negative candidate results corresponding to each candidate result set are combined into a triple (x, y w ,y l ), and then a structured second sample set is formed after unified organization, which is used in the subsequent reinforcement learning training stage.

[0366] Optionally, the sample set can be format-standardized and deduplicated, cleaned, and scaled according to actual task requirements to ensure the stability and generalization ability of the training process.

[0367] The second sample set finally constructed not only contains high-quality and diverse sample pairs, but also reflects the guiding role of human preferences on model output. It is an important basis for achieving model optimization based on human feedback.

[0368] The execution order of S231 and S232 is not specifically limited herein.

[0369] In this method, quality standards and content preference standards for candidate results are first defined. For each candidate result set, by comparing the degree of match between the candidate results and these standards, a positive candidate result that meets the quality standards and whose match with the content preference standard is higher than the positive match threshold is screened out; at the same time, a negative candidate result whose match with the content preference standard is lower than the negative match threshold is screened out. Finally, the second samples corresponding to each candidate result set and their corresponding positive and negative candidate results are combined to form a second sample set. This process requires ensuring the consistency of the standards and the accuracy of the match calculation to ensure the reliability and effectiveness of the screening results.

[0370] The following is a simple example using topic generation tasks and event generation tasks as examples:

[0371] In the topic generation task, the input is a trending word or headline, such as "Huang AB officially announced their relationship and then deleted their updates." Based on this input, the model generates multiple candidate topics, such as "Huang AB officially announced their relationship," "Huang AB deleted their updates," "Huang AB's love life review," and "Huang AB's relationship progress with his rumored partner."

[0372] Then, the server first determines whether each candidate topic is qualified based on quality standards, such as whether the key information in the input is fully expressed (such as the subject "Huang AB", the behavior "official announcement of relationship", "deletion of dynamics"), whether there are factual errors or semantic gaps, etc.

[0373] After passing the quality screening, the content preference criteria evaluation phase begins. The server calculates the degree of match between each candidate topic and the preference criteria, such as whether it is concise, closely aligned with user reading habits, and possesses a certain degree of abstractness and generalization. Ultimately, if "Huang AB officially announces their relationship" exceeds the positive match threshold, it is selected as a positive candidate. However, "Huang AB's relationship progress with the rumored partner" introduces unmentioned information and deviates from the topic, resulting in a match below the negative match threshold and is therefore selected as a negative candidate.

[0374] In the event generation task, the input is a set of event description fragments, such as "Zhang XX attended a charity dinner," "Zhang XX gave a speech," and "Zhang XX took a photo with Li XX." The model generates multiple candidate event phrases, such as "Zhang XX attended a charity dinner and gave a speech," "Zhang XX took a photo with Li XX," "Zhang XX participated in a public welfare activity," and "Zhang XX went out late at night and attracted attention."

[0375] The server first determines, based on quality criteria, whether the candidate event accurately reflects the original information and whether key entities, time, and location are omitted. For example, the event "Zhang XX's late-night outing attracted attention" was deemed to be non-compliant and was removed because it included unmentioned information.

[0376] Candidates that meet the quality standards are then sorted based on content preference criteria. For example, "Zhang XX attended a charity dinner and delivered a speech" is selected as a positive candidate due to its concise language and complete information, with a preference match above the positive match threshold. However, "Zhang XX participated in a charity event" is selected as a negative candidate due to its overly general information and lack of details, with a preference match below the negative match threshold.

[0377] From the above two task examples, it can be seen that the screening process of candidate results combines the two dimensions of quality control and preference guidance, ensuring that the final constructed second sample set has both technical usability and good user acceptance, thereby providing high-quality training data support for subsequent target mining model training.

[0378] Optionally, different quality standards and content preference standards can be formulated for different task types to ensure that the dual standards of quality and preference of candidate results are effectively applied in the actual annotation process, for example:

[0379] a. Topic generation task.

[0380] Quality standards: The output must be grammatically correct and coherent; key event arguments in the title (such as subject, behavior, object) must not be omitted; and false information not mentioned in the title must be avoided.

[0381] Content preference criteria: Candidate results should be abstract and general; should reflect the natural continuity of events (intuitively have subsequent progress); prefer clear, concise and information-dense expressions.

[0382] b. Topic merging task.

[0383] Quality standards: The merged topic must cover the core elements of all sub-topics (referring to the topics to be merged); key information of any input topic must not be omitted; there should be no obvious duplication or irrelevant content.

[0384] Content preference criteria: The merged result should be more abstract and general than the original topic; the expression should be more in line with natural language habits; and it should be more inclined to reflect the integration logic and relevance between different topics.

[0385] c. Event generation task.

[0386] Quality standards: The output should accurately reflect the core information of the event; avoid adding unmentioned facts or details; the sentence structure should be concise and clear to avoid ambiguity.

[0387] Content preference criteria: prefer concise and focused event descriptions; among multiple candidate results, give priority to results with a unified language style and smooth logic; and are more in line with the target users' acceptance of event description methods (such as news style, colloquial language, etc.).

[0388] In addition, in addition to the quality standards set for candidate results and the content preference standards set for candidate results, the preset evaluation criteria may also include some other auxiliary evaluation criteria, such as format standardization standards: used to evaluate whether the candidate results meet the preset output format requirements, such as whether the specified structure is adopted, whether the punctuation is used in a standardized manner, whether illegal characters are included, etc., which are suitable for task scenarios with clear constraints on the model output form; for example, diversity control standards: used to measure the level of diversity of candidate results in semantics or expression, to avoid high duplication or convergence of multiple candidate results; and so on.

[0389] These supplementary standards can be flexibly introduced according to actual task requirements to further improve the comprehensiveness and practicality of candidate result screening. These are just simple examples. Other supplementary standards are also applicable to the embodiments of this application and will not be described in detail here.

[0390] In summary, the above implementation further refines the pre-set evaluation criteria. By combining quality and content preference criteria to screen candidate results, it can more accurately identify high-quality positive candidate results that meet expectations and low-quality negative candidate results. This not only improves the overall quality of the second sample set but also optimizes the data foundation for subsequent model training, significantly improving the performance of the final model.

[0391] S24: Training the intermediate mining model based on the second sample set to obtain a target mining model.

[0392] In an embodiment of the present application, the second sample set is constructed in the above manner and includes samples related to the task content corresponding to at least one task (referred to as second samples). These tasks require further preference enhancement. Like the first sample, this also refers to the input information required for the task. Please refer to the above embodiment for details and will not be repeated here.

[0393] In this step, the server performs preference alignment training on the intermediate mining model after SFT based on the second sample set to obtain the final target mining model. This training process can adopt the RLHF framework and can include two stages: reward model training and strategy optimization:

[0394] First, the preference comparison data in the second sample set is used to train the reward model. This model is used to measure the degree of match between candidate outputs and human preferences. Its training goal is to make the reward value of positive candidate results higher than that of negative candidate results, thereby establishing the ability to rank high-quality outputs.

[0395] Subsequently, a reinforcement learning algorithm is used to optimize the intermediate mining model's strategy, using the trained reward model as an external feedback signal. During this process, the model continuously adjusts its generation strategy to maximize the score of the generated output in the reward model, gradually approaching an output style and expression habits that better suits user preferences.

[0396] Among them, the reinforcement learning algorithms involved in the embodiments of the present application include but are not limited to: Proximal Policy Optimization (PPO), Trust Region Policy Optimization (TRPO) and Direct Preference Optimization (DPO), etc. Among them, PPO ensures training stability during the policy update process by introducing a clipping mechanism; TRPO controls the update step size based on strict mathematical constraints, which is the theoretical basis of PPO; and DPO, as an alternative method that does not require an explicit reinforcement learning process, can directly optimize the model based on human preference data, simplifying the traditional RLHF process. The above algorithms can all be used for the training process of the intermediate mining model based on the second sample set, and the specific implementation method can be flexibly selected according to the task requirements.

[0397] Through this step, the target mining model not only has basic language understanding and generation capabilities, but also achieves a high degree of alignment with user subjective preferences. It can output high-quality content that is more in line with actual application scenarios and more acceptable in tasks such as topic generation, topic merging, and event generation.

[0398] To improve the model's adaptability to human preferences, the present embodiment proposes an iterative training method when implementing S24. This method compares the probability distribution differences between the positive and negative candidate results output by the model in each round of training and optimizes the model parameters accordingly, thereby gradually approaching an output strategy that is more in line with user preferences. An optional implementation method is as follows:

[0399] The intermediate mining model is iteratively trained based on the second sample set; each round of training performs the following steps S241 to S242 ( Figure 2 not shown):

[0400] S241: Input the second sample selected from the second sample set in this round into the intermediate mining model and the model to be trained in this round respectively, and obtain a set of positive probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding positive candidate results, and a set of negative probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding negative candidate results.

[0401] During each round of training, one or more second samples are first randomly selected from the second sample set as the input data of the current batch. The sample selection method can be random sampling or other sample selection strategies mentioned above, which will not be described here one by one.

[0402] Specifically, in each round of training, the server randomly selects a batch of samples from the second sample set and inputs them into the current intermediate mining model and the model to be trained in this round. In the first round of training, the model to be trained in this round is the initial intermediate mining model itself, which has not yet undergone preference alignment optimization.

[0403] In subsequent training rounds, the model to be trained uses the output model from the previous training round as its starting point, employing the updated parameters from the previous round for a new round of inference and training. This approach creates a continuous model evolution process, enabling training to gradually approach output strategies that better align with human preferences while maintaining stability and monitorability.

[0404] Through this mechanism, the server can achieve continuous optimization based on the historical status of the current model without relying on external pre-trained models, thereby improving the generalization ability and user satisfaction of the target mining model in practical applications.

[0405] For each input sample, the two models generate corresponding positive and negative candidate results and output corresponding generation probabilities. These probabilities reflect the model's preference for different candidate results, providing a quantitative basis for subsequent parameter adjustments, thereby guiding the model to gradually approach an output strategy that better aligns with human preferences.

[0406] In an embodiment of the present application, if the mining model is a large language model, when the model generates a complete candidate result (such as a sentence or phrase), it will predict the next token one by one in a serialized manner. For each token, the model will output a probability distribution based on the current context, indicating the probability of each token that may appear at that position. In order to further clarify the probability calculation mechanism of the model when generating candidate results, the embodiment of the present application also provides a specific calculation method for the probability of candidate result generation, namely:

[0407] The probability of a mining model outputting a candidate result is the product of the output probabilities corresponding to each token in the candidate result.

[0408] A mining model can be an intermediate mining model or the current round of training; a candidate result can be a positive candidate result or a negative candidate result. This unified probabilistic modeling approach ensures good consistency and comparability between the probabilities of different models and different categories of candidate results.

[0409] Specifically, when a mining model (including the intermediate mining model or the model to be trained in this round) outputs a candidate result, its corresponding probability is the product of the probabilities of each word in the candidate result generated in sequence, that is, the product of the conditional probabilities of all words in a sequence at their respective positions.

[0410] For example, when generating the candidate result "Huang AB officially announced their relationship", the model will output the words "Huang", "AB", "official announcement", "relationship" and so on in sequence, and calculate the conditional probability of each word in the context. Finally, these conditional probabilities are multiplied together to obtain the joint probability of the entire candidate result.

[0411] During the training process, the intermediate mining model is used to provide a reference probability distribution for the current round, and the model to be trained in this round generates corresponding candidate results based on the same input sample and outputs the corresponding probability value. For each input x in the second sample, the two models generate corresponding positive candidate results y w and negative candidate y l , and calculate the corresponding joint probability.

[0412] Subsequently, the differences between the two sets of probabilities, such as the ratio, Kullback-Leibler (KL) divergence, cross entropy, or Jensen-Shannon (JS) divergence, can be compared to guide the parameter update of the current round of training model.

[0413] This mechanism is applicable to the generation process of all candidate results. Whether they are positive or negative candidate results, probability modeling can be performed in the same way, thereby ensuring the consistency and comparability of training data and improving the stability of model training and preference alignment effects.

[0414] In the above embodiment, by defining the probability of a candidate result as the product of the generation probabilities of its various word units, the embodiment of the present application provides a unified and highly interpretable probabilistic modeling method, which helps to improve the model's ability to understand language structure, enhance the controllability and evaluability of the generated results, and thus improve the overall training effect and output quality.

[0415] S242: According to the difference between a set of positive probabilities and a set of negative probabilities, the parameters of the model to be trained in this round are adjusted to obtain the output model of this round.

[0416] Specifically, in this step, the server calculates the difference in generation probabilities between positive and negative candidate results in the two models, constructs a target loss function, and uses optimization methods such as backpropagation to update the parameters of the current model to be trained, allowing the model to gradually learn an output strategy that better aligns with human preferences. During the update process, optimizers (such as AdamW), learning rate scheduling strategies, and gradient clipping can also be introduced to improve training stability and convergence speed.

[0417] Among them, the loss function calculated during the training process can be an objective function based on ranking loss or KL divergence, or other objective functions suitable for preference alignment training, such as the binary cross entropy loss used by Direct Preference Optimization (DPO) or the squared error loss used by the similar Identity Preference Optimization (IPO), the clipped surrogate objective in Proximal Policy Optimization (PPO), or other forms of ratio-based loss functions.

[0418] These loss functions can be used to measure the relative preference differences between positive and negative candidate results in the model output, guiding model parameter updates accordingly, so that the generated results gradually approach output strategies that are more consistent with human preferences. Compared to traditional reinforcement learning methods, DPO and IPO both have advantages such as not requiring explicit training of reward models and a more stable training process, making them particularly suitable for the preference alignment task of large-scale language models.

[0419] It should be noted that the loss functions listed above are just simple examples. Different loss functions have their own characteristics in terms of training stability, implementation complexity, and efficiency in utilizing preferred data. In practical applications, they can be flexibly selected according to specific task requirements. This article will not go into details one by one.

[0420] In an embodiment of the present application, in the process of adjusting the parameters of the model to be trained in this round, the learnable parameters of the model (such as the weight matrix and bias term in the neural network layer, the query (Q), key (K), value (V) projection parameters in the attention mechanism, the scaling factor and offset factor of the normalization layer, etc.) are gradient updated to directly optimize the probability distribution difference between the positive candidate results and the negative candidate results of the model, thereby adjusting the model preference distribution. These model parameters constitute the network structure of the model to be trained in this round and directly affect its input-output mapping relationship.

[0421] Optionally, some other parameters besides the model parameters may be adjusted during the training process, such as learning rate configuration parameters, optimizer state parameters, task adaptation head parameters, etc. For details, please refer to the above embodiments and will not be repeated here.

[0422] After multiple rounds of iteration, the model is more inclined to generate high-quality output consistent with human preferences, thereby obtaining the output model after this round of optimization. The stopping conditions of the iterative training process at this stage can also be referred to in the above embodiment and will not be repeated here.

[0423] In summary, the iterative training method provided in the embodiments of this application effectively improves the model's ability to adapt to human preferences by continuously comparing the probability distribution differences between the model's positive and negative candidate results and continuously optimizing the model accordingly. Compared to a one-time training approach, this method is more adaptable and stable, gradually guiding the model to output content that better meets user expectations, significantly enhancing the model's generalization ability and interactive experience in real-world application scenarios.

[0424] To further enhance the ability to characterize preference differences during model training, the present embodiment further refines the specific calculation method of positive and negative probabilities based on the above. The specific implementation method is as follows:

[0425] In each round of training, the intermediate mining model and the model to be trained in this round each output a positive candidate result and a negative candidate result corresponding to a sample, and generate corresponding probabilities for each result. A set of positive probabilities includes: the positive baseline probability output by the intermediate mining model and the positive prediction probability output by the model to be trained in this round; a set of negative probabilities includes: the negative baseline probability output by the intermediate mining model and the negative prediction probability output by the model to be trained in this round.

[0426] The server constructs a loss function based on probability ratio by comparing the ratio of positive prediction probability to positive baseline probability, and the ratio of negative prediction probability to negative baseline probability, so as to measure the degree of deviation of the current round of training model relative to the intermediate mining model in the preferred direction, and adjust the model parameters accordingly.

[0427] Therefore, when executing S242, an optional implementation method is as follows S2421-S2422 ( Figure 2 not shown):

[0428] S2421: Obtain the positive probability ratio between the positive prediction probability and the positive baseline probability, and the negative probability ratio between the negative prediction probability and the negative baseline probability.

[0429] In this step, the server mines the model (denoted as π) from the middle for the current second sample x. ref ) and the model to be trained in this round (denoted as πθ ) to obtain the generated positive candidate result y w and negative candidate y l Specifically:

[0430] For the positive candidate results, the positive baseline probability output by the intermediate mining model is recorded as π θ (y w |x), the forward prediction probability output by the model to be trained in this round is recorded as π ref (y w |x); Similarly, for negative candidate results, the negative baseline probability output by the intermediate mining model is π θ (y l |x), the negative prediction probability of the output of the training model in this round is π ref (y l |x).

[0431] The server then calculates the positive probability ratio and the negative probability ratio These two ratios reflect the degree of deviation of the current round of training model relative to the intermediate mining model in the preferred direction.

[0432] S2422: Adjust the parameters of the model to be trained in this round based on the difference between the positive probability ratio and the negative probability ratio.

[0433] In this step, after obtaining the positive and negative probability ratios, the server constructs the target loss function based on these ratios and updates the parameters of the model to be trained in this round accordingly.

[0434] Optionally, when constructing the target loss function, the loss function can include two parts: one is to encourage the model to increase the probability of generating positive candidate results (i.e. The larger the better), and secondly, suppress the model's preference for negative candidate results (i.e. The smaller the better). This can enhance the model's ability to distinguish between preferred and non-preferred samples through bidirectional guidance, thereby more effectively aligning with human preferences and improving the model's output quality and user satisfaction in practical applications.

[0435] Through the above process, the model can gradually learn an output strategy that is more in line with human preferences, making the generated results better in terms of quality and user acceptance.

[0436] The above design principles are fully reflected in the preference alignment training based on human feedback, especially when directly using the Determined Proportional Opportunity (DPO) algorithm for RLHF alignment. DPO directly optimizes the model's output distribution on the preference data by constructing a binary loss function based on the probability ratio of positive and negative candidate results, thereby learning and modeling generation strategies that better align with human preferences.

[0437] The following takes the tasks corresponding to the second sample as the topic generation task, topic merging task, and event generation task as an example, and the specific implementation methods are as follows:

[0438] In the process of RLHF alignment using the DPO algorithm, the preference distribution of the model is directly optimized, with the goal of making the model give a higher probability to the output that is more in line with human preferences, thereby adjusting the model generation strategy. Let the model after multi-task SFT be π ref In this application, the three sets of data under the three tasks that have been labeled by human preferences are randomly shuffled as training data for the DPO algorithm. For each set of comparison data (x, y w ,y l ), the optimization objective loss function is:

[0439]

[0440] Among them, π θ Represents the model to be trained in this round, whose parameter is θ. The parameter θ is optimized by gradient descent, and the goal is to make the preference response y w The probability of y l Higher. ref Represents the intermediate mining model, specifically the model before the DPO algorithm is aligned, that is, the model version after SFT. Its parameter is ref, and its parameters are frozen to avoid training instability. Its role is to provide a baseline probability distribution for the training data to prevent the strategy from deviating too far from the original capability. w represents the winning sample in human preference annotation, i.e., the positive candidate result, y l Indicates the failed sample, that is, the negative candidate result.

[0441] Here, σ is the sigmoid function, which maps the input to the interval [0, 1]. β is the temperature coefficient, which controls the degree to which the model deviates from the reference policy. Its function is to balance the deviation of the model optimization from the reference policy. A small β (such as 0.1-0.5) limits the model update amplitude and prevents overfitting; a larger β allows for more aggressive optimization. E represents the expected value of the sample from the distribution.

[0442] The calculation steps of this loss function are briefly summarized as follows: for each second sample, first calculate two ratios: the logarithmic form of the positive probability ratio and the logarithmic form of the negative probability ratio; then calculate the weighted difference of these two ratios, with the weight β; then, convert the above difference into a probability value between 0 and 1 using the Sigmoid function; finally, calculate the loss value corresponding to the second sample and find the expected value for all second samples. Among them, all second samples here can refer to each sample randomly selected from the second sample set in the current round of training, or historical samples accumulated from multiple training rounds (optionally used to improve training stability or alleviate data bias), and are not specifically limited here.

[0443] The goal of this loss function is to increase the probability of winning samples generated by the model relative to the baseline model, while reducing the probability of losing samples generated, thereby achieving consistency optimization with human preferences.

[0444] It should be noted that the loss function listed in the above formula 3 is just a simple example. Other loss function calculation methods are also applicable to the embodiments of this application and will not be described in detail here.

[0445] In the above embodiment, by introducing the probability ratio of positive and negative candidate results as the basis for model parameter adjustment, the embodiment of the present application can more finely characterize the response differences of the model to the preference samples during the training process, thereby more accurately capturing its evolution trend in the dimension of human preference. Compared with the traditional absolute probability optimization method, the modeling method based on probability ratio can effectively suppress excessive fluctuations in the model output, enhance the stability of the training process, and accelerate the convergence of the model. The target mining model finally obtained is not only more consistent in generation quality, but also closer to the subjective preference standards of human users, which significantly improves the usability and interactive experience of the model in practical applications.

[0446] In summary, taking the large language model as an example, Figure 6 As shown, it is a model training logic block diagram in an embodiment of the present application. The present application uses multi-task supervised fine-tuning (SFT) and reinforcement learning based on human feedback (RLHF) to train the model in stages. First, preliminary alignment is performed on multiple tasks through SFT; for generation tasks (such as topic generation, topic merging and event generation), the candidate samples sampled by human feedback are further combined for reinforcement learning alignment to make their output more in line with human preferences, where loss represents negative candidate results and win represents positive candidate results. The entire process can be completed independently by a single 7B-scale LLM for all subtasks, which significantly improves the model's understanding and generation capabilities in event context mining scenarios while reducing deployment costs, thereby improving mining efficiency and result quality.

[0447] The target mining model trained in this way can efficiently and accurately complete multiple key sub-processes in the event context mining task. By jointly modeling multi-task information, this model possesses strong semantic understanding and logical reasoning capabilities. It can automatically identify representative thematic clues in massive amounts of text data and organize related events into clearly structured and temporally coherent event evolution paths.

[0448] Optionally, after obtaining the target mining model, the model can be dynamically optimized and iteratively updated based on user feedback information during the actual application process. For example, when users propose modifications or annotation corrections to the event context structure output by the model, the server can incorporate these feedback data into the training process and use online learning or incremental training to fine-tune the model parameters, thereby continuously improving the accuracy of the model output results and user satisfaction. In addition, the A / B testing mechanism can be combined to evaluate the performance differences of different versions of the model, or a reinforcement learning strategy can be introduced to use behavioral data such as user click-through rate and dwell time as reward signals to further optimize the relevance and readability of the content generated by the model. Through continuous data feedback and model iteration, it is ensured that the target mining model can adapt to the ever-changing information environment and diverse user needs.

[0449] In addition, the target mining model also supports real-time dynamic updating and expansion of newly emerging information, thereby realizing continuous monitoring and intelligent analysis of event context, and providing solid technical support and data foundation for subsequent application scenarios (such as news aggregation, knowledge graph construction, etc.).

[0450] In summary, this application designs a solution for fully automatic mining of hot event contexts based on a single large model. On this basis, the target mining model trained by this application can be deployed in the context mining system to build a complete context mining system, including persistent storage, retrieval, model interaction and other channels for context topics, node data, etc. In the process of event context mining, multiple types of natural language processing and information retrieval tasks are involved, including but not limited to text multi-classification (such as event topic mounting, key event node identification, event node deduplication), text generation (such as topic generation, topic merging, event generation), recall sorting (such as information retrieval related content below) and other task forms. The system can be used to identify topics, generate events, mount related information from massive news, social media and user input in real time, and dynamically construct a clearly structured and semantically coherent event evolution path to provide efficient support for application scenarios such as news aggregation and event analysis.

[0451] Next, the event context mining process based on the context mining system in this application is described in detail:

[0452] Optionally, the system introduced in this application includes at least four NLP tasks: topic generation, topic merging, event generation, and event topic mounting, among which the generation task includes topic generation, topic merging, and event generation; and the decision task includes event topic mounting.

[0453] Still taking the server (which can be regarded as a context mining system, or a part of the context mining system) as the execution subject as an example, the server in the embodiment of the present application is also used to execute the following process through the target mining model to realize event context mining, wherein:

[0454] In the topic generation sub-process, the target mining model generates initial topics based on the topic information source;

[0455] In the topic merging sub-process, the target mining model merges the initial topic with the existing topics in the existing topic set, and updates the existing topic set to expand the scope of events covered by the existing topics in the existing topic set;

[0456] In the event generation sub-process, the target mining model generates events based on the event information source;

[0457] In the event topic mounting sub-process, after constructing prompt information based on the event, the summary of the event information source and at least one recall topic, the target mining model attributes the event to the corresponding recall topic based on the prompt information to construct the event context; wherein, at least one recall topic is an existing topic recalled from the existing topic set, and its similarity with the event meets the preset similarity condition.

[0458] Among them, the summary extraction of event information sources can adopt a dynamic summary extraction algorithm, such as TextRank based on text graph ranking, BERT Extractive Summarization model based on semantic representation, or end-to-end Transformer sequence model, etc., which are not specifically limited here.

[0459] For example, the server first semantically encodes the input text, identifying key sentences or core information units, and then uses a scoring mechanism to select the most representative sentences to form a summary. The server can also improve the quality of the summary by combining factors such as keyword weight, sentence position, and contextual relevance to ensure that the output content is concise and retains the key elements of the event.

[0460] Specifically, the analysis of each task in this application is as follows:

[0461] a) Topic generation aims to generate topics based on topic information sources, such as popular words on the Internet, and allow the target mining model (such as LLM) to generate more general topics with follow-up progress through judgment and filtering in an end-to-end manner.

[0462] Specifically, in the topic generation sub-process, the server receives topic information sources from channels such as news headlines, social media hot searches, and user input, and the target mining model automatically generates representative initial topic tags based on semantic understanding. To improve the accuracy and coverage of topic generation, the original information source can optionally be pre-processed in conjunction with keyword extraction algorithms or clustering methods to assist the model in more accurately identifying potential topics. This process outputs one or more initially extracted "initial topics" as the basic units for subsequent information organization.

[0463] Among them, keyword extraction algorithms include but are not limited to term frequency-inverse document frequency (TF-IDF), TextRank, etc., which will not be detailed here.

[0464] b) Topic merging: Each time the target mining model generates a new topic, the server retrieves existing topics from the existing topic set and allows the target mining model to merge the new topic with historically similar topics, thereby continuously abstracting the topic hierarchy, transferring topics from lower-level events to higher-level events, and expanding the scope of events covered by the topic.

[0465] Specifically, in the topic merging sub-process, the server compares the currently generated initial topic with the existing topics in the existing topic set by semantic vector matching (for example, using cosine similarity calculation). If the semantic similarity between the two exceeds the preset threshold, they are considered to belong to the same topic category (i.e., historically similar topics), and then the merge operation is performed. During the merging process, the server will update the set of event nodes covered by the existing topic (which can be obtained through event topic mounting) so that it can cover more related events, thereby continuously expanding the information boundary of the topic. Finally, the merged new topic is added to the existing topic set, and the updated existing topic set is output, in which each topic has a wider range of event coverage capabilities, which improves the consistency and scalability of the server. Among them, there can be one or more historically similar topics corresponding to an initial topic.

[0466] like Figure 7 As shown, it is a topic merging logic block diagram in an embodiment of the present application. For example, the initial topic currently generated is topic_2, and there are historical similar topics retrieved from the existing topic set, such as topic_1; next, topic_1 and topic_2 are merged to generate a new topic, namely topic_12, and topic_12 is added to the existing topic set. In addition, the event node set (referred to as node set) can also be updated, such as Figure 7The node set [node 1, node 2, node 3] of topic_1 is merged with the node set [node 4, node 5] of topic_2 to generate the node set [node 1, node 2, node 3, node 4, node 5] of topic_12. In addition, you can delete the merged topic_1 and its corresponding node set from the existing topic set.

[0467] When merging sets of event nodes related to similar topics, one can simply take the union or intersection to fuse the information. Alternatively, a weighting mechanism can be introduced to sort and filter nodes based on event time, source credibility, or semantic similarity, retaining key events and removing redundant information. Furthermore, graph modeling methods can be combined to organize event nodes into a directed time-series graph, making the merged topic context more logical and interpretable, thereby improving the quality of the expression of the event evolution path. This is not specifically limited here.

[0468] In addition, it should be noted that if there is no topic similar to the newly generated topic in the existing topic set, the newly generated topic can be directly stored in the existing topic set.

[0469] c) Event generation: News article titles are often not standardized and contain a lot of noise. This task is based on the title, text, or other event information sources of news articles, and uses the target mining model to generate concise event expressions that are more in line with the expression habits of hot events and improve the quality of context nodes.

[0470] Specifically, in the event generation sub-process, the server can construct input prompts based on the event information source (such as news headlines, news text, timestamp data, location descriptions, character relationships, etc.) to guide the target mining model to generate a structured event description. The output content includes but is not limited to key elements such as the time, location, main characters, and source of the event to ensure the integrity and standardization of the event information. To enhance the consistency and parsability of the output format. Optionally, a template filling mechanism or a structured decoding strategy can be used to ensure that the generated results meet the usage requirements of downstream tasks (such as event context construction and visual display). The output of this process is an event, specifically a description related to the event. Of course, it can also be converted into standardized event node data before output, providing basic support for subsequent event classification and context organization.

[0471] d) Event topic attachment: Based on the events in the news article, similar topics are recalled from the existing topic set. Using the events and summaries of the news article, the target mining model attaches the events to the appropriate topics. Compared with existing solutions, this model can leverage its powerful text understanding and event association capabilities to perform more accurate attachment.

[0472] Specifically, in the event topic attachment sub-process, the server first constructs prompt information containing the event itself, the event summary, and related topics recalled from the existing topic set. The target mining model then determines which topic the event should belong to based on the prompt and attaches it to the corresponding topic. This process not only relies on the model's language understanding capabilities, but also allows the introduction of additional classifiers or scoring mechanisms to further improve the accuracy of event attribution. Through this process, the server can achieve dynamic association between events and topics, gradually building a clearly structured and hierarchical event context system, laying the foundation for subsequent key event identification, deduplication processing, and context visualization.

[0473] For example, a user enters a trending search: "Celebrity A and Celebrity B officially announce their divorce." In the topic generation subprocess, the model generates an initial topic tag based on the input information: "Celebrity divorce." In the topic merging subprocess, the server discovers the existing topic "Entertainment Celebrity Relationship Changes" and, through semantic similarity analysis, determines that the newly generated topic is highly related to the existing one. The server then merges the two into a broader topic: "Celebrity Marital Changes."

[0474] In the event generation sub-process, enter the specific event information source: "On January 1, 2023, celebrity A posted a message announcing the official end of her marriage with celebrity B." Based on this information, the model generates a structured event node: "On January 1, 2023, celebrity A and celebrity B officially announced the end of their marriage."

[0475] In the event topic attachment subprocess, a prompt is constructed, including an event summary and relevant results retrieved from the "Celebrity Marital Changes" topic. Based on this prompt, the model determines that the event belongs to the "Celebrity Marital Changes" topic and attaches it to that topic, further enhancing the event context.

[0476] The above implementation method achieves automated identification and organization of potential topics and events within massive amounts of information by establishing multiple sub-processes, including topic generation, topic merging, event generation, and event topic attachment. Compared to traditional manual organization or single-module processing methods, this solution not only significantly improves the efficiency and breadth of information processing, but also enhances the ability to understand semantic information and integrate multi-source heterogeneous data through a model-driven approach, thereby constructing a clearly structured and logically coherent event context system, providing a solid foundation for subsequent information retrieval, trend analysis, and decision support.

[0477] To efficiently identify topics and events within large amounts of heterogeneous information and build a clearly structured and logically sound event context system, the present application also provides a full-process processing mechanism that includes multiple tasks, including topic generation, event generation, event topic mounting, key event node identification, and event node deduplication. These sub-processes work together to support the intelligent operation of the event context mining server, thereby achieving efficient organization and presentation of complex information.

[0478] In an optional implementation, the decision-making task introduced in this application also includes two NLP tasks: key event node identification and event node deduplication. After the event topic mounting sub-process, the server can also execute the following sub-process through the target mining model:

[0479] In the key event node identification sub-process, when the target mining model identifies that an event attached to a topic belongs to a key event node related to a topic, the event is included in the event context corresponding to the topic;

[0480] In the event node deduplication sub-process, the target mining model performs semantic deduplication operations on each event node under the same topic to remove redundant event nodes.

[0481] In the embodiment of the present application, the key event node identification sub-process and the event node duplication removal sub-process for optimizing the quality of event context construction are further included. Specifically, the analysis of these two tasks is as follows:

[0482] e) Identification of key event nodes. Event context requires that event nodes within a topic must represent key developments, such as the official announcement of a resolution or the voice of a party involved. Through training and alignment, the target mining model can accurately identify whether events related to a topic are these key developments, thereby determining whether they can serve as high-quality context nodes.

[0483] Specifically, in the event topic mounting sub-process, the server can evaluate multiple event nodes that have been mounted under a certain topic through the target mining model, identify events with significant characteristics as key event nodes, and delete non-key event nodes. Among them, significance can be reflected in features such as large event influence, wide dissemination range, and special time nodes. The server can judge the importance of the event through the model, and can also combine some external indicators (such as the number of readings, comments, and media reports of event-related content) to assist in identification, and finally generate a list of key event nodes for highlighting or priority presentation in the event context diagram.

[0484] f) Deduplication of event nodes. The event context requires that there should be no duplicate event nodes under the topic. Therefore, after the discrimination task is completed, duplication determination between nodes is required to ensure the uniqueness of the context event nodes.

[0485] Specifically, in order to improve the clarity and readability of the event context structure, the server also performs event node deduplication operations. The server can perform semantic similarity analysis on all event nodes under the same topic through the target mining model, and calculate the degree of semantic matching between each event. When the semantic similarity of two event nodes exceeds a preset threshold, it is determined to be a duplicate event. At this time, the server can retain the most representative event node and remove redundant information. In some optional implementations, a clustering algorithm can also be used to batch process event nodes to improve deduplication efficiency. Through this sub-process, the context confusion caused by duplicate events can be effectively avoided, thereby ensuring that the final output event context is more refined and accurate, which is convenient for users to understand and use in subsequent analysis.

[0486] Continuing with the aforementioned case of "Celebrity Marriage Changes":

[0487] In the key event node identification sub-process, the server identified "On January 1, 2023, celebrity A and celebrity B officially announced their divorce" as the core event node under this topic and marked it as a "key event."

[0488] During the event node deduplication subprocess, the server detected two different descriptions of the same event: "Celebrity A posted that they had amicably broken up" and "Celebrity B responded to marital rumors." Semantic analysis determined that these two records were actually different perspectives on the same event. To eliminate redundancy, the server retained only one record, ensuring a concise and non-duplicate event context.

[0489] The above implementation further introduces a mechanism for identifying key event nodes and removing duplicate event nodes, which improves the accuracy and simplicity of event context construction. Through intelligent evaluation of event importance, the server can automatically screen out representative core event nodes, and combine semantic similarity analysis to effectively eliminate redundant events, thereby avoiding context confusion caused by information duplication or noise interference. Overall, the final constructed event context is more refined and focused, making it easier to use in application scenarios such as news monitoring and historical review. While improving user understanding efficiency, it also provides high-quality data support for automated analysis and decision support.

[0490] It should be noted that the specific implementation methods of these tasks / sub-processes during the model training process are the same as above and will not be repeated here.

[0491] like Figure 8 As shown, it is a link diagram of a context mining system in an embodiment of the present application, and the six NLP tasks mentioned above are mentioned in the link diagram. In addition, the context mining system of the present application can also use various forms of media to store intermediate data and final data, such as Figure 8The context Json data in the ContextData can be persistently stored in the form of key-value pairs using Redis. The topic database and context database are stored in the form of vector index databases, namely the topic vector index database and the context vector index database. The sequence numbers in the link diagram represent the various steps of the entire system, as follows:

[0492] ① The target mining model generates contextual topics based on the hot list words; ② The target mining model merges the generated topics with historical topics; ③ Retrieve historical topics from the topic vector index library; ④ Merge all event nodes under each topic; ⑤ Enter the new topic into the topic vector index library; ⑥ Enter the contextual data into the context vector index library; ⑦ Use the dynamic summary extraction algorithm to obtain the summary of the news article; ⑧ The target mining model generates hot events based on the news article; ⑨ Use events, summaries, and recalled existing topics to construct prompt information, allowing the target mining model to complete the attachment of events and contextual topics; ⑩ The target mining model identifies whether the event attached to the topic is a key development; The target mining model removes duplicate event nodes under the topic; Get all node data under the topic from the context Json database.

[0493] Among them, ① corresponds to the topic generation task; ②-⑤ correspond to the topic merging task, where ④ specifically means merging the event node sets of similar topics that have been merged, such as Figure 7 As shown;⑧ corresponds to the event generation task;⑨ corresponds to the event topic mounting task;⑩ corresponds to the key event node identification task; Corresponding event node deduplication task. The specific implementation process of these steps can be found in the above embodiments and will not be repeated here. The remaining steps not mentioned mainly involve the retrieval and storage of information. For example, in ⑤ and ⑥, the vector corresponding to the new topic can be stored in the topic vector index library and the context vector index library; wherein, the topic vector index library can be used to retrieve topics, and the context vector index library can be used to retrieve event contexts. After finding the topic related to the context retrieval request, the various event nodes corresponding to the topic can be obtained by querying the context Json data, so that a visual event context can be generated and displayed to the user.

[0494] In the above system architecture, there are many nodes involving the recall of existing resources, such as retrieving historical topics for new topics, retrieving related topics for events, and retrieving event context in response to user requests. The following will explain this part of information retrieval in detail:

[0495] To more efficiently and accurately locate target information from massive amounts of event or topic data, this application also provides a hybrid information retrieval method that integrates semantic understanding and text matching. This method fully considers the differences between the semantic expression of user input and the actual text features. It uses a dual-channel retrieval mechanism to capture semantic relevance and literal relevance, and then fuses the results based on this, thereby obtaining retrieval results that better meet the user's intentions.

[0496] An optional implementation is to follow the Figure 9 The process shown in FIG. 10 is for retrieving information, such as retrieving existing topics or retrieving event context, and specifically includes the following steps S91 to S93:

[0497] S91: Obtain a first search result based on the similarity between the semantic vector of the information to be retrieved and the semantic vector of each candidate information in the information set.

[0498] This step involves semantic relevance retrieval. The server first semantically encodes the information being retrieved, obtaining its corresponding semantic vector representation. This semantic vector is then compared to the semantic vectors of each candidate information in the information set to determine the first search result or results.

[0499] When semantically encoding the information to be retrieved or the candidate information in the information set, a pre-trained language model can be used to encode the input text to obtain a vector representation with rich semantic information. These models can effectively capture semantic features at the lexical, syntactic, and contextual levels and are suitable for complex semantic matching tasks. In addition, lightweight models can be selected according to the actual application scenario to reduce computing resource consumption while ensuring semantic expression capabilities. For scenarios that require cross-language or cross-modal processing, multilingual models or multimodal encoders can also be used for unified representation learning.

[0500] Among them, the pre-trained language model can be a graph-enhanced embedding model (Bidirectional Generative Encoder, BGE), BERT, a sentence-based BERT model (Sentence-BERT, SBERT), a contrastive learning sentence vector model (Sentence Embedding using Contrastive pre-training, SimCSE), etc., which is not specifically limited in this article.

[0501] The lightweight model can be a lightweight BERT model (Tiny Bidirectional Encoder Representations from Transformers, TinyBERT), a distilled BERT model (Distilled Bidirectional Encoder Representations from Transformers, DistilBERT), etc., which is not specifically limited in this article.

[0502] The multilingual model can be a multilingual BERT model (Multilingual Bidirectional Encoder Representations from Transformers, mBERT), a cross-lingual language pre-training model (Cross-lingual Language Model Pretraining, XLM-R), etc., which is not specifically limited in this article.

[0503] The multimodal encoder can be a contrastive language-image pre-training model (CLIP), a multimodal foundational model (FLAVA), etc., and is not specifically limited in this article.

[0504] In systems with high real-time requirements, model optimization techniques such as knowledge distillation and quantization compression can be combined to further improve semantic encoding efficiency, ensuring good responsiveness even in large-scale data retrieval scenarios. Furthermore, for structured information (such as event nodes and topic tags), key fields can be concatenated and uniformly encoded.

[0505] When calculating the semantic vector similarity between the information to be retrieved and its candidate information, the server may use one or more similarity metrics to measure the proximity between the two in the vector space, including but not limited to: cosine similarity, which is used to measure the angle between two vector directions, is insensitive to vector length, and is widely used in high-dimensional semantic space matching; Euclidean distance, which reflects the absolute distance between two vectors in space and is suitable for scenarios that emphasize the positional relationship of vectors; and dot product, whose value is not only related to the vector direction but also affected by the modulus length, and is often used in the model's internal scoring mechanism.

[0506] Furthermore, based on specific task requirements, we can introduce extension methods such as weighted similarity functions, Manhattan distance, and Jaccard similarity coefficient to adapt to different data distributions and application scenarios. For multimodal retrieval tasks, we can also combine cross-modal similarity modeling strategies, such as using contrastive learning objectives to optimize cross-modal matching relationships, thereby improving overall retrieval performance.

[0507] After calculating the semantic vector similarity between the information being retrieved and each candidate, the server sorts all candidates in descending order based on the similarity score. High-quality first-results can then be selected using two methods: one is to set a preset similarity threshold and retain only candidates with scores above that threshold; the other is to select the top-N candidates with the highest ranking. These two strategies can be used independently or in combination to achieve a balanced balance between search precision and recall.

[0508] In addition, a dynamic threshold adjustment mechanism can be introduced to adaptively adjust the screening criteria based on historical query distribution or user feedback data, thereby improving the robustness and generalization ability of the model in different scenarios.

[0509] Optionally, in order to improve retrieval efficiency, the information set may store a semantic vector representation corresponding to each candidate information. The semantic vector can be obtained through the above-mentioned semantic encoding method, which will not be repeated here.

[0510] In another alternative implementation, when faced with large-scale information sets, if similarity is calculated one by one, the retrieval efficiency may be significantly reduced, affecting the server's real-time response capability. To this end, this application also introduces the Hierarchical Navigable Small World (HNSW) algorithm to improve the efficiency of large-scale semantic vector retrieval.

[0511] Specifically, the server pre-builds a semantic vector index based on each candidate information in the information set and uses HNSW to construct an efficient Approximate Nearest Neighbor (ANN) retrieval structure to accelerate the similarity sorting process. When performing a semantic search, the server can quickly find the candidate information most similar in semantics to the information being retrieved within this index structure, thereby improving response efficiency and significantly reducing computational overhead while ensuring recall quality.

[0512] Through the above approach, the accuracy of the semantic matching method is retained, and the efficient retrieval capability of HNSW is combined to achieve high-performance semantic retrieval on large-scale topic sets, thereby improving the practicality and scalability of the server.

[0513] In summary, the first search result is calculated based on the semantic vector similarity, which enables the server to identify candidate information that is highly semantically related but has different literal expressions, thereby improving the generalization ability and recall rate.

[0514] S92: Obtain a second search result based on the correlation between the text field included in the information to be retrieved and the text field included in each candidate information in the information set.

[0515] This step belongs to literal relevance retrieval. In this step, the server obtains one or more second retrieval results based on the relevance between the text fields contained in the information to be retrieved and the text fields of each candidate information in the information set.

[0516] Specifically, the server can measure the lexical relevance between the two based on one or more text analysis methods, such as keyword matching, word frequency statistics, and text overlap. For example, algorithms such as TF-IDF, BestMatching 25 (BM25), or N-gram models can be used to quantify the degree of matching and perform precise matching on structured fields such as titles, abstracts, tags, and keywords.

[0517] In addition, the server can also support a variety of flexible text matching strategies such as wildcard matching, fuzzy matching, regular expressions, etc. to adapt to different types of query requirements.

[0518] Finally, the candidate information is sorted by relevance score, and the results with higher scores are selected as the second search results. These are then merged with the semantically matched first search results, thereby improving the accuracy and applicability of the overall search. The specific method for selecting the second search results can be found in the description of the first search results, and will not be repeated here.

[0519] In summary, the second search result is based on the text field correlation analysis, which enhances the ability to accurately match structured information such as keywords and tags, ensuring that key information is not missed.

[0520] S93: Merge the first search result and the second search result to obtain a search result for the information to be searched.

[0521] After obtaining the search results of the two dimensions, the server uses a fusion strategy to merge the two. The merged search results can be one or more, depending on the actual needs, to improve the overall search accuracy and recall rate. Specific fusion strategies include but are not limited to:

[0522] Fusion strategy 1: weighted sorting method.

[0523] Each search result is assigned a corresponding confidence weight (which can be assigned based on the needs of the application scenario or historical data feedback), and a weighted ranking is performed based on the semantic match score and the literal relevance score, thereby comprehensively considering the advantages of both search methods. Specifically, the server evaluates the degree of match of the search results at the semantic level and the text level respectively. By setting a reasonable weight ratio, the two scores are combined into a comprehensive score, thereby selecting the results with better overall semantic relevance and keyword matching.

[0524] Optionally, since the semantic matching score and the literal relevance score may come from model outputs of different dimensions, the server can also normalize the scores of different dimensions, uniformly map them to the same interval (such as [0,1]), and then perform a comprehensive score to eliminate the impact of dimensional differences.

[0525] Fusion strategy 2: intersection / union screening.

[0526] Use intersection filtering to retain high-quality candidate information shared by two results to improve retrieval accuracy; or use union filtering to merge all candidate results to enhance recall capabilities.

[0527] Fusion strategy three: machine learning sorting optimization.

[0528] As an optional implementation, a learning-based ranking model (Learning to Rank), such as a ranking neural network (RankNet), a decision tree ensemble-based ranking model (Lambda MART, LambdaMART), etc., can also be introduced to perform secondary ranking on the obtained search results, thereby achieving more refined result optimization and further improving the search quality.

[0529] Through the above fusion strategy, the final output of the search results not only retains the understanding ability at the semantic level, but also takes into account the precise matching requirements at the keyword level, significantly improving the intelligence level of the retrieval system and user satisfaction.

[0530] In addition, it should be noted that the several fusion strategies listed above are just simple examples. Other fusion strategies are also applicable to the embodiments of this application and will not be repeated here.

[0531] For example, in this application, all data retrieval processes are completed using data vectorization storage combined with ANN retrieval algorithms. Specifically, this application uses the BGE model to vectorize the data and construct a vector index library as the forward feature of the data, while also constructing an inverted index of the text. During retrieval, the HNSW algorithm is used to complete the vector retrieval process of similar data, while literal correlation matching is achieved based on BM25, and finally the two data are merged to obtain the final retrieval results.

[0532] Among them, retrieval based on semantic vectors can better consider the semantic relevance between texts, show better performance when facing scenarios such as synonyms and polysemous words, and improve precision and recall rate. Retrieval based on literals can supplement the results of semantic retrieval and improve the richness and diversity of the results.

[0533] Optionally, the specific application scenarios of the above retrieval method in the embodiment of the present application are as follows:

[0534] Scenario 1: The information to be retrieved is an initial topic or event.

[0535] In this scenario, the information set is an existing topic set used to store existing topics; the candidate information is each existing topic. Optionally, each topic in the existing topic set has a corresponding semantic vector representation, such as that obtained by encoding using a BGE model. In this case, the existing topic set can also be called a topic vector index library.

[0536] In addition, the existing topic set can also include a set of event nodes associated with each topic. Each set of event nodes associated with an existing topic contains one or more mounted event nodes. These data can be stored in a structured manner using Json or other data formats. Figure 8 What is listed is just a simple example, and this article does not make any specific limitations on this.

[0537] For example, when searching for a new topic with a history of existing topics, a user enters a new query topic, such as "The impact of climate change on agriculture." The server first uses a Chinese word segmentation tool (such as Jieba) to segment the query text into terms: "climate change," "to," "agriculture," "of," and "impact." The server then calculates the importance score for each term. Then, using the BM25 algorithm, it retrieves candidate topics with a high literal match from the event topic database, which become the second search result.

[0538] In addition, the server can use the BGE model to semantically encode the query text, generate the corresponding semantic vector, and use the HNSW algorithm to quickly find the most semantically relevant candidate topic in the topic vector index library, that is, the first search result.

[0539] On this basis, the server merges and ranks the BM25-based search results with the HNSW-based search results, ultimately selecting the highest-ranked existing topic as the most similar to the initial topic. The initial topic and this result can then be merged. Because all data is stored in vector form and efficiently retrieved using the ANN search algorithm, it maintains good search performance even in complex semantic scenarios such as synonyms and polysemous words. The resulting search results retain semantic understanding while also meeting the requirements for precise keyword-level matching, thereby improving overall accuracy and recall.

[0540] For example, a user might want to see discussions related to a specific event, such as "Event X occurred in a certain location." The steps are the same as above, but in this scenario, the query text may contain more specific entity names or event descriptions. This information can help more precisely locate relevant topics and achieve event topic targeting. Furthermore, in this case, semantic vector retrieval can be more reliant on capturing the deeper meaning and connections behind the event, rather than just superficial word matching.

[0541] like Figure 10A As shown in FIG, it is a topic search diagram in an embodiment of the present application. Among them, the existing topic set includes several topics, such as Figure 10A As shown, topics_1234, topic_5, topic_6, topic_78, topic_9, ... are retrieved for recall topics corresponding to a certain event. For example, if semantic relevance search yields the first search result: topic_9, and literal relevance search yields the second search result: topic_5, topic_6. These two results can be combined to obtain the three recall topics corresponding to the event: topic_5, topic_6, and topic_9. The topic retrieval process for topic merging is similar and will not be further elaborated here.

[0542] Scenario 2: The information to be retrieved is a context search request.

[0543] In this scenario, the information set is the event context set used to store the completed event context structured data. The candidate information is each complete event context structured data, which can include but is not limited to information on multiple dimensions such as the event timeline, participants, location, related entities, and event impact. This information together constitutes a complete semantic representation of the event and is associated with the corresponding topic and event node in a structured manner.

[0544] Optionally, each event context structured data in the event context set can be in Json format or other structured data formats. In addition, the topic involved in each event context can have a corresponding semantic vector representation, which can be stored separately or in the event context set. Figure 8What is listed is just a simple example, and this article does not make any specific limitations on this.

[0545] For example, a user enters a contextual search request, "The evolution of event X at location D in 2023," hoping to obtain the complete development context of event X. The server first parses the query text, extracting key entities and timelines. Based on keyword matching, it uses the BM25 algorithm to retrieve relevant event records from the event context database as the second search result. Furthermore, the server constructs an inverted index for the text field to support precise keyword matching.

[0546] The server also converts query content into semantic vectors. Models such as BGE have already vectorized the text in the event context and constructed a corresponding context vector index as a positive ranking feature. Combined with approximate nearest neighbor algorithms like HNSW, the server can efficiently retrieve semantically similar historical event contexts from a large-scale event vector database as the first search result.

[0547] Finally, the second search result based on keywords is combined and ranked with the first search result based on semantic understanding, outputting a list of event contexts that both aligns with the user's literal expression and encompasses deeper semantic connections, for subsequent visualization or further analysis. Specifically, this could include selecting the highest-ranked result as the final search result, or combining the first and second search results to output the final result.

[0548] like Figure 10B As shown in FIG. 1 , it is a topic search diagram in an embodiment of the present application. In which, the context data set contains several topics and their corresponding node sets, such as Figure 10B As shown, topic_1234 ([node 1, node 2, ...]), topic_5 ([node 3, node 4, ...]), topic_6 ([node 5, ...]), topic_78 ([node 6, node 7, ...]), topic_9 ([node 8, node 9, ...]), .... When performing a context search based on a context search request, for example, a first search result of topic_9 is obtained through a semantic relevance search, and a second search result of topic_9 is obtained through a literal relevance search. Then, an event context can be directly generated based on topic_9 and its related event nodes.

[0549] Optionally, when performing a contextual search, if the first and second search results differ, an appropriate strategy can be selected based on the application scenario. For example, a more confident result can be prioritized, such as by calculating a similarity score to determine the preferred option. Alternatively, a comprehensive ranking list can be generated by combining the strengths of both, using a weighted scoring mechanism to balance semantic relevance and literal match, thereby improving the accuracy and comprehensiveness of the search results. This approach preserves key information while enhancing the richness and reliability of the results.

[0550] like Figure 11 As shown in FIG, it is a schematic diagram of an event context in an embodiment of the present application, which shows five time nodes from September 1, 2023 to September 9, 2024. The corresponding node summary information is displayed on the right side of each time node. When the user clicks on the summary information of any time node, it can further display detailed content related to the node, such as event background, related characters or extended topics and other related information ( Figure 11 (not shown), thereby realizing the layer-by-layer expansion and in-depth browsing of the event context, and improving the convenience and completeness of information acquisition.

[0551] The above implementation provides a hybrid retrieval mechanism that integrates semantic matching and literal relevance, effectively improving the accuracy and comprehensiveness of information retrieval. By obtaining search results from both the semantic and textual levels and integrating them using a multi-strategy fusion strategy, the model not only enhances its understanding of natural language semantics but also ensures support for keyword-driven queries, thus meeting diverse information retrieval needs.

[0552] It should be noted that the specific implementation method of information retrieval during the model training process is the same as above and will not be repeated here.

[0553] In summary, this application proposes a fully automatic event context mining solution based on a large language model (such as an LLM). This solution fully leverages the LLM's powerful text understanding and generation capabilities, reduces manual intervention, and eliminates reliance on fixed patterns or schemas. By breaking down the context mining system into multiple key subtasks, and using a single LLM to perform multi-task fine-tuning and reinforcement learning within a two-stage refined training framework, the interpretability of the mining chain is improved, effectively alleviating the hallucination problem, and significantly improving the quality of event context mining. This provides a more practical technical path and application ideas for the implementation of large models in industrial scenarios.

[0554] See Figure 12 As shown, it is a logic block diagram of interaction between a terminal device and a server in an embodiment of the present application. Figure 12The server shown can deploy a trained target mining model to perform real-time event context mining using the target mining model and store the results in a context vector index library, as shown in 121. A browser is installed on the terminal device. A user can enter a context search request through the browser interface, such as "evolution of event X at location D in 2023." The terminal device sends this request to the server, which then searches the context vector index library to obtain event context search results and returns them to the terminal device, which then displays them to the user through the browser interface, as shown in 122. The specific implementation process is described in the above embodiment and will not be repeated here.

[0555] Based on the same inventive concept, the present application also provides a training device for a mining model. Figure 13 As shown, it is a schematic diagram of the structure of the mining model training device 1300, which may include:

[0556] A first training unit 1301 is configured to train an initial mining model based on a pre-constructed first sample set to obtain an intermediate mining model; the initial mining model corresponds to multiple tasks, each task representing a sub-process in the event context mining process; the first sample set includes: a first sample and a sample label associated with the task content of each task;

[0557] The sample construction unit 1302 is configured to input a second sample associated with the task content of at least one of the tasks into the intermediate mining model to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining; the candidate results contained in each obtained candidate result set are screened according to a preset evaluation standard to construct a second sample set; the second sample set includes: each second sample, and a positive candidate result and a negative candidate result corresponding to each second sample; the degree of match between the positive candidate result and the negative candidate result and the preset evaluation standard meets a preset matching condition;

[0558] The second training unit 1303 is configured to train the intermediate mining model based on the second sample set to obtain a target mining model.

[0559] Optionally, the multiple tasks include at least one generation task and at least one decision task related to event context mining; wherein the event context is composed of a topic and at least one event node associated with the topic;

[0560] The generation task is used to generate topics and events based on input information; the decision task is used to attribute events to corresponding topics to build event context; each event can be used as an event node for building event context;

[0561] The task corresponding to the second sample is at least one generation task.

[0562] Optionally, the generation task includes topic generation, topic merging, and event generation; the decision task includes event topic mounting; and the device further includes:

[0563] An application unit 1304 is configured to generate an initial topic based on a topic information source by the target mining model in a topic generation sub-process;

[0564] In the topic merging sub-process, the target mining model merges the initial topic with existing topics in the existing topic set, and updates the existing topic set to expand the scope of events covered by the existing topics in the existing topic set;

[0565] In the event generation sub-process, the target mining model generates events based on event information sources;

[0566] In the event topic mounting sub-process, after constructing prompt information based on the event, the summary of the event information source and at least one recall topic, the target mining model attributes the event to the corresponding recall topic based on the prompt information to construct the event context; wherein, the at least one recall topic is an existing topic recalled from the existing topic set, and its similarity with the event meets the preset similarity condition.

[0567] Optionally, the decision-making task further includes: identifying key event nodes and removing duplicate event nodes; after the event topic mounting sub-process, the application unit 1304 is further configured to:

[0568] In the key event node identification sub-process, when the target mining model identifies that an event attached to a topic belongs to a key event node related to the topic, the event is included in the event context corresponding to the topic;

[0569] In the event node deduplication sub-process, the target mining model performs a semantic deduplication operation on each event node under the same topic to remove redundant event nodes.

[0570] Optionally, the application unit 1304 is further configured to retrieve information in the following manner:

[0571] Obtaining a first search result based on the similarity between the semantic vector of the information to be retrieved and the semantic vectors of each candidate information in the information set;

[0572] Obtaining a second search result based on the correlation between the text field included in the information to be retrieved and the text field included in each candidate information in the information set;

[0573] Merging the first search result and the second search result to obtain a search result for the information to be searched;

[0574] Wherein, when the information to be retrieved is an initial topic or event, the information set is an existing topic set for storing various existing topics, and the candidate information is an existing topic;

[0575] When the information to be retrieved is a context search request, the information set is an event context set for storing constructed event context structured data, and the candidate information is event context structured data.

[0576] Optionally, the second training unit 1303 is configured to:

[0577] Iteratively train the intermediate mining model based on the second sample set; each round of training executes:

[0578] Inputting the second sample selected from the second sample set in this round into the intermediate mining model and the model to be trained in this round, respectively, to obtain a set of positive probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding positive candidate results, and a set of negative probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding negative candidate results; the model to be trained in this round in the first round is the intermediate mining model; the model to be trained in this round in other rounds is the output model of the previous round;

[0579] According to the difference between the set of positive probabilities and the set of negative probabilities, the parameters of the model to be trained in this round are adjusted to obtain the output model of this round.

[0580] Optionally, the set of positive probabilities includes: the positive baseline probability output by the intermediate mining model and the positive prediction probability output by the model to be trained in this round; the set of negative probabilities includes: the negative baseline probability output by the intermediate mining model and the negative prediction probability output by the model to be trained in this round;

[0581] Then the second training unit 1303 is used to:

[0582] Obtaining a positive probability ratio between the positive prediction probability and the positive reference probability, and a negative probability ratio between the negative prediction probability and the negative reference probability;

[0583] According to the difference between the positive probability ratio and the negative probability ratio, the parameters of the model to be trained in this round are adjusted.

[0584] Optionally, the probability of a mining model outputting a candidate result is the product of the output probabilities corresponding to each word in the candidate result;

[0585] The one mining model is the intermediate mining model or the model to be trained in this round; and the one candidate result is the positive candidate result or the negative candidate result.

[0586] Optionally, each candidate result included in the candidate result set is: a variety of candidate results generated based on different generation strategies for the same second sample; the preset matching condition includes: a positive matching threshold and a negative matching threshold, and the positive matching threshold is greater than the negative matching threshold; then:

[0587] The positive candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample whose matching degree with the preset evaluation standard is higher than the positive matching degree threshold;

[0588] The negative candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample, the matching degree between which the candidate result and the preset evaluation criterion is lower than the negative matching degree threshold.

[0589] Optionally, the preset evaluation criteria include: a quality standard set for the candidate results, and a content preference standard set for the candidate results; the sample construction unit 1302 is used to:

[0590] For each candidate result set, a candidate result in the candidate result set that meets the quality standard and has a matching degree with the content preference standard higher than the positive matching degree threshold is selected as a positive candidate result corresponding to the second sample matched by the candidate result set;

[0591] and for each candidate result set, taking a candidate result in the candidate result set whose matching degree with the content preference standard is lower than the negative matching degree threshold as a negative candidate result corresponding to the second sample matched by the candidate result set;

[0592] The second samples corresponding to each candidate result set, and the corresponding positive candidate results and negative candidate results are combined to form a second sample set.

[0593] Optionally, the first training unit 1301 is configured to:

[0594] The initial mining model is iteratively trained based on the first sample set; each round of training performs:

[0595] Inputting the first sample selected from the first sample set in this round into the current initial mining model, and obtaining a predicted probability distribution output by the current initial mining model guided by the task instructions related to the first sample;

[0596] According to the difference between the predicted probability distribution and the true probability distribution of the corresponding sample label, the parameters of the current initial mining model are adjusted to obtain the initial mining model output in this round.

[0597] Optionally, the first training unit 1301 is configured to:

[0598] According to the difference between the initial prediction probability and the corresponding sample label, the current initial mining model is adjusted in a low-rank adaptive manner.

[0599] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0600] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0601] After introducing the method and apparatus for training a mining model according to an exemplary embodiment of the present application, an electronic device according to another exemplary embodiment of the present application is introduced next.

[0602] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0603] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Figure 14 As shown, it includes a memory 1401 , a communication module 1403 and one or more processors 1402 .

[0604] Memory 1401 is used to store computer programs executed by processor 1402. Memory 1401 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0605] Memory 1401 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing a desired computer program in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1401 may be a combination of the aforementioned memories.

[0606] The processor 1402 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 1402 is configured to implement the above-mentioned mining model training method when calling the computer program stored in the memory 1401 .

[0607] The communication module 1403 is used to communicate with terminal devices and other servers.

[0608] The specific connection medium between the memory 1401, the communication module 1403 and the processor 1402 is not limited in the embodiment of the present application. Figure 14 In the embodiment, the memory 1401 and the processor 1402 are connected via a bus 1404. Figure 14 The connections between the other components are shown in bold lines for illustration only and are not intended to be limiting. The bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 14 The diagram shows a single thick line, but this does not indicate that there is only one bus or one type of bus.

[0609] The memory 1401 stores a computer storage medium, which stores computer executable instructions. The computer executable instructions are used to implement the training method of the mining model of the embodiment of the present application. The processor 1402 is used to execute the above-mentioned training method of the mining model, such as Figure 2 shown.

[0610] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. In this embodiment, the structure of the electronic device can be as follows: Figure 15 As shown, it includes: a communication component 1510, a memory 1520, a display unit 1530, a camera 1540, a sensor 1550, an audio circuit 1560, a Bluetooth module 1570, a processor 1580 and other components.

[0611] The communication component 1510 is used to communicate with the server. In some embodiments, it may include a wireless fidelity (WiFi) module. The WiFi module is a short-range wireless transmission technology. Electronic devices can help users send and receive information through the WiFi module.

[0612] The memory 1520 can be used to store software programs and data. The processor 1580 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 1520. The memory 1520 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1520 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 1520 can store the operating system and various application programs, and may also store a computer program for executing the training method of the mining model in the embodiment of the present application.

[0613] The display unit 1530 can also be used to display information input by the user or provided to the user, as well as a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1530 may include a display screen 1532 disposed on the front of the terminal device 110. The display screen 1532 may be configured in the form of a liquid crystal display, a light-emitting diode, etc. The display unit 1530 can be used to display the browser interface in the embodiments of the present application, etc.

[0614] The display unit 1530 can also be used to receive input digital or character information and generate signal input related to the user settings and function control of the terminal device 110. Specifically, the display unit 1530 may include a touch screen 1531 set on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0615] The touch screen 1531 can be covered on the display screen 1532, or the touch screen 1531 and the display screen 1532 can be integrated to realize the input and output functions of the terminal device 110. The integrated display screen can be simply called a touch screen. In this application, the display unit 1530 can display applications and corresponding operation steps.

[0616] The camera 1540 can be used to capture still images, and users can publish images captured by the camera 1540 through an application. There can be one or more cameras 1540. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the processor 1580 for conversion into a digital image signal.

[0617] The terminal device may further include at least one sensor 1550, such as an acceleration sensor 1551, a distance sensor 1552, a fingerprint sensor 1553, and a temperature sensor 1554. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0618] The audio circuit 1560, speaker 1561, and microphone 1562 provide an audio interface between the user and the terminal device 110. The audio circuit 1560 can convert the received audio data into an electrical signal and transmit it to the speaker 1561, which converts it into a sound signal for output. The terminal device 110 may also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1560 and converted into audio data. The audio data is then output to the communication component 1510 for transmission to, for example, another terminal device 110, or the audio data is output to the memory 1520 for further processing.

[0619] The Bluetooth module 1570 is used to exchange information with other Bluetooth devices having a Bluetooth module through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1570 to exchange data.

[0620] The processor 1580 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing software programs stored in the memory 1520 and calling data stored in the memory 1520, it performs various functions of the terminal device and processes data. In some embodiments, the processor 1580 may include one or more processing units; the processor 1580 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1580. In the present application, the processor 1580 can run the operating system, application programs, user interface display and touch response, as well as the training method of the mining model in the embodiment of the present application. In addition, the processor 1580 is coupled to the display unit 1530.

[0621] In some possible implementations, various aspects of the mining model training method provided in the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the mining model training method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device may execute the following steps: Figure 2 Follow the steps shown in .

[0622] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0623] The program product of the embodiment of the present application may be a portable compact disc read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with a command execution system, apparatus, or device.

[0624] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with a command execution system, apparatus, or device.

[0625] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0626] The computer program for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer program can be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0627] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0628] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0629] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0630] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0631] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising a command device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0632] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0633] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0634] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for training a mining model, characterized in that: The method comprises: An initial mining model is trained based on a pre-built first sample set to obtain an intermediate mining model; the initial mining model corresponds to multiple tasks, each task representing a sub-process in the event context mining process; the first sample set includes: a first sample and a sample label associated with the task content of each task; Inputting a second sample associated with the task content of at least one of the tasks into the intermediate mining model to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining; According to a preset evaluation criterion, each candidate result included in each obtained candidate result set is screened to construct a second sample set; the second sample set includes: each second sample, and a positive candidate result and a negative candidate result corresponding to each second sample; the degree of match between the positive candidate result and the negative candidate result and the preset evaluation criterion meets a preset matching condition; The intermediate mining model is trained based on the second sample set to obtain a target mining model.

2. The method according to claim 1, wherein The multiple tasks include at least one generation task and at least one decision task related to event context mining; wherein the event context is composed of a topic and at least one event node associated with the topic; The generation task is used to generate topics and events based on input information; the decision task is used to attribute events to corresponding topics to build event context; each event can be used as an event node for building event context; The task corresponding to the second sample is at least one generation task.

3. The method according to claim 2, wherein The generation tasks include topic generation, topic merging and event generation; The decision-making task includes event topic mounting; The method further comprises: In the topic generation sub-process, the target mining model generates initial topics based on the topic information source; In the topic merging sub-process, the target mining model merges the initial topic with existing topics in the existing topic set, and updates the existing topic set to expand the scope of events covered by the existing topics in the existing topic set; In the event generation sub-process, the target mining model generates events based on event information sources; In the event topic mounting sub-process, after constructing prompt information based on the event, the summary of the event information source and at least one recall topic, the target mining model attributes the event to the corresponding recall topic based on the prompt information to construct the event context; wherein, the at least one recall topic is an existing topic recalled from the existing topic set, and its similarity with the event meets the preset similarity condition.

4. The method according to claim 3, wherein The decision-making task further includes: identifying key event nodes and removing duplicate event nodes; after the event topic mounting sub-process, the method further includes: In the key event node identification sub-process, when the target mining model identifies that an event attached to a topic belongs to a key event node related to the topic, the event is included in the event context corresponding to the topic; In the event node deduplication sub-process, the target mining model performs a semantic deduplication operation on each event node under the same topic to remove redundant event nodes.

5. The method according to claim 3 or 4, wherein: Retrieve information by: Obtaining a first search result based on the similarity between the semantic vector of the information to be retrieved and the semantic vectors of each candidate information in the information set; Obtaining a second search result based on the correlation between the text field included in the information to be retrieved and the text field included in each candidate information in the information set; Merging the first search result and the second search result to obtain a search result for the information to be searched; Wherein, when the information to be retrieved is an initial topic or event, the information set is an existing topic set for storing various existing topics, and the candidate information is an existing topic; When the information to be retrieved is an event context query request, the information set is an event context set for storing constructed event context structured data, and the candidate information is event context structured data.

6. The method according to any one of claims 1 to 4, wherein: The training of the intermediate mining model based on the second sample set includes: Iteratively train the intermediate mining model based on the second sample set; each round of training executes: Inputting the second sample selected from the second sample set in this round into the intermediate mining model and the model to be trained in this round, respectively, to obtain a set of positive probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding positive candidate results, and a set of negative probabilities when the intermediate mining model and the model to be trained in this round respectively output corresponding negative candidate results; the model to be trained in this round in the first round is the intermediate mining model; the model to be trained in this round in other rounds is the output model of the previous round; According to the difference between the set of positive probabilities and the set of negative probabilities, the parameters of the model to be trained in this round are adjusted to obtain the output model of this round.

7. The method according to claim 6, wherein The set of positive probabilities includes: the positive baseline probability output by the intermediate mining model and the positive prediction probability output by the model to be trained in this round; the set of negative probabilities includes: the negative baseline probability output by the intermediate mining model and the negative prediction probability output by the model to be trained in this round; Then, adjusting the parameters of the model to be trained in this round according to the difference between the set of positive probabilities and the set of negative probabilities includes: Obtaining a positive probability ratio between the positive prediction probability and the positive reference probability, and a negative probability ratio between the negative prediction probability and the negative reference probability; According to the difference between the positive probability ratio and the negative probability ratio, the parameters of the model to be trained in this round are adjusted.

8. The method according to claim 6, wherein The probability of a mining model outputting a candidate result is the product of the output probabilities corresponding to each word in the candidate result; The one mining model is the intermediate mining model or the model to be trained in this round; and the one candidate result is the positive candidate result or the negative candidate result.

9. The method according to any one of claims 1 to 4, wherein: The candidate results included in the candidate result set are: diverse candidate results generated based on different generation strategies for the same second sample; the preset matching condition includes: a positive matching threshold and a negative matching threshold, and the positive matching threshold is greater than the negative matching threshold; then: The positive candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample whose matching degree with the preset evaluation standard is higher than the positive matching degree threshold; The negative candidate result corresponding to each second sample is: a candidate result in the candidate result set corresponding to the second sample, the matching degree between which the candidate result and the preset evaluation criterion is lower than the negative matching degree threshold.

10. The method according to claim 9, wherein The preset evaluation criteria include: a quality standard set for the candidate results, and a content preference standard set for the candidate results; then, according to the preset evaluation criteria, each candidate result contained in each obtained candidate result set is screened to construct a second sample set, including: For each candidate result set, a candidate result in the candidate result set that meets the quality standard and has a matching degree with the content preference standard higher than the positive matching degree threshold is selected as a positive candidate result corresponding to the second sample matched by the candidate result set; and for each candidate result set, taking a candidate result in the candidate result set whose matching degree with the content preference standard is lower than the negative matching degree threshold as a negative candidate result corresponding to the second sample matched by the candidate result set; The second samples corresponding to each candidate result set, and the corresponding positive candidate results and negative candidate results are combined to form a second sample set.

11. The method according to any one of claims 1 to 4, characterized in that: The training of the initial mining model based on the pre-built first sample set includes: The initial mining model is iteratively trained based on the first sample set; each round of training performs: Inputting the first sample selected from the first sample set in this round into the current initial mining model, and obtaining a predicted probability distribution output by the current initial mining model guided by the task instructions related to the first sample; According to the difference between the predicted probability distribution and the true probability distribution of the corresponding sample label, the parameters of the current initial mining model are adjusted to obtain the initial mining model output in this round.

12. The method according to claim 11, wherein The adjusting parameters of the current initial mining model according to the difference between the initial prediction probability and the corresponding sample label includes: According to the difference between the initial prediction probability and the corresponding sample label, the current initial mining model is adjusted in a low-rank adaptive manner.

13. A training device for a mining model, characterized in that: The device comprises: A first training unit is configured to train an initial mining model based on a pre-constructed first sample set to obtain an intermediate mining model; the initial mining model corresponds to a plurality of tasks, each task representing a sub-process in the event context mining process; the first sample set includes: a first sample and a sample label associated with the task content of each task; A sample construction unit is configured to input a second sample associated with the task content of at least one of the tasks into the intermediate mining model to obtain a candidate result set output for each second sample; the first sample and the second sample are both training data related to event context mining; the candidate results contained in each obtained candidate result set are screened according to a preset evaluation standard to construct a second sample set; the second sample set includes: each second sample, and a positive candidate result and a negative candidate result corresponding to each second sample; the degree of match between the positive candidate result and the negative candidate result and the preset evaluation standard meets a preset matching condition; The second training unit is configured to train the intermediate mining model based on the second sample set to obtain a target mining model.

14. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is enabled to perform the steps of any one of the methods of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The method comprises a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the methods according to claims 1 to 12.

16. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the methods described in claims 1 to 12.

Images