Dataset management method, model training method, user preference perception method, agent, device, equipment, medium and product

Abstract

The disclosure provides a dataset management method, a model training method, a user preference perception method, an agent, an apparatus, a device, a medium and a product, and relates to the technical field of computers. A specific embodiment of the dataset management method comprises: dividing multi-modal user historical data into multi-modal data packets; inputting the multi-modal data packets into a large language model to extract key features of the multi-modal data packets; based on the key features, performing inference through the large language model to output target text containing key information in the multi-modal data packets; based on the target text, establishing a local knowledge graph, and updating a global knowledge graph with the local knowledge graph to obtain a user historical dataset. The embodiment can provide high-quality, fine-grained and comprehensive user data for model training of an agent.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a dataset management method, a model training method, a user preference perception method, an intelligent agent, a dataset management device, a model training device, an electronic device, a computer storage medium, and a computer program product. Background Technology

[0002] With the development of artificial intelligence technology, using AI agents to meet the needs of different usage scenarios in electronic products is a promising application direction. An ideal auxiliary agent should understand the user's preferences and habits; that is, the agent's behavior needs to be differentiated according to the individual.

[0003] However, existing electronic products, such as televisions, typically use AI agents trained on servers. Sending users' personal information and the content they view directly to the server raises privacy concerns and presents significant challenges. Therefore, the AI ​​agents used in current electronic products often only provide vague, directional data, making it difficult to build personalized, customized AI assistance tools for each user. Summary of the Invention

[0004] This disclosure provides a dataset management method, a model training method, a user preference awareness method, an intelligent agent, a device, an equipment, a medium, and a product.

[0005] According to a first aspect, embodiments of this disclosure provide a dataset management method, comprising: dividing multimodal user historical data into multimodal data packets; inputting the multimodal data packets into a large language model to extract key features of the multimodal data packets; performing inference based on the key features through the large language model to output target text containing key information in the multimodal data packets; establishing a local knowledge graph based on the target text, and updating the global knowledge graph with the local knowledge graph to obtain a user historical dataset.

[0006] According to a second aspect, embodiments of this disclosure provide a model training method, wherein the model includes: a pre-trained large language model and multiple adaptation layers adapted to various task scenarios; the training method includes: constructing training sample sets and sample labels corresponding to each task scenario based on a user historical dataset, using prompt word templates and response templates, wherein the user historical dataset is obtained according to the dataset management method described in the first aspect, the prompt word templates are configured according to the task requirements of the corresponding task scenario, and the response templates are configured according to the response format of the corresponding task scenario; inputting the training sample set of each task scenario into the pre-trained large language model, obtaining an output result through the large language model and the corresponding adaptation layer, and training the adaptation layer based on the output result and the corresponding sample labels.

[0007] According to a third aspect, embodiments of this disclosure provide a user preference perception method applied to an intelligent agent, the intelligent agent including a large model trained according to the model training method described in the second aspect; the method includes: in response to receiving a user's access operation to a target task scene, generating a scene token corresponding to the target task scene; based on the scene token, determining an adaptation layer and a prompt word template corresponding to the target task scene; generating prompt word text based on the prompt word template, triggering a pre-trained large language model and the adaptation layer to perform inference, generating auxiliary information reflecting the user preference; generating corresponding control instructions based on the auxiliary information, and performing specific operations on the target task scene based on the control instructions.

[0008] According to a fourth aspect, embodiments of this disclosure provide an intelligent agent, comprising: an input module for receiving input information; a processing module for determining a target task based on the input information received by the input module, determining a large model based on the target task, and obtaining output information by invoking the method described in the third aspect; and an output module for outputting the output information obtained by the processing module.

[0009] According to a fifth aspect, embodiments of this disclosure provide a dataset management device, comprising: a data packetization module configured to divide multimodal user historical data into multimodal data packets; a feature extraction module configured to input the multimodal data packets into a large language model and extract key features from the multimodal data packets; a text generation module configured to perform reasoning based on the key features through the large language model and output target text containing key information from the multimodal data packets; and a data update module configured to establish a local knowledge graph based on the target text and update a global knowledge graph with the local knowledge graph to obtain a user historical dataset.

[0010] According to a sixth aspect, this disclosure provides a model training apparatus, wherein the model includes: a pre-trained large language model and multiple adaptation layers adapted to various task scenarios; the training apparatus includes: a sample generation module configured to construct training sample sets and sample labels corresponding to each task scenario based on a user historical dataset, using prompt word templates and response templates, wherein the user historical dataset is obtained according to the dataset management method described in the first aspect, the prompt word templates are configured according to the task requirements of the corresponding task scenario, and the response templates are configured according to the response format of the corresponding task scenario; and a model training module configured to input the training sample set of each task scenario into the pre-trained large language model, obtain an output result through the large language model and the corresponding adaptation layer, and train the adaptation layer based on the output result and the corresponding sample labels.

[0011] According to a seventh aspect, embodiments of this disclosure provide an electronic device including one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in the first, second, or third aspects.

[0012] According to an eighth aspect, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon that, when executed by a processor, implements the methods described in the first, second, or third aspects.

[0013] According to a ninth aspect, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the methods described in the first, second, or third aspects.

[0014] The dataset management method, model training method, user preference perception method, intelligent agent, device, equipment, medium, and product disclosed herein utilize the scalability and attention mechanism of large language models to extract relationships from multimodal data and adaptively select and filter the output of large language models. This avoids a large amount of redundant and inefficient analysis, saves resources, and enables the local generation and maintenance of user-specific datasets, facilitating efficient management of user data. It provides high-quality, fine-grained, and comprehensive user data for intelligent agent model training, allowing the intelligent agent to understand user preferences and habits, and enabling the intelligent agent's behavior to be differentiated according to individual user differences. It provides data support for the intelligent agent to simultaneously consider individual user focus and multi-scenario adaptation, and the constructed user historical dataset can also be used as an external auxiliary resource for third-party services.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] Figure 1 This is an exemplary system architecture diagram of an embodiment of the dataset management method, model training method, and user preference awareness method disclosed herein; Figure 2 This is a flowchart of a dataset management method according to some embodiments of the present disclosure; Figure 3 This is a schematic diagram illustrating the generation of a user history dataset according to some embodiments of this disclosure; Figure 4 This is a schematic diagram illustrating key features of some embodiments of this disclosure; Figure 5 This is a flowchart illustrating the extraction of key features of multimodal data packets according to some embodiments of this disclosure; Figure 6 This is a schematic diagram of the target text according to some embodiments of this disclosure; Figure 7 This is a flowchart illustrating how key features guide large language model reasoning according to some embodiments of this disclosure; Figure 8 This is a flowchart of a dataset management method according to other embodiments of this disclosure; Figure 9 This is a schematic diagram illustrating the generation of a user history dataset according to other embodiments of this disclosure; Figure 10 This is a flowchart of a model training method according to some embodiments of the present disclosure; Figure 11 This is a schematic diagram of a model training method according to some embodiments of this disclosure; Figure 12 This is a flowchart of a user preference awareness method according to some embodiments of this disclosure; Figure 13 This is a schematic diagram of a user preference awareness method according to some embodiments of this disclosure; Figure 14 This is an overall flowchart of an application scenario based on the method provided in this disclosure; Figure 15 This is a schematic diagram of the first application scenario using the user preference awareness method disclosed herein; Figure 16 This is a schematic diagram of a second application scenario using the user preference awareness method disclosed herein; Figure 17This is a schematic diagram of a third application scenario using the user preference awareness method disclosed herein; Figure 18 This is a schematic diagram of the fourth application scenario using the user preference awareness method disclosed herein; Figure 19 This is a schematic diagram of a dataset management apparatus according to some embodiments of the present disclosure; Figure 20 This is a schematic diagram of a model training apparatus according to some embodiments of the present disclosure; Figure 21 These are schematic diagrams of intelligent agents according to some embodiments of this disclosure; Figure 22 This is a schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0020] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the dataset management method, model training method, and user preference awareness method of this disclosure can be applied.

[0021] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, network 104, and server 105. Network 104 serves as a medium for providing communication links between terminal devices 101, 102, and 103 and server 105, and between the terminal devices themselves. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0022] Users can use terminal devices 101, 102, and 103 to interact with other terminal devices or servers 105 via network 104 to receive or send messages, etc. Client application software, such as video playback applications, can be installed on terminal devices 101, 102, and 103.

[0023] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smart TVs, laptops, tablets, and mobile phones. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here.

[0024] Server 105 can be a server providing various services, such as a video playback server. Server 105 can be hardware or software. When server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide video playback services), or as a single software program or software module. No specific limitations are made here.

[0025] It should be noted that the dataset management method, model training method, and user preference awareness method provided in the embodiments of this disclosure can be executed by terminal devices 101, 102, and 103. Accordingly, the various parts (e.g., modules, sub-modules) of the dataset management device, model training device, and intelligent agent using the user preference awareness method can all be set in terminal devices 101, 102, and 103.

[0026] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0027] Figure 2 A flow 200 of a dataset management method according to some embodiments of this disclosure is shown. For example... Figure 2 As shown, process 200 may include the following steps: Step 201: Divide the multimodal user historical data into multimodal data packets.

[0028] In this embodiment, the execution entity (e.g., Figure 1Terminal devices 101, 102, and 103 can divide multimodal user history data into multimodal data packets. This multimodal user history data can be a large amount of coarse and poorly structured user history records (UHR) collected by data collection tools each time a user uses the terminal device; it can also be referred to as coarse-grained metadata. For example, ... Figure 3 As shown, the multimodal user history data 301 may include video data 302, user behavior data 303, audio data 304, and time data 305, etc. The embodiments of this disclosure do not limit the data types included in the multimodal user history data, nor the type of data collection tool used to collect the multimodal user history data.

[0029] In this embodiment, a data sharding strategy can be pre-set in the execution entity. After collecting multimodal user historical data through a data collection tool, the execution entity can divide the multimodal user historical data into multiple data blocks according to the data sharding strategy, where each data block is a multimodal data packet. The embodiments of this disclosure do not limit the implementation method of dividing multimodal user historical data into multimodal data packets. In an optional example, the multimodal user historical data can be divided into multimodal data packets based on a preset time window. For example, the time length corresponding to the time window can be set in the data sharding strategy. The time length corresponding to the time window can be set according to requirements, and the embodiments of this disclosure do not limit this. The time window is a mechanism used to divide an infinite data stream into a finite time interval for analysis. By defining a clear start and end time, it can divide continuous data into manageable "data blocks".

[0030] For example, such as Figure 3 As shown, the time length corresponding to the preset time window 306 is The data collection tool gathers 301 multimodal user historical data, which can be divided into several batches of consecutive time periods. The length of each batch is... The batch quantity is / Each batch can contain video data, user behavior data, audio data, and time data, and each batch can be considered as a multimodal data unit 307 or a multimodal data packet.

[0031] Step 202: Input the multimodal data packets into the large language model and extract the key features of the multimodal data packets.

[0032] In this embodiment, the aforementioned execution entity can input multimodal data packets into a large language model to extract key features of the multimodal data packets. The large language model (LLM) can be a multimodal large model that achieves cross-modal input processing by increasing the perception and understanding capabilities of other modal data such as video, audio, and images. For example, the large language model can adopt a pure decoder-only architecture, which is more streamlined, efficient, and easily scalable compared to an encoder-decoder architecture. Key features of the multimodal data packets can be locations within the multimodal data packets that require focused analysis. For example, key features of the multimodal data packets can be regions of interest (ROIs) in video frames. The embodiments of this disclosure do not limit the implementation method of extracting key features of multimodal data packets through a large language model.

[0033] In an optional example, the executing agent inputs multimodal data packets into a large language model. The large language model can then calculate an attention matrix for the multimodal data packets, analyze the packets, determine the positions with the highest attention within the packets, and extract features to obtain the key features of the multimodal data packets. This disclosure does not limit the implementation method for extracting key features from multimodal data packets based on the attention matrix. For example, the executing agent can obtain the attention matrix of the multimodal data packets from the large language model, process the matrix to obtain attention scores for each position in the packets, determine the positions with the highest attention based on these scores, and extract key features. The attention matrix is ​​one of the core mechanisms for multimodal large models to achieve cross-modal understanding and fusion. It dynamically assigns weights to information from different modalities through a Query, Key, and Value (QKV) mechanism, enabling the focus and integration of key information.

[0034] Step 203: Based on key features, reasoning is performed using a large language model to output target text containing key information from the multimodal data packet.

[0035] In this embodiment, the aforementioned execution entity can infer using a large language model based on key features, and output target text containing key information from the multimodal data packet. The key information in the multimodal data packet can be information reflecting user preferences and habits, such as content of interest, user screen brightness settings, etc., and this disclosure does not limit this. After extracting the key features of the multimodal data packet, the execution entity can use these key features to guide the inference and filtering of the large language model, enabling the large language model to output target text containing key information from the multimodal data packet.

[0036] In this embodiment, the target text can be text containing user-personalized information from multimodal data packets. For example... Figure 3 As shown, the multimodal data packet input to the large language model 308 can be coarse-grained, low-information-density data 309, and the target text output by the large language model 308 can be fine-grained, high-information-density data 310. The embodiments of this disclosure do not limit the implementation method of using key features to guide the reasoning and filtering of the large language model. For example, key features can be used as latent variables, and the output of the large language model decoder can be guided by latent variable comparison and difference matching to achieve the filtering of the output text and obtain the target text containing key information from the multimodal data packet.

[0037] Step 204: Build a local knowledge graph based on the target text, and update the global knowledge graph with the local knowledge graph to obtain the user's historical dataset.

[0038] In this embodiment, the aforementioned execution entity can construct a local knowledge graph based on the target text and update the global knowledge graph with the local knowledge graph to obtain the user's historical dataset. A knowledge graph is a method for structured representation and organization of knowledge; it presents knowledge in the form of a graph, where nodes represent entities or concepts, and edges connecting nodes represent relationships between entities. In this implementation example, a local knowledge graph reflecting the entity relationships in the target text can be constructed based on the target text of each multimodal data packet, and then the global knowledge graph is supplemented and updated based on the local knowledge graph. Furthermore, as the user's usage cycle of the execution entity increases, the steps in the aforementioned dataset management method are iteratively executed repeatedly.

[0039] In this embodiment, the global knowledge graph is a structured user history dataset with a unified data format, obtained by refining and de-redundantizing the multimodal historical data of user execution entities. It can provide high-quality, authentic user history data for model learning. For example, ... Figure 3As shown, the executing entity can obtain a local knowledge graph 311 for each batch based on the fine-grained, high-information-density target text output by the large language model 308, and use the local knowledge graph 311 to update / add the global knowledge graph 312, thereby obtaining a user history dataset composed of the global knowledge graph 312.

[0040] The dataset management method provided in this disclosure divides multimodal user historical data into multimodal data packets; inputs the multimodal data packets into a large language model to extract key features; based on the key features, inference is performed through the large language model to output target text containing key information from the multimodal data packets; a local knowledge graph is built based on the target text, and the global knowledge graph is updated with the local knowledge graph to obtain the user historical dataset. The scalability and attention mechanism of the large language model can be used to extract relations from the multimodal data, and the output of the large language model can be adaptively selected and filtered. This avoids a large amount of redundant and inefficient analysis, saves resources, and enables the local generation and maintenance of user-specific datasets, facilitating efficient management of user data. It provides high-quality, fine-grained, and comprehensive user data for the training of intelligent agents, allowing agents to understand user preferences and habits, and enabling agents to tailor their behavior to individual user differences. It provides data support for agents to simultaneously consider individual user needs and adapt to multiple scenarios. Furthermore, the constructed user historical dataset can also be used as an external auxiliary resource for third-party services.

[0041] In some optional embodiments of this disclosure, step 202, which involves inputting the multimodal data packet into a large language model and extracting key features from the multimodal data packet, may include: inputting the multimodal data packet into the large language model to obtain the attention matrix during the pre-filling stage of the large language model; normalizing each row of the attention matrix and calculating the average value of each column of the normalized attention matrix to obtain an attention score vector; and extracting key features based on the features corresponding to the high-score positions in the attention score vector.

[0042] like Figure 4As shown, to extract the most attention-grabbing positions in multimodal data packets, the attention matrix from the prefill stage of a large language model can be used to analyze the attention matrix of the multimodal data packets calculated during the prefill stage and determine the positions of tokens of interest. First, a normalized softmax operation can be performed on each row of the attention matrix to obtain the attention level relative to all tokens. Then, the mean of the key / value pairs in each column of the normalized attention matrix is ​​calculated to obtain the attention score vector (ROI region tokens) for all positions. Finally, key features can be determined based on the positions with high scores in the attention score vector. For example, in... Figure 4 In the video frames and audio stream, the key features correspond to the competition text information area 401 in the upper left corner and the athlete area 402, which is located in a key position below. In this disclosure, a token can refer to the smallest unit of measurement for text processed and generated by an artificial intelligence model.

[0043] This embodiment utilizes the attention matrix from the pre-filling stage of the large language model for key feature extraction. Since the model needs to process the attention relationships of all input features during the pre-filling stage to generate a complete attention matrix, the complete input features are beneficial for capturing the dependency relationship between any two features in the input features through the attention matrix, thereby gaining a more comprehensive understanding of the semantic structure of the entire context and accurately identifying key features.

[0044] Figure 5 The flowchart illustrating some embodiments of this disclosure describes a process for extracting key features of multimodal data packets. For example... Figure 5 As shown, inputting multimodal data packets into a large language model and extracting key features from the multimodal data packets can include the following steps: Step 501: Encode the video data and audio data according to their respective modalities to obtain video feature sequences and audio feature sequences.

[0045] In this embodiment, the execution entity (e.g., Figure 1 The terminal devices 101, 102, and 103 can encode video and audio data according to their respective modalities to obtain video feature sequences and audio feature sequences. Before inputting the multimodal data packets into the large language model, the multimodal data needs to be tokenized. For data of different modalities, they can be encoded independently to obtain their own independent serialized feature vectors.

[0046] For example, such as Figure 6As shown, multimodal data at time T is input into the large language model. This multimodal data includes t-frame images (601) and audio streams (clips) from frames 0 to t (602) from the video. A text prompt (603) is also input along with the multimodal data. For example, the text prompt could be set to "This is a video clip; please analyze and extract information such as events and entities contained in the image." Three independent encoders (604, 605, and 606) can be set for the image, audio, and text modal data in the video, respectively. These are the V-encoder, A-encoder, and T-encoder. These encoders serialize the image frames, audio streams, and text prompts to obtain a series of tokens, which serve as the basic input to the large language model.

[0047] Step 502: The video feature sequence and the audio feature sequence are fused according to the time dimension to obtain the target audio and video feature sequence.

[0048] In this embodiment, the aforementioned execution entity can fuse the video feature sequence and the audio feature sequence according to the time dimension to obtain the target audio-video feature sequence. Specifically, if the multimodal data packet includes video data and audio data, since there is a strong spatiotemporal correlation between the images and audio in the video, after obtaining the video feature sequence and the audio feature sequence, they can be fused according to the time dimension to obtain a target audio-video feature sequence in which video features and audio features alternate in time.

[0049] For example, such as Figure 6 As shown, before inputting the video feature sequence 607, audio feature sequence 608, and text feature sequence 609 obtained from the encoder into the large language model, it is also possible to reorder a video feature (usually with dimensions T×N×C) containing t frames with audio features in time according to the token index, based on the strong temporal correlation between the image frames and audio stream in the video, to obtain the target audio and video feature sequence 610 in which audio and video features coexist.

[0050] Step 503: Input the target audio and video feature sequence into the large language model to extract key features from the video and audio data.

[0051] In this embodiment, the aforementioned execution entity can input the target audio-visual feature sequence into a large language model to extract key features from the video and audio data. Specifically, after inputting the target audio-visual feature sequence into the large language model, feature extraction can be performed based on the attention matrix of the target audio-visual feature sequence calculated by the large language model to obtain key features of the video and audio data. These key features are then used as content of interest to guide and filter the output of the large language model. For example, the attention matrix could be the attention matrix from the pre-filling stage of the large language model.

[0052] For example, such as Figure 6 As shown, the target audio / video feature sequence 610 is concatenated with the text feature sequence 609 and input into the large language model 611 to obtain the attention matrix 612 in the pre-filling stage. Key features 614 are extracted from the attention matrix of the non-text part (prompt), i.e., the attention matrix 613 of the content part, to obtain the content of interest in the video frame, i.e., high interest point elements 615, which are used to guide and filter the output of the large language model 611. The high interest point elements 615 can be the competition text information in the upper left corner of the video frame and the athletes in key positions below, such as... Figure 4 The areas indicated by 401 and 402.

[0053] This embodiment serializes the video and audio data included in the multimodal data packet separately, and fuses the video feature sequence and audio feature sequence according to the time dimension. This allows the target audio and video features of the input large language model to satisfy the strong correlation between video data and audio data in the spatiotemporal dimension, which can ensure the accuracy of key feature extraction.

[0054] Figure 7 This disclosure illustrates some embodiments of a process that guides large language model reasoning through key features. For example... Figure 7 As shown, based on key features, reasoning is performed using a large language model to output target text containing key information from multimodal data packets. This can include the following steps: Step 701: Match the text features of the multimodal data packets determined by the large language model with the key features.

[0055] In this embodiment, the execution entity (e.g., Figure 1Terminal devices 101, 102, and 103 can match the text features of multimodal data packets determined by the large language model with key features. Since the text output by the large language model may contain a lot of non-key information, by using the key features extracted from the attention matrix as latent variables and matching them with the text features of the multimodal data packets determined by the large language model, contrastive learning can be used to filter the text features in the multimodal data packets determined by the large language model.

[0056] Step 702: Based on the matching results, adjust the scores of the words in the vocabulary score vector corresponding to the text features determined by the large language model.

[0057] In this embodiment, the aforementioned execution entity can adjust the scores of words in the vocabulary score vector corresponding to the text features determined by the large language model based on the matching results. Specifically, the text features determined by the large language model score (logits) the words in the vocabulary, resulting in a vocabulary score vector. The vocabulary words can be matched with key features, and the scores of the corresponding words in the vocabulary score vector corresponding to the text features determined by the large language model are adjusted based on the matching results. For example, a word's score in the vocabulary can be increased if it is similar to a key feature, or its score can be decreased if it is unrelated to a key feature.

[0058] Step 703: Decode the adjusted vocabulary score vector to generate target text containing key information from the multimodal data packet.

[0059] In this embodiment, the aforementioned execution entity can decode the adjusted vocabulary score vector to generate target text containing key information from the multimodal data packet. Specifically, after obtaining the vocabulary score vector adjusted according to key features, a decoder can be used to decode the adjusted vocabulary score vector to obtain the target text containing key information from the multimodal data packet.

[0060] For example, such as Figure 6As shown, the text output by the large language model may contain non-critical information such as "grassland," "audience," and "narration subtitles." By proposing key features 614 as latent variables and performing contrastive learning, non-critical information in the text 616 output by the large language model can be filtered out, resulting in target text containing key information, i.e., high-information elements 617. The contrastive learning difference matching strategy can include: constructing a latent variable feature library containing all key features from the key features obtained from the attention matrix; searching the vocabulary score vector output by the large language model in the feature library; and using the idea of ​​contrastive learning to match and score all output words, adjusting the scores of words in the vocabulary score vector based on the matching scores: if a word has similar features in the feature library, it can be considered a positive sample; otherwise, it is considered a negative sample. After obtaining the filtered vocabulary score vector, the decoder decodes it to obtain the corresponding content.

[0061] This embodiment utilizes key features extracted from the attention matrix to perform inference on a large language model. This can enhance the key information in the multimodal data packets contained in the text output by the large language model, enabling adaptive selection and filtering of the output of the large language model. It can automatically extract all key information from massive amounts of noisy data, thereby improving the quality of the dataset.

[0062] In some optional embodiments of this disclosure, the dataset management method may further include: performing keyword matching between the target text and text stored in a data server; encoding the target text to obtain a corresponding feature vector in response to a successful keyword match; performing information retrieval and text generation on the feature vector based on a retrieval enhancement generation system; and updating the target text based on the generated text. Specifically, after the large language model outputs target text containing key information from the multimodal data package, the executing entity can further perform keyword matching on the target text through a preset data server, and perform information retrieval and text generation based on the retrieval enhancement generation system to update the target text, ensuring that the obtained target text contains all key information and is information-rich, thereby further improving the quality of the dataset.

[0063] For example, such as Figure 6As shown, the preset data server may include a text data server 618 and a vector data server 619. The target text, i.e., the high-information element 617, can be matched with the text stored in the data server 618 using Levenshtein distance for keyword matching. The high-information element 617 that successfully matches the keywords can be encoded, for example, through feature encoding using the CLIP text encoder 620, or through other text encoders, to obtain the feature vector of the high-information element 617. After obtaining the feature vector of the high-information element 617, semantic matching can be performed between the feature vector and the pre-stored text feature vectors in the data server 619. The text corresponding to the text feature vector that semantically matches the feature vector is used to update the high-information element. The text pre-stored in the data server 619 is generated using a Retrieval-augmented Generation (RAG) system. The resulting high-information-content element should meet two conditions: 1) The keyword matching results are consistent, for example, "AXX vsBXX" in the high-information-content element matches the corresponding match title on the server; 2) There is a record in the server's RAG that has a high feature similarity to the high-information-content element.

[0064] Figure 8 The flowcharts of dataset management methods according to other embodiments of this disclosure are shown. For example... Figure 8 As shown, this dataset management method may include the following steps: Step 801: Perform time step alignment and filtering on the multimodal user historical data to obtain preprocessed multimodal data.

[0065] In this embodiment, the execution entity (e.g., Figure 1 The server 105 or terminal devices 101, 102, and 103 can perform time-step alignment and filtering on multimodal user historical data to obtain preprocessed multimodal data. Specifically, after collecting a large amount of coarse and poorly structured multimodal user historical data through data collection tools, time-step alignment and preliminary filtering can be performed on the multimodal historical data to filter out some invalid or useless data, resulting in preprocessed multimodal data. For example, data such as video and audio that do not contain information can be filtered out to facilitate subsequent data processing. The implementation method of the filtering process in this disclosure is not limited.

[0066] Step 802: Based on a preset time window determined by a preset time interval, sample the preprocessed multimodal data and divide the preprocessed multimodal data into multimodal data packets.

[0067] In this embodiment, the aforementioned execution entity can sample the preprocessed multimodal data according to a preset time window determined by a preset time interval, and divide the preprocessed multimodal data into multimodal data packets. Specifically, the time window can be determined according to the time interval in a preset data fragmentation strategy, and the preprocessed multimodal data can be sampled according to the time window, dividing the preprocessed multimodal data into multiple multimodal data packets, thereby achieving batch processing of the multimodal data.

[0068] Step 803: Construct an initial knowledge graph associated with the multimodal data packet, wherein the initial knowledge graph contains time information and entity relationship pairs, and the entity relationship pairs contain entity placeholders.

[0069] In this embodiment, the aforementioned execution entity can construct an initial knowledge graph associated with the multimodal data packets. This initial knowledge graph can contain time information and entity relationship pairs, where each entity relationship pair contains entity placeholders. To achieve structured data construction and storage, after dividing the multimodal data into multimodal data packets, an associated small knowledge graph, i.e., the initial knowledge graph, can be constructed for each multimodal data packet. For example, as... Figure 3 As shown, t can be k The initial knowledge graph corresponding to the time window is denoted as KG_t k , k≤T / a, initial knowledge graph KG_t k It only contains entity-relationship (ER) pairs of the current time step and data bits. For example... Figure 9 As shown, t k The initial knowledge graph KG_t corresponding to the time window k It can contain several default entity placeholders, such as Program, Actor, Genre, etc. Optionally, the global primitive knowledge graph KG_T can also be initialized at the same time.

[0070] Step 804: Input the multimodal data packets into the large language model and extract the key features of the multimodal data packets.

[0071] Step 805: Based on key features, reasoning is performed using a large language model to output target text containing key information from the multimodal data packet.

[0072] Steps 804 and 805 and as follows Figure 2 Steps 202 and 203 shown are the same; please refer to the original text for the identical parts. Figure 2 The corresponding parts of the illustrated implementation are not described in detail here.

[0073] Step 806: Based on the target text, update the entity placeholders in the associated initial knowledge graph to obtain a local knowledge graph.

[0074] In this embodiment, the aforementioned execution entity can update the entity placeholders in the associated initial knowledge graph based on the target text to obtain a local knowledge graph. Specifically, after obtaining the target text corresponding to the multimodal data packet, the target text can be filled into the initial knowledge graph to replace the entity placeholders, thus obtaining a local knowledge graph. For example, as... Figure 9 As shown, the multimodal data packet 901 and its associated initial knowledge graph 902 can be used as input to the multimodal data mining module 903. The multimodal data mining module 903 uses a large language model to extract keywords and other information to obtain target text 904 containing key information from the multimodal data packet. After obtaining the target text 904, the entity placeholders in the initial knowledge graph 902 can be updated based on the target text 904, replacing the entity placeholders with more refined content, such as updating the entity placeholders [program] to [FIFA], [Actor] to [Cxxx], and [Genre] to [sports / football], thereby obtaining the corresponding local knowledge graph 905.

[0075] Step 807: Update the global knowledge graph based on the local knowledge graph to obtain the user historical dataset.

[0076] In this embodiment, the aforementioned execution entity can update the global knowledge graph based on the local knowledge graph to obtain the user's historical dataset. Specifically, after obtaining the local knowledge graph, the global knowledge graph can be populated and updated based on the currently obtained local knowledge graph, and the used local knowledge graph can be released after updating the global knowledge graph, such as... Figure 3 As shown. Optionally, when updating entity placeholders in the initial knowledge graph based on the target text, the target text can also be filtered based on time information in the initial knowledge graph to remove invalid information. For example, as Figure 9 As shown, the time information of target text 906 is 02 / 16, which does not match the time information 05 / 06 in the initial knowledge graph. Therefore, target text 906 is invalid information and should be filtered out. The time information of target text 907 is 05 / 06, which matches the time information 05 / 06 in the initial knowledge graph. Therefore, target text 907 is valid information and can be used to update the initial knowledge graph.

[0077] This embodiment uses a sequence-to-sequence approach to model multimodal user historical data, constructs a historical information retrieval pipeline, extracts high-quality datasets under an end-to-end paradigm, and adopts a data batch generation framework, using a knowledge graph as a record container for data structuring.

[0078] Figure 10 The flowcharts of model training methods according to some embodiments of this disclosure are shown. For example... Figure 10 As shown, the model training method may include the following steps: Step 1001: Based on the user's historical dataset, construct the training sample set and sample labels for each task scenario using prompt word templates and response templates.

[0079] In this embodiment, the execution entity (e.g., Figure 1 The server 105 or terminal devices 101, 102, and 103 can construct training sample sets and sample labels corresponding to each task scenario based on the user's historical dataset, using prompt word templates and response templates. The model can include a pre-trained large language model and multiple adaptation layers adapted to various task scenarios. The user historical dataset can be obtained using the dataset management method described in any of the above embodiments. The prompt word templates can be configured according to the task requirements of the corresponding task scenario, and the response templates can be configured according to the response format of the corresponding task scenario.

[0080] In this embodiment, the pre-trained large language model can be a model pre-trained on existing general knowledge corpora, serving as the upstream base model. A series of independent adapter layers can be defined based on the actual application task scenario to learn domain-specific knowledge for each application task scenario, and each adapter layer can be randomly initialized. The user's historical dataset can be obtained using any of the dataset management methods described above, and training sample sets and their corresponding sample labels can be constructed for each task scenario. Prompt word templates and response templates specific to each task scenario can be pre-constructed to build the training sample sets and sample labels for each task scenario.

[0081] Step 1002: Input the training sample set of each task scenario into the pre-trained large language model, obtain the output result through the large language model and the corresponding adaptation layer, and train the adaptation layer based on the output result and the corresponding sample label.

[0082] In this embodiment, the aforementioned execution entity can input the training sample set of each task scenario into a pre-trained large language model. The large language model and its corresponding adaptation layer then produce an output result. Based on the output result and the corresponding sample labels, the adaptation layer is trained. Specifically, during model training based on the training sample set and its labels for each task scenario, the parameters of the pre-trained large language model, which serves as the upstream foundation model, are frozen and do not participate in parameter updates. Only the parameters of the adaptation layer for each task scenario are updated, thereby obtaining an expert model corresponding to each task scenario through model training.

[0083] In some optional implementations, the training sample set for each task scenario is input into the pre-trained large language model, and the output result is obtained after passing through the large language model and the corresponding adaptation layer. The adaptation layer is trained based on the output result and the corresponding sample labels, which may include: determining a first training sample set and a first sample label, as well as a second training sample set and a second sample label, based on the training sample set and sample labels for each task scenario, wherein the first training sample set and the second training sample set contain different training samples; inputting the first training sample set into the pre-trained large language model, and the output result is obtained after passing through the large language model and the corresponding adaptation layer; determining the loss value based on the output result and the first sample label; performing cold-start supervised fine-tuning on the adaptation layer to obtain the basic adaptation layer; inputting the second training sample set into the pre-trained large language model, and the output result is obtained after passing through the large language model and the corresponding basic adaptation layer; determining the reward value based on the output result and the second sample label; and performing online policy reinforcement learning training on the basic adaptation layer, wherein the reward value includes a format reward component and a recall reward component.

[0084] For example, such as Figure 11 As shown, the user history dataset 1101 can be used as the basis for model optimization and iteration in this stage. An open-source LLM can be used as the base model 1102, which has been pre-trained based on general knowledge. Corresponding adaptation layers 1103, 1104, 1105, and 1106, also known as adapters, can be constructed for different task scenarios, acting as domain experts. Task scenarios may include game scenarios, film and television scenarios, art modes, and behavior records. These task scenarios are merely examples and are not intended to limit the task scenarios applied to the implementation methods of this disclosure. Prompt word templates 1107 and response templates are defined for different task scenarios. Based on the user history dataset 1101, training sample sets and their sample labels 1108 for each task scenario can be generated using the prompt word templates and response templates. To improve the robustness and effectiveness of the model response, the training sample sets for each task scenario can be divided into a first training sample set and a second training sample set, and each adaptation layer can be trained in two stages using the first and second training sample sets respectively.

[0085] The first stage is Code-start Supervised Fine-tuning (SFT) 1109. In the SFT stage, real user history records and response templates can be used as ground truth (GT). The model can learn to adapt its knowledge and responses to different tasks based on the prompt word templates, and can quickly learn response formats and domain knowledge in different scenarios. The second stage is On-Policy Reinforcement Learning (RL) 1110. The reward 1111 in the reinforcement learning stage consists of format reward and recall reward.

[0086] The format reward function standardizes the model's output format, maximizing its consistency with the output labels in the dataset, which is beneficial for extracting key elements during the inference phase. The recall reward function measures the amount of key historical information about the user contained in the model's output, making the optimization objective to maximize the recall of the user's historical key data. This training phase employs multi-reward reinforcement learning, which can improve the model's ability in terms of output format and key element recall.

[0087] Optionally, in a batch of training data, different expert paths can be selected through a preset scene token as a gating mechanism to reach the adaptation layer of different task scenarios.

[0088] This embodiment designs a continuous training pipeline with multiple expert adapters, and achieves dynamic learning by diverting knowledge from different domains. Knowledge from the user's historical dataset can be injected into multiple lightweight expert adapters. Key information can be extracted from low-quality user behavior records to optimize dedicated models for different tasks, resulting in a set of service systems that support any scenario and provide diverse agent-assisted services.

[0089] Figure 12 The flowchart of a user preference-aware method according to some embodiments of this disclosure is shown. For example... Figure 12 As shown, this user preference awareness method is applied to an agent, which may include a large model trained according to any of the above model training methods. The user preference awareness method may include the following steps: Step 1201: In response to receiving the user's access operation for the target task scenario, generate a scenario token corresponding to the target task scenario.

[0090] In this embodiment, the execution entity (e.g., Figure 1 Terminal devices 101, 102, and 103 can generate a scene token corresponding to the target task scene in response to receiving a user's access operation for the target task scene. The scene tokens can be pre-set according to different task scenes. For example, ... Figure 13 As shown, the scene token corresponding to Art Mode 1301 is <art>1302, while the scene token corresponding to Game Center 1303 is <game>1304. During the training phase, the scene token is encoded as a discrete latent space vector and further assigned a route ID. During the inference phase, the corresponding expert adaptation layer and prompt word template can be dynamically selected based on the scene token. For example, a mapping table from scene tokens to IDs can be predefined, as shown in Figure 1.

[0091] Table 1

[0092] For example: Step 1202, based on the scene token, determine the adaptation layer and prompt word template corresponding to the target task scene.

[0093] In this embodiment, the execution entity can determine the adaptation layer and prompt word template corresponding to the target task scenario based on the scenario token. Specifically, when a user enters an app or page, a scenario token specific to that scenario is generated. After determining the corresponding scenario token based on the target task scenario, the execution entity can determine the matching adaptation layer and prompt word template based on the scenario token. For example, the corresponding adaptation layer and prompt word template can be determined through the mapping table 1 from scenario token to ID. For example, as... Figure 13 As shown, the executing entity maps to the red expert path ② through a gating mechanism based on scenario token 1302, and to the blue expert path through a gating mechanism based on scenario token 1304. The adaptation layer corresponding to expert path ② is the art mode expert. The corresponding adaptation layer is Game Expert.

[0094] Step 1203: Generate prompt text based on prompt word template, trigger the pre-trained large language model and adaptation layer to perform inference, and generate auxiliary information reflecting user preferences.

[0095] In this embodiment, the aforementioned execution entity can generate prompt text based on the prompt template and trigger the pre-trained large language model and adaptation layer to perform inference, generating auxiliary information reflecting user preferences. Specifically, the expert adapter and base model in the large model trained according to any of the above model training methods can be used as a whole model to perform model inference, obtaining fine-grained contextual auxiliary information reflecting user preferences. For example, such as... Figure 13 As shown, the executing entity aggregates the prompt word template 1305 into the basic large language model 1306 to generate context information 1307.

[0096] Step 1204: Generate corresponding control commands based on auxiliary information, and perform specific operations on the target task scenario based on the control commands.

[0097] In this embodiment, the aforementioned execution entity can generate corresponding control commands based on the auxiliary information, and perform specific operations on the target task scenario according to the control commands. Specifically, after obtaining fine-grained contextual auxiliary information reflecting user preferences, the execution entity can generate control commands for the auxiliary agent to perform various targeted actions based on the auxiliary information, and perform specific operations on the target task scenario according to the control commands.

[0098] The user preference perception method provided in this embodiment does not require users to make choices during the inference process. Instead, it is adaptively adjusted by the model and the scene, which can greatly improve the user experience when using various intelligent agents.

[0099] Figure 14 A flowchart illustrating an application scenario using the method provided in this disclosure is shown. Figure 14 As shown, the overall process includes the following three main processes: I. Generation / Maintenance of User History Dataset 1401, The data collection tool collects the content that a user watches every time they use the TV, resulting in a large amount of coarse-grained state metadata.

[0100] 1402. Preprocess the data, organize it in a structured manner, and unify the data format.

[0101] 1403. The structured coarse-grained data is refined and redundancy is removed to obtain a high-quality real user history dataset for subsequent model learning.

[0102] 1404, this process is repeatedly executed iteratively as the usage cycle increases.

[0103] II. Continuous Model Training

[0104] 1405. First, a model that has been pre-trained on existing general knowledge corpus is used as the upstream basic model.

[0105] 1406. Based on the actual application scenario, define a series of independent adapters to learn the application domain's proprietary knowledge and initialize them randomly.

[0106] 1407. Use the user history dataset maintained in process one as the training dataset in this step, and construct a prompt word input template specific to the task scenario.

[0107] 1408. The first step is to perform supervised fine-tuning during the cold start. The model learns to adapt its knowledge and responses to different tasks based on the prompt word templates.

[0108] 1409. The second step uses multi-reward reinforcement learning to improve the model's ability to handle output format and recall key elements.

[0109] III. Multi-expert route reasoning

[0110] 1410. In process two, several expert models adapted to different scenarios are obtained.

[0111] 1411, the user's current activity corresponds to a scene token.

[0112] 1412. Select the corresponding domain expert based on the scenario token and mix it with the base model.

[0113] 1413. Select the prompt word template for this scenario, use the hybrid model for reasoning, and obtain a user-centric, fine-grained environmental context.

[0114] The user preference perception method provided in this disclosure will be described below with reference to specific application scenarios.

[0115] like Figure 15 As shown, Figure 15 This paper illustrates a first application scenario using the user preference awareness method of this disclosure. This application scenario is an agent interaction service scenario within a personalized context. Specifically, area A at the top represents a typical agent interaction service scenario, while area B at the bottom represents an agent interaction service scenario employing the user preference awareness method of this disclosure.

[0116] The user preference awareness method disclosed herein can provide implicit context preloading when a user uses a program agent, allowing the user's usage state to be context-free. Simultaneously, it enables the server-side agent to better understand the current user profile and provide more accurate responses. In a typical scenario for area A, when a user turns on their smart TV, the AI ​​assistant initiates the agent service and asks for recommendations for TV series. However, it doesn't provide much context, such as the user's interests, favorite actors, or production era. Users need to engage in multiple rounds of interaction to continuously provide content information, resulting in a diminished user experience.

[0117] In the scenario where the user preference perception method disclosed herein is used in region B, when a user turns on their smart TV, the AI ​​assistant initiates the intelligent agent service and asks for recommendations for TV series. The optimized environment model of the user-specific intelligent agent can dynamically infer based on the current application scenario, providing the user's historical records in the current environment, helping the intelligent agent service to pre-establish the user's personal context. Users can avoid multiple rounds of interaction and manually providing redundant information.

[0118] like Figure 16 As shown, Figure 16 This paper illustrates a second application scenario using the user preference awareness method of this disclosure. This application scenario is a smart tool calling assistant scenario. The upper area A represents a typical existing smart tool calling assistant scenario, while the lower area B represents a smart tool calling assistant scenario employing the user preference awareness method of this disclosure.

[0119] The user preference perception method disclosed herein may include a user behavior data maintenance component. The model can learn different aspects of user behavior patterns from these datasets, thereby constructing a user preference-based intelligent operation assistant. For example, in a typical scenario for area A, a user intends to watch a movie on their smart TV late at night and wants to set the screen brightness to night mode. However, the user hasn't used the smart TV for some time and is unsure where to find this setting. The user needs to spend time searching for the relevant function area, resulting in a decreased user experience.

[0120] For the scenario in region B where the user preference awareness method of this disclosure is used, if a user intends to watch a movie on their smart TV late at night and wants to set the screen brightness to night mode as before, the model routes to the expert adaptation layer, outputs the user's previous movie-watching related actions, determines the user's intent based on the current environment, and helps the user set their usual settings for nighttime movie watching. When the user turns on the smart TV at night, the system can automatically adjust the user's preference settings according to the environment.

[0121] like Figure 17 As shown, Figure 17 A third application scenario using the user preference-aware method of this disclosure is illustrated. This application scenario involves constructing user preference-based content recommendation based on a user's historical dataset. In this scenario, area A at the top represents a typical existing preference-based content recommendation scenario, while area B at the bottom represents a preference-based content recommendation scenario using the user preference-aware method of this disclosure.

[0122] The dataset used for model optimization using the user preference perception method disclosed herein can also be provided as a database of individual user preferences to relevant third-party services, such as the "Personalized Content" section of a smart TV operating system. However, in the general case of area A, most of the content recommendations provided in the "Personalized Content" section are based on general trending topics. There may be a significant gap between the system's recommendations and the content that users are truly interested in.

[0123] In the case where the user preference perception method disclosed herein is used in area B, the user's historical dataset is provided as a database to the "personalized content" service of the smart TV operating system, which can support dynamic retrieval. The "personalized content" area can provide content that users are truly interested in.

[0124] like Figure 18 As shown, Figure 18 This paper illustrates a fourth application scenario using the user preference awareness method disclosed herein. This scenario is a "personalized prompt" pop-up notification service. In a typical case, a popular TV series, "xxxx," is airing, and the user is watching it daily, not wanting to miss any episodes. However, the system fails to provide a notification, potentially causing the user to miss important plot points.

[0125] In a scenario employing the user preference perception method disclosed herein, where the TV series "xxxx" is currently airing and users are following it daily, not wanting to miss any plot developments, the prompt service can provide plot hint pop-ups based on an intelligent agent's large language model. Users can then access the channel in advance. This allows users to promptly track the series they are currently watching, enhancing their user experience.

[0126] Further reference Figure 19 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a dataset management device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0127] like Figure 19 As shown, the dataset management device in this embodiment may include: a data sub-packaging module 1901, a feature extraction module 1902, a text generation module 1903, and a data update module 1904.

[0128] Among them, the data packet module 1901 is configured to divide multimodal user historical data into multimodal data packets.

[0129] The feature extraction module 1902 is configured to input multimodal data packets into a large language model and extract key features from the multimodal data packets.

[0130] The text generation module 1903 is configured to infer based on key features using a large language model and output target text containing key information from multimodal data packets.

[0131] The data update module 1904 is configured to build a local knowledge graph based on the target text and update the global knowledge graph with the local knowledge graph to obtain the user's historical dataset.

[0132] In some alternative implementations of this disclosure, the feature extraction module 1902 includes: The attention matrix acquisition unit is configured to input the multimodal data packet into the large language model and acquire the attention matrix of the large language model during the pre-filling stage. The attention score calculation unit is configured to normalize each row of the attention matrix and calculate the average value of each column of the normalized attention matrix to obtain an attention score vector. The key feature extraction unit is configured to extract key features based on the features corresponding to high-score positions in the attention score vector.

[0133] In some optional implementations of this disclosure, the multimodal data packet includes video data and audio data; Feature extraction module 1902 includes: The serialization processing unit is configured to encode the video data and the audio data according to their respective modalities to obtain video feature sequences and audio feature sequences; The sequence fusion unit is configured to fuse the video feature sequence and the audio feature sequence according to the time dimension to obtain the target audio and video feature sequence; The key feature extraction unit is configured to input the target audio and video feature sequence into a large language model to extract key features from the video data and the audio data.

[0134] In some alternative implementations of this disclosure, the text generation module 1903 includes: The key feature matching unit is configured to match the text features of the multimodal data packet determined by the large language model with the key features; The vocabulary score adjustment unit is configured to adjust the scores of words in the vocabulary score vector corresponding to the text features determined by the large language model based on the matching results. The target text generation unit is configured to decode the adjusted vocabulary score vector to generate target text containing key information from the multimodal data packet.

[0135] In some alternative implementations of this disclosure, the dataset management apparatus further includes: The keyword matching module is configured to perform keyword matching between the target text and the text stored in the data server; The feature encoding module is configured to encode the target text in response to a successful keyword match, thereby obtaining a corresponding feature vector; The target text update module is configured to perform information retrieval and text generation on the feature vector based on the retrieval enhancement generation system, and update the target text based on the generated text.

[0136] In some optional implementations of this disclosure, the data packet module 1901 includes: The preprocessing unit is configured to perform time-step alignment and filtering on multimodal user historical data to obtain preprocessed multimodal data; The data sampling unit is configured to sample the preprocessed multimodal data within a preset time window determined by a preset time interval, and to divide the preprocessed multimodal data into multimodal data packets.

[0137] In some optional implementations of this disclosure, the data packet module 1901 further includes: An initial knowledge graph construction unit is configured to construct an initial knowledge graph associated with the multimodal data packet, wherein the initial knowledge graph contains time information and entity relationship pairs, and the entity relationship pairs contain entity placeholders; Data update module 1904 includes: The local knowledge graph update unit is configured to update the entity placeholders in the associated initial knowledge graph based on the target text to obtain a local knowledge graph. The global knowledge graph update unit is configured to update the global knowledge graph based on the local knowledge graph to obtain the user's historical dataset.

[0138] Further reference Figure 20 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a model training device, which is similar to... Figure 10 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0139] like Figure 20 As shown, the dataset management device in this embodiment may include a sample generation module 2001 and a model training module 2002. The model includes a pre-trained large language model and multiple adaptation layers adapted to various task scenarios.

[0140] The sample generation module 2001 is configured to construct training sample sets and sample labels for each task scenario based on the user historical dataset, using prompt word templates and response templates, respectively. The user historical dataset is obtained according to the dataset management method described in any of the above embodiments, the prompt word template is configured according to the task requirements of the corresponding task scenario, and the response template is configured according to the response format of the corresponding task scenario. The model training module 2002 is configured to input the training sample set of each task scenario into the pre-trained large language model, obtain the output result through the large language model and the corresponding adaptation layer, and train the adaptation layer based on the output result and the corresponding sample label.

[0141] In some alternative implementations of this disclosure, the model training module 2002 includes: The training sample partitioning unit is configured to determine a first training sample set and a first sample label, as well as a second training sample set and a second sample label, based on the training sample set and the sample label for each task scenario, wherein the first training sample set and the second training sample set contain different training samples. The cold start supervised fine-tuning unit is configured to input the first training sample set into the pre-trained large language model, obtain the output result through the large language model and the corresponding adaptation layer, determine the loss value based on the output result and the first sample label, and perform cold start supervised fine-tuning on the adaptation layer to obtain the basic adaptation layer. The online policy reinforcement learning unit is configured to input the second training sample set into the pre-trained large language model, obtain the output result through the large language model and the corresponding basic adaptation layer, determine the reward value based on the output result and the second sample label, and perform online policy reinforcement learning training on the basic adaptation layer, wherein the reward value includes a format reward component and a recall reward component.

[0142] like Figure 21 As shown, the intelligent agent in this embodiment may include: an input module 2101, a processing module 2102, and an output module 2103. The model includes: a pre-trained large language model and multiple adaptation layers adapted to various task scenarios.

[0143] The input module 2101 is used to receive input information.

[0144] Processing module 2102 is configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and execute the large model by calling it. Figure 11 The method described above yields output information.

[0145] The output module 2103 is used to output the output information obtained by the processing module.

[0146] The solution disclosed herein overcomes the shortcomings of inefficiency and difficulty in maintenance in existing technologies. It proposes a multi-task joint learning framework that extracts key information from low-quality user behavior records to optimize dedicated models for different tasks. This effectively avoids the defects in existing solutions, enabling a single system to support any use case and provide a variety of agent-assisted services.

[0147] The solution disclosed herein provides a user-centric interactive environment when users interact with various AI agents or service functions, and reduces the user's input burden by relying on a professional large language model for the AI ​​agent.

[0148] The solution disclosed herein addresses the server load problem caused by the growth of massive historical data by proposing a dynamic automatic training dataset maintenance engine that injects knowledge into multiple lightweight expert adapters.

[0149] The solution provided in this public disclosure uses a customized intelligent agent large language model, which allows users to have their own "butler model" to improve the user experience and efficiency.

[0150] The solution disclosed herein proposes a novel dataset generation and maintenance engine for efficiently managing user-level data, providing high-quality, fine-grained, and comprehensive datasets for building user-specific intelligent agent large language models.

[0151] The solution provided in this disclosure addresses the problems of exploitation of various user history records (UHR) and catastrophic forgetting by designing a continuous training pipeline based on a multi-expert adapter, which achieves dynamic learning by diverting knowledge from different domains.

[0152] The solution disclosed herein uses a gating token as the trigger mechanism for scene switching, thereby improving system flexibility.

[0153] The dataset constructed using the scheme disclosed herein can serve as an external auxiliary resource for various types of services.

[0154] The solutions provided in this disclosure are not limited to televisions and smartphones, but can also be deployed on other devices, such as smart home appliances and other platforms.

[0155] According to embodiments of this disclosure, this disclosure also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0156] Figure 22 This is a block diagram of an electronic device suitable for implementing embodiments of the present disclosure. For example... Figure 22 As shown, the electronic device includes one or more processors 2201, a memory 2202, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 22 Take a processor 2201 as an example.

[0157] The memory 2202 is the non-transitory computer-readable storage medium provided in this disclosure. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the methods provided in this disclosure. The non-transitory computer-readable storage medium of this disclosure stores computer instructions for causing a computer to perform the methods provided in this disclosure.

[0158] Memory 2202, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure (e.g., appendix). Figure 19 The data packetization module 1901, feature extraction module 1902, text generation module 1903, and data update module 1904 are shown. The processor 2201 executes various server functions and data processing by running non-transient software programs, instructions, and modules stored in the memory 2202, thereby implementing the methods in the above method embodiments.

[0159] Memory 2202 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the electronic device during historical video playback. Furthermore, memory 2202 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 2202 may optionally include memory remotely located relative to processor 2201, and these remote memories may be connected to the electronic device performing the method via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0160] The electronic device executing the method may further include an input device 2203 and an output device 2204. The processor 2201, memory 2202, input device 2203, and output device 2204 may be connected via a bus or other means. Figure 22 Taking the example of a connection between China and Israel via a bus.

[0161] Input device 2203 can receive input digital or character information, as well as key signal input related to user settings and function control of electronic devices playing video streams, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 2204 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0162] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0163] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0164] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0165] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0166] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0167] The technical solution according to the embodiments of this disclosure effectively improves the accuracy and effectiveness of fault location by the trained fault location model.

[0168] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0169] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / game> < / art>

Claims

1. A dataset management method, comprising: Divide multimodal user historical data into multimodal data packets; The multimodal data packets are input into a large language model to extract the key features of the multimodal data packets; Based on the key features, reasoning is performed using the large language model to output target text containing key information from the multimodal data packet; A local knowledge graph is built based on the target text, and the global knowledge graph is updated with the local knowledge graph to obtain the user's historical dataset.

2. The method according to claim 1, wherein, The step of inputting the multimodal data packet into a large language model and extracting the key features of the multimodal data packet includes: The multimodal data packet is input into the large language model to obtain the attention matrix of the large language model in the pre-filling stage; Normalize each row of the attention matrix and calculate the average value of each column of the normalized attention matrix to obtain the attention score vector; Key features are extracted based on the features corresponding to high-score positions in the attention score vector.

3. The method according to claim 1, wherein, The multimodal data packet includes video data and audio data; The step of inputting the multimodal data packet into a large language model and extracting the key features of the multimodal data packet includes: The video data and the audio data are encoded according to their respective modalities to obtain video feature sequences and audio feature sequences; The video feature sequence and the audio feature sequence are fused along the time dimension to obtain the target audio-video feature sequence; The target audio and video feature sequence is input into a large language model to extract key features from the video data and the audio data.

4. The method according to claim 1, wherein, The step of reasoning based on the key features using the large language model to output target text containing key information from the multimodal data packet includes: The text features of the multimodal data packets determined by the large language model are matched with the key features; Based on the matching results, adjust the scores of the words in the vocabulary score vector corresponding to the text features determined by the large language model; The adjusted vocabulary score vector is decoded to generate target text containing key information from the multimodal data packet.

5. The method according to claim 1, further comprising: The target text is matched with the text stored in the data server using keywords; In response to a successful keyword match, the target text is encoded to obtain the corresponding feature vector; Based on the retrieval enhancement generation system, information retrieval and text generation are performed on the feature vector, and the target text is updated based on the generated text.

6. The method according to any one of claims 1-5, wherein, The process of dividing multimodal user historical data into multimodal data packets includes: Time step alignment and filtering are performed on multimodal user historical data to obtain preprocessed multimodal data; Based on a preset time window determined by a preset time interval, the preprocessed multimodal data is sampled and divided into multimodal data packets.

7. The method according to claim 6, wherein, The step of dividing multimodal user historical data into multimodal data packets also includes: Construct an initial knowledge graph associated with the multimodal data packet, wherein the initial knowledge graph contains time information and entity relationship pairs, and the entity relationship pairs contain entity placeholders; The process of establishing a local knowledge graph based on the target text and updating the global knowledge graph with the local knowledge graph to obtain the user's historical dataset includes: Based on the target text, the entity placeholders in the associated initial knowledge graph are updated to obtain a local knowledge graph; Based on the local knowledge graph, the global knowledge graph is updated to obtain the user's historical dataset.

8. A model training method, wherein, The model includes: a pre-trained large language model and multiple adaptation layers adapted to various task scenarios; The training method includes: Based on the user history dataset, training sample sets and sample labels corresponding to each task scenario are constructed through prompt word templates and response templates, respectively. The user history dataset is obtained according to the dataset management method according to any one of claims 1-7, the prompt word template is configured according to the task requirements of the corresponding task scenario, and the response template is configured according to the response format of the corresponding task scenario. The training sample set for each task scenario is input into the pre-trained large language model. The large language model and the corresponding adaptation layer are used to obtain the output result. The adaptation layer is trained based on the output result and the corresponding sample label.

9. The method according to claim 8, wherein, The step of inputting the training sample set for each task scenario into the pre-trained large language model, obtaining the output result through the large language model and the corresponding adaptation layer, and training the adaptation layer based on the output result and the corresponding sample labels includes: Based on the training sample set and the sample label for each task scenario, a first training sample set and a first sample label, as well as a second training sample set and a second sample label, are determined, wherein the first training sample set and the second training sample set contain different training samples; The first training sample set is input into the pre-trained large language model. The output result is obtained through the large language model and the corresponding adaptation layer. The loss value is determined based on the output result and the first sample label. The adaptation layer is then subjected to cold start supervised fine-tuning to obtain the basic adaptation layer. The second training sample set is input into the pre-trained large language model. The output result is obtained through the large language model and the corresponding basic adaptation layer. The reward value is determined based on the output result and the second sample label. Online policy reinforcement learning training is performed on the basic adaptation layer. The reward value includes a format reward component and a recall reward component.

10. A user preference perception method applied to an intelligent agent, the intelligent agent comprising a large model trained according to any one of claims 8-9; The method includes: In response to receiving a user's access operation for a target task scenario, a scenario token corresponding to the target task scenario is generated; Based on the scene token, determine the adaptation layer and prompt word template corresponding to the target task scene; Based on the prompt word template, prompt word text is generated, triggering the pre-trained large language model and the adaptation layer to perform inference and generate auxiliary information reflecting the user's preferences; Based on the auxiliary information, corresponding control commands are generated, and specific operations are performed on the target task scenario based on the control commands.

11. An intelligent agent, comprising: The input module is used to receive input information; The processing module is configured to determine a target task based on the input information received by the input module, determine a large model based on the target task, and execute the method described in claim 10 by calling the large model to obtain output information; An output module is used to output the output information obtained by the processing module.

12. A dataset management device, comprising: The data packetization module is configured to divide multimodal user historical data into multimodal data packets; The feature extraction module is configured to input the multimodal data packet into a large language model and extract key features of the multimodal data packet; The text generation module is configured to perform inference based on the key features using the large language model, and output target text containing key information from the multimodal data packet. The data update module is configured to build a local knowledge graph based on the target text and update the global knowledge graph with the local knowledge graph to obtain the user's historical dataset.

13. A model training device, wherein, The model includes: a pre-trained large language model and multiple adaptation layers adapted to various task scenarios; The training device includes: The sample generation module is configured to construct training sample sets and sample labels for each task scenario based on the user's historical dataset, using prompt word templates and response templates, respectively. The user's historical dataset is obtained according to the dataset management method according to any one of claims 1-7. The prompt word template is configured according to the task requirements of the corresponding task scenario, and the response template is configured according to the response format of the corresponding task scenario. The model training module is configured to input the training sample set of each task scenario into the pre-trained large language model, obtain the output result through the large language model and the corresponding adaptation layer, and train the adaptation layer based on the output result and the corresponding sample labels.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores information that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-7, 8-9, and 10.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7, 8-9, and 10.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7, 8-9, and 10.

Images