Data processing method, terminal device and program product

Abstract

The invention is suitable for the technical field of computers, and provides a data processing method and device, terminal equipment and a program product, and the method comprises the steps: obtaining an interaction record between an intelligent agent and a user; performing vector conversion processing on the interaction record to obtain vectorized data corresponding to the interaction record; analyzing the interaction record to obtain graph data corresponding to the interaction record; the graph data is knowledge graph structure data, the graph data comprises nodes, edges and attributes, the nodes are used for describing entities, and the edges are used for describing the relation between any two entities; storing the vectorized data into a vector database, and storing the graph data into a graph database; all graph data in the graph database are associated through edges. Compared with the prior art, the interaction records are stored in the vector database and the graph database, so that the memory storage fragmentation problem is solved, and the interaction records can form a coherent memory network.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a data processing method, terminal equipment and program product. Background Art

[0002] An Artificial Intelligence Agent (AI Agent) is an intelligent entity capable of perceiving its environment, making decisions, and performing actions. It uses a large language model as its core computing engine, enabling it to engage in dialogue, perform tasks, reason, and exhibit a degree of autonomy. When engaging in dialogue, performing tasks, and reasoning, the AI ​​Agent typically needs to analyze various stored memories to determine its next steps. Before analyzing these stored memories, it is necessary to retrieve all of the AI ​​Agent's memories to obtain the required information. Therefore, to achieve information retrieval for the AI ​​Agent, it is necessary to manage the memories formed from the interaction data generated during the interaction between the AI ​​Agent and the user.

[0003] However, existing technologies typically store these interaction records directly as isolated text. Text is typically organized into units such as characters, words, sentences, and paragraphs. In computer storage, these elements are stored in a dispersed manner. This demonstrates that existing technologies suffer from the problem of memory fragmentation. Summary of the Invention

[0004] This application provides a data processing method, terminal device, and program product to solve the problem of memory storage fragmentation in the prior art.

[0005] In a first aspect, embodiments of this application provide a data processing method, including:

[0006] Acquire records of interactions between the intelligent agent and the user;

[0007] The interaction record is subjected to vector conversion processing to obtain the vectorized data corresponding to the interaction record;

[0008] The interaction record is parsed to obtain the graph data corresponding to the interaction record;

[0009] The vectorized data is stored in a vector database, and the graph data is stored in a graph database; the graph data in the graph database are associated with each other through edges.

[0010] Optionally, the interaction record includes dialogue information between multiple users and the intelligent agent; after acquiring the interaction record between the intelligent agent and the user, it further includes:

[0011] Evaluate the plurality of dialogue messages based on a set rule to obtain initial weights for the plurality of dialogue messages;

[0012] Obtain the user's preference information;

[0013] adjusting the initial weights of the plurality of conversation messages according to the preference information to obtain target weights of the plurality of conversation messages;

[0014] Determine a target score for each of the plurality of conversation messages based on the target weight; the target score for each message is used to describe the importance of each conversation message;

[0015] Accordingly, storing the vectorized data in a vector database and storing the graph data in a graph database includes:

[0016] The target scores of each of the multiple dialogue messages are associated with the vectorized data corresponding to each of the multiple dialogue messages and stored in the vector database. The target scores of each of the multiple dialogue messages are associated with the graph data corresponding to each of the multiple dialogue messages and stored in the graph database.

[0017] Optionally, the interaction record includes dialogue information between multiple users and the intelligent agent; before storing the vectorized data in a vector database and storing the graph data in a graph database, the method further includes:

[0018] Determining response results corresponding to each of the plurality of dialogue messages; the response results are used to describe whether the agent responds to the user's message during the interaction with the user;

[0019] Filtering target messages whose response results are unanswered from the plurality of conversation messages;

[0020] Analyzing the target information to obtain target response information corresponding to the target information;

[0021] Accordingly, storing the vectorized data in a vector database and storing the graph data in a graph database includes:

[0022] The vectorized data corresponding to the target response information is stored in the vector database, and the graph data corresponding to the response information is stored in the graph database.

[0023] Optionally, analyzing the target information to obtain target response information corresponding to the target information includes:

[0024] Inputting the target information into the first language model for processing to obtain question information corresponding to the target information;

[0025] Search the vector database and the graph database for historical response information corresponding to the question information;

[0026] The historical response information and the question information are input into the second language model for processing to obtain the target response information corresponding to the question information.

[0027] Optionally, after storing the vectorized data in a vector database and storing the graph data in a graph database, the method further includes:

[0028] When the interaction between the user and the intelligent agent is detected, the content information input by the user is obtained;

[0029] The first content corresponding to the content information is selected from the vector database;

[0030] The second content corresponding to the content information is selected from the graph database;

[0031] The first content and the second content are combined to obtain the target retrieval content corresponding to the content information.

[0032] Optionally, each vectorized data in the vector database carries a score and a generation time, and a target score of each vectorized data is used to describe the importance of each vectorized data; the first content includes multiple items; and filtering out the first content corresponding to the content information from the vector database includes:

[0033] The content information is vectorized to obtain the first vectorized data corresponding to the content information;

[0034] The similarity between each vectorized data in the vector database and the first vectorized data is calculated to obtain each similarity score;

[0035] Obtain the first number of backup vectorized data in descending order of similarity scores;

[0036] evaluating each of the spare vectorized data based on the similarity score, the score, and the generation time of each of the spare vectorized data to obtain a target score for each of the spare vectorized data;

[0037] A second quantity of candidate vectorized data is obtained in descending order of target scores, and the second quantity of candidate vectorized data is determined as the first content; the second quantity is smaller than the first quantity.

[0038] Optionally, the graph database includes multiple graph data; each graph data includes edges and nodes; the edges are used to describe the relationship between any two entities; the step of filtering out the second content corresponding to the content information from the graph database includes:

[0039] The similarity between each node in the graph database and the content information is calculated to obtain the similar nodes corresponding to the content information;

[0040] Each node connected to the similar node is identified as an associated node;

[0041] The content corresponding to the similar nodes and the associated nodes is determined as the second content.

[0042] Optionally, combining the first content and the second content to obtain target search content corresponding to the content information includes:

[0043] Filtering out third content corresponding to the second content from the vector database;

[0044] The first content, the second content, and the third content are combined to obtain the target search content.

[0045] Secondly, embodiments of this application provide a data processing apparatus, including:

[0046] A first acquisition unit, configured to acquire interaction records between the agent and the user;

[0047] a first processing unit, configured to perform vector conversion processing on the interaction record to obtain vectorized data corresponding to the interaction record;

[0048] a parsing unit, configured to parse the interaction record to obtain graph data corresponding to the interaction record;

[0049] The first storage unit is used to store the vectorized data in a vector database and the graph data in a graph database; the graph data in the graph database are associated with each other through edges.

[0050] Thirdly, embodiments of this application provide a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data processing method as described in any one of the first aspects above.

[0051] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data processing method as described in any one of the first aspects above.

[0052] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, enables the terminal device to execute the data processing method described in any one of the first aspects.

[0053] The beneficial effects of the embodiments in this application compared with the prior art are:

[0054] A data processing method provided by an embodiment of the present application obtains the interaction records between an intelligent agent and a user; performs vector conversion processing on the interaction records to obtain vectorized data corresponding to the interaction records; performs analysis on the interaction records to obtain graph data corresponding to the interaction records; stores the vectorized data in a vector database, and stores the graph data in a graph database; and the graph data in the graph database are associated through edges. On the one hand, the present application vectorizes the interaction records so that the processed vectorized data is more concise, which facilitates the subsequent retrieval of related data from the vector database, i.e., improves the retrieval efficiency. At the same time, the present application stores the vectorized data in a vector database. Since vectorized data usually represents data features and relationships in numerical form, the distance between vectorized data with similar values ​​in the vector database is closer, and the distance between vectorized data with far values ​​is far, thereby achieving the storage consistency of the memory corresponding to the intelligent agent in the vector database and solving the problem of memory storage fragmentation. On the other hand, the present application parses the interaction records to obtain graph data which is knowledge graph structure data. Since the graph data includes nodes and edges, by constructing nodes and connecting related nodes with edges, the internal structure of the interaction is clearly displayed. At the same time, each graph data in the graph database is associated by edges to form a complex relationship network, which can intuitively present the logical connection, hierarchical structure, etc. between the interaction records, that is, it can intuitively display the connection and dependency between each entity, so that when data is subsequently retrieved from the graph database, all relevant graph data can be retrieved through the connection and dependency between the above-mentioned entities, thereby improving the associative memory ability during retrieval. At the same time, the present application stores the graph data in the graph database. Since the graph data includes nodes, edges and attributes, each graph data in the graph database can be connected to the graph data stored at this time through attributes and edges, thereby realizing the storage continuity of the memory corresponding to the intelligent agent in the graph database, further solving the problem of memory storage fragmentation, and allowing the interaction records to form a coherent memory network. Attached Figure Description

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present application;

[0057] Figure 2 is a flowchart of an implementation of a data processing method provided in another embodiment of the present application;

[0058] Figure 3 is a flowchart of a data processing method according to another embodiment of the present application;

[0059] Figure 4 This is a flowchart of a data processing method according to another embodiment of the present application;

[0060] Figure 5 This is a flowchart of a data processing method according to another embodiment of the present application;

[0061] Figure 6 is a structural diagram of a data processing device provided in one embodiment of the present application;

[0062] Figure 7 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0064] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0065] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0066] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0067] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0068] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0069] In practical applications, an artificial intelligence agent (AI) is an intelligent entity capable of perceiving its environment, making decisions, and executing actions. It uses a large language model as its core computing engine, enabling it to engage in conversation, execute tasks, reason, and exhibit a degree of autonomy. To enable information retrieval within an AI, it is necessary to manage the memory formed by the interaction data generated by the AI ​​and the user.

[0070] However, the existing technology usually directly stores the above-mentioned interaction records in the form of isolated text. Text is usually organized in units such as characters, words, sentences, and paragraphs. In computer storage, these elements are stored in a dispersed manner. For example, a long document will be divided into multiple parts and stored in different locations on the disk. This is because when the file system allocates storage resources, it arranges the storage blocks of the file according to the free space on the disk. For example, the document includes 10KB of content, but at this time there are only 3 consecutive free storage blocks and some scattered free storage blocks in other locations. The file system will first store part of the content in 3 consecutive storage blocks, and then look for other free storage blocks to store the remaining content. It may find 2 free storage blocks in other locations on the disk to store the remaining content. It can be seen that the existing technology has the problem of memory storage fragmentation.

[0071] Based on this, this application proposes a data processing method that can form a systematic memory storage mechanism by storing the interaction records of users and agents in a vector database and a graph database respectively, so that the interaction records can form a coherent memory network, solving the problem of memory storage fragmentation in the existing technology.

[0072] See also Figure 1 , Figure 1 This is a flowchart of a data processing method according to an embodiment of the present application. In this embodiment, the data processing method is performed by a terminal device. The terminal device includes but is not limited to a laptop, desktop computer, or computer.

[0073] like Figure 1 As shown, the data processing method provided in one embodiment of the present application may include S101 to S104, which are described in detail as follows:

[0074] In S101, the interaction record between the agent and the user is obtained.

[0075] In practical applications, in order to achieve efficient management of the interaction records between the agent and the user, relevant personnel can send agent memory processing requests to the terminal device.

[0076] In the embodiment of the present application, the terminal device detecting the aforementioned agent memory processing request may be: detecting a preset operation for the terminal device. The preset operation can be set as needed and is not limited here. For example, the preset operation may be clicking a preset control on the terminal device. Based on this, when the terminal device detects that a preset control on its own has been clicked, it indicates that the preset operation has been detected, i.e., the aforementioned agent memory processing request has been detected.

[0077] After detecting the above intelligent agent memory processing request, the terminal device can obtain the interaction record between the user and the intelligent agent in real time through the intelligent agent wirelessly connected to it.

[0078] In the embodiment of the present application, the above interaction record may be all chat records between the user and the intelligent agent in a historical time period. The historical time period can be determined according to actual needs and is not limited here. Exemplarily, the end moment of the historical time period may be the moment when the terminal device detects the intelligent agent memory processing request.

[0079] In S102, perform vector conversion processing on the interaction record to obtain the vectorized data corresponding to the interaction record.

[0080] In the embodiment of the present application, the terminal device can perform vector conversion processing on the interaction record to obtain the vectorized data corresponding to the interaction record.

[0081] Specifically, the terminal device can perform text preprocessing on the interaction record. The text preprocessing includes but is not limited to text cleaning, text tokenization, and stop word removal, etc.

[0082] Text cleaning specifically refers to removing punctuation marks, numbers, special characters, etc. from the text.

[0083] Text tokenization specifically refers to dividing the text into words or characters. For example, dividing the sentence "I love natural language processing" into "I", "love", and "natural language processing".

[0084] Stop word removal specifically refers to removing words that contribute little to the semantics, such as "de", "shi", "zai", etc.

[0085] After that, the terminal device can perform feature extraction processing on the first text information after text preprocessing, so as to obtain the vectorized data corresponding to the interaction record.

[0086] It should be noted that the above feature extraction processing method may be the Bag of Words model.

[0087] Specifically, the terminal device can count the number of times each word appears in the first text information and generate vectorized data according to the number of times each word appears in the first text information. Exemplarily, assuming there are two texts "I like cats" and "I like dogs", and the vocabulary is [I, like, cat, dog], then the first vectorized data is [1, 1, 1, 0], and the second vectorized data is [1, 1, 0, 1].

[0088] In S103, parse the interaction record to obtain the graph data corresponding to the interaction record.

[0089] In this embodiment, the terminal device can parse the interaction record to obtain the graph data corresponding to the interaction record. The graph data is knowledge graph structure data, including nodes, edges, and attributes. Nodes describe entities, and edges describe the relationship between any two entities.

[0090] Specifically, the terminal device can first perform text preprocessing on the interaction record to obtain the processed first text information. The text preprocessing includes text cleaning, as well as part-of-speech tagging and named entity recognition.

[0091] It should be noted that text cleaning specifically refers to removing irrelevant punctuation, numbers, special characters, etc. For example, removing URLs and extra spaces in the text.

[0092] Part-of-speech tagging and named entity recognition specifically refer to determining the parts of speech of nouns, verbs, etc. in text, as well as entities such as personal names, place names, and organization names, which may become nodes in graph data.

[0093] Afterwards, the terminal device may extract entities and relationships from the first text information.

[0094] Specifically, the terminal device can identify key entities from the first text information according to pre-stored preset entities, and use these key entities as nodes in the graph data. The preset entities can be set according to actual needs, and no restrictions are imposed here.

[0095] Simultaneously, the terminal device can input the aforementioned first text information into a semantic analysis model for text semantic analysis to determine the relationships between the various entities. The semantic analysis model can be trained from a pre-built large language model.

[0096] Among them, large language models include but are not limited to: Llama2 model and Chatgpt model.

[0097] It's important to note that the large language model employs a multi-layered, stacked neural network structure, with each layer consisting of multiple neurons. The input text sequence is processed and transformed through a series of network layers, ultimately yielding the model's output. The large language model can utilize a Transformer structure as its neural network architecture. This Transformer structure incorporates a self-attention mechanism.

[0098] In large language models, the self-attention mechanism is implemented by performing matrix multiplication and softmax activation on the input text sequence. For each position, the large language model calculates its similarity to other positions and performs a weighted average of the sequences based on the similarity weights. In this way, the large language model can learn different relationships in the sequence, such as relationships between words and sentences, across different network layers.

[0099] It should be noted that the training process of the aforementioned large language model includes two stages: pre-training and fine-tuning. In the pre-training stage, the large language model uses a large amount of text corpus (the initial text information in the sample dataset) for unsupervised learning, learning the linguistic patterns and semantic relationships within the text sequences. In the fine-tuning stage, the large language model uses a small amount of labeled data (such as relationships between historical entities) for supervised learning to optimize the model parameters.

[0100] Based on this, in this embodiment, when training a pre-built large language model, the terminal device can input a sample data set into the large language model for pre-training and fine-tuning to obtain a trained semantic analysis model.

[0101] The terminal device can then create nodes based on the extracted entities, assigning each entity to a node. The terminal device can then add edges based on the relationships between the entities identified above, connecting the related nodes with edges and adding labels to the edges to indicate the relationship type, thereby generating graph data.

[0102] In S104, the vectorized data is stored in a vector database, and the graph data is stored in a graph database; each graph data in the graph database is associated through edges.

[0103] In the embodiment of the present application, after obtaining the vectorized data and the graph data, the terminal device may store the vectorized data in a vector database and the graph data in a graph database, wherein the graph data in the graph database are associated with each other through edges.

[0104] From the above, it can be seen that the data processing method provided by the embodiment of the present application obtains the interaction record between the intelligent agent and the user; performs vector conversion processing on the interaction record to obtain vectorized data corresponding to the interaction record; performs analysis on the interaction record to obtain graph data corresponding to the interaction record; the graph data is knowledge graph structure data, and the graph data includes nodes, edges and attributes, nodes are used to describe entities, and edges are used to describe the relationship between any two entities; the vectorized data is stored in the vector database, and the graph data is stored in the graph database; the various graph data in the graph database are associated through edges. On the one hand, the present application vectorizes the interaction record, so that the processed vectorized data is more concise, which facilitates the subsequent retrieval of related data from the vector database, that is, improves the retrieval efficiency. At the same time, the present application stores the vectorized data in the vector database. Since the vectorized data usually represents the data features and relationships in numerical form, the distance between the vectorized data with similar values ​​in the vector database is closer, and the distance between the vectorized data with farther values ​​is farther, thereby achieving the storage consistency of the memory corresponding to the intelligent agent in the vector database and solving the problem of memory storage fragmentation. On the other hand, since graph data includes nodes and edges, by constructing nodes and connecting related nodes with edges, the internal structure of the interaction is clearly displayed; at the same time, each graph data in the graph database is associated by edges, forming a complex relationship network, which can intuitively present the logical connection, hierarchical structure, etc. between the interaction records, that is, it can intuitively display the connection and dependency between each entity, so that when data is subsequently retrieved from the graph database, all relevant graph data can be retrieved through the connection and dependency between the above-mentioned entities, thereby improving the associative memory ability during retrieval. At the same time, the present application stores the graph data in the graph database. Since the graph data includes nodes, edges and attributes, each graph data in the graph database can be connected to the graph data stored at this time through attributes and edges, thereby realizing the storage continuity of the memory corresponding to the intelligent agent in the graph database, further solving the problem of memory storage fragmentation, and allowing the interaction records to form a coherent memory network.

[0105] In practical applications, when an interaction record includes multiple pieces of information, since different pieces of information have different levels of importance to the user, in order to facilitate the rapid retrieval of more important data when searching vector databases and graph databases later, the terminal device can evaluate the importance of multiple pieces of information.

[0106] Based on this, see Figure 2 , Figure 2 This is a flowchart of the data processing method provided by another embodiment of the present application. Figure 1Correspondingly, this embodiment may further include S201 to S204 after S101. Accordingly, step S104 may specifically include step S205, which is described in detail as follows:

[0107] In S201 , the plurality of dialogue information are evaluated based on a set rule to obtain initial weights of the plurality of dialogue information.

[0108] In S202, the user's preference information is obtained.

[0109] In S203, the initial weights of the plurality of dialogue messages are adjusted according to the preference information to obtain target weights of the plurality of dialogue messages.

[0110] In S204, target scores of each of the plurality of dialogue messages are determined based on the target weights; the target score of each message is used to describe the importance of each dialogue message.

[0111] In S205, the target scores of the plurality of conversation messages are associated with the vectorized data corresponding to the plurality of conversation messages and stored in the vector database, and the target scores of the plurality of conversation messages are associated with the graph data corresponding to the plurality of conversation messages and stored in the graph database.

[0112] In this embodiment, the setting rules can be set according to actual needs and are not limited here.

[0113] In some possible embodiments, the setting rule includes setting weights for each preset information. Therefore, in this embodiment, for any conversation information, the terminal device can compare the conversation information with each preset information one by one to obtain the preset information corresponding to the conversation information. The terminal device can then determine the set weight corresponding to the preset information as the initial weight of the conversation information.

[0114] It should be noted that the initial weight of each conversation information is used to describe the initial importance of each conversation information.

[0115] In this embodiment, the user's preference information is used to describe the importance the user places on various information.

[0116] Afterwards, the terminal device can adjust the initial weights of the multiple conversation messages according to the preference information to obtain the target weights of the multiple conversation messages.

[0117] It should be noted that the higher the importance of a certain dialogue information, the greater the corresponding target weight; the lower the importance of a certain dialogue information, the smaller the corresponding target weight.

[0118] In this embodiment, after obtaining the target weights of each dialogue message, the terminal device can determine the target score of each dialogue message based on its target weights. The target score for each dialogue message describes its importance.

[0119] In some possible embodiments, in order to improve the efficiency of determining the score of each conversation information, the terminal device may determine the target weight of each conversation information as the target score of each conversation information.

[0120] In other possible embodiments, to improve the accuracy of determining the score for each dialogue message, the terminal device can sequentially input the target weights of each dialogue message into a weighted scoring model for processing, thereby obtaining the target score for each dialogue message. The weighted scoring model is trained from a pre-built neural network model.

[0121] It should be noted that the weight scoring model can be obtained by training a pre-built neural network model based on a preset sample set. Each sample data in the preset sample set includes a sample weight and a sample score corresponding to the sample weight. When training the pre-built neural network model, the sample weight in each sample data is used as the input of the neural network model, and the sample score corresponding to the sample weight in each sample data is used as the output of the neural network model. Through training, the neural network model can learn the correspondence between all possible sample weights and sample scores, and the trained neural network model is used as the weight scoring model.

[0122] In this embodiment, after obtaining the target scores of each dialogue message, the terminal device can associate and store the target scores of each dialogue message with the vectorized data corresponding to each dialogue message in the vector database, and associate and store the target scores of each dialogue message with the graph data corresponding to each dialogue message in the graph database.

[0123] As can be seen from the above, the data processing method provided in this embodiment, by acquiring user preference information, no longer uses a fixed and uniform weight setting for all information, but dynamically adjusts the initial weight of each piece of information based on the user's unique preferences. This means that when different users face the same set of information, the weights corresponding to the information will also differ due to their different preferences, fully reflecting the personalized characteristics and making subsequent processing more in line with individual user needs. Then, the target score of the information is determined based on the adjusted target weight, establishing a progressive logical connection. The change in weight is directly reflected in the score, so that the score can accurately reflect the change in the importance of each piece of information after considering user preferences. This dynamic connection ensures the accuracy and rationality of measuring the importance of information. The target score with the user's personalized imprint is associated with vectorized data and graph data and stored in the corresponding vector database and graph database, respectively. Personalized elements are integrated into different data storage levels, ensuring that the data storage structure has customized attributes for different users from the beginning. This allows the terminal device to automatically identify important information for users with the help of the target score, saving users' time and energy, especially in an information overload environment, helping users quickly focus on content that is valuable to them.

[0124] In actual applications, during the interaction between the user and the intelligent agent, there may be certain problems with the user input that the intelligent agent cannot respond to. Therefore, in order to further improve the intelligence of the intelligent agent so that the intelligent agent can successfully respond when encountering the same problem again in the future, the terminal device can summarize and analyze the information that the intelligent agent cannot respond to.

[0125] Based on this, see Figure 3 , Figure 3 This is a flowchart illustrating the implementation of a data processing method provided in another embodiment of this application. Compared to... Figure 1 In a corresponding embodiment, this embodiment may include steps S301 to S303 before step S104. Accordingly, step S104 may specifically include step S304, as detailed below:

[0126] In S301, the response results corresponding to the plurality of dialogue messages are determined; the response results are used to describe whether the agent responds to the user's information during the interaction process with the user.

[0127] In S302, target information whose response result is "no response" is selected from multiple dialogue information.

[0128] In S303, the target information is analyzed to obtain the target response information corresponding to the target information.

[0129] In S304 , the vectorized data corresponding to the target response information is stored in the vector database, and the graph data corresponding to the response information is stored in the graph database.

[0130] In this embodiment, when an agent is unable to respond to a message input by a user, it typically responds with a prompt message informing the user that the message cannot be responded to. Therefore, the terminal device can determine the response results corresponding to each of the multiple dialogue messages in the interaction record based on the prompt message. The response result describes whether the agent responded to the user's message during the interaction with the user.

[0131] The response results include, but are not limited to, no response and a response.

[0132] In this embodiment, the terminal device can filter out the context information corresponding to the prompt information from multiple dialogue information based on the response results of each dialogue information. Then, the terminal device can filter out the user input information that is adjacent to the generation time of the prompt information and is generated before the generation time of the prompt information, and determine it as the unresponded target information.

[0133] After obtaining the unresponded target information, the terminal device may input the target information into the second largest language model for analysis, thereby obtaining the target response information corresponding to the target information.

[0134] The training process for the large language model consists of two phases: pre-training and fine-tuning. During the pre-training phase, the large language model uses a large amount of text corpus (individual unanswered sample information in the sample dataset) for unsupervised learning, learning the linguistic patterns and semantic relationships in text sequences. During the fine-tuning phase, the large language model uses a small amount of labeled data (such as sample response information) for supervised learning to optimize the model parameters.

[0135] Based on this, in this embodiment, when the terminal device trains the pre-built large language model, it can input the sample dataset into the large language model for pre-training and fine-tuning to obtain a trained second large language model.

[0136] In one embodiment of this application, since there may be response information corresponding to similar information that is similar to the unresponsive target information in the vector database and graph database, in order to improve the accuracy of generating target response information, the terminal device can specifically generate target response information corresponding to the target information according to the following steps, detailed below:

[0137] Inputting the target information into the first language model for processing to obtain question information corresponding to the target information;

[0138] Search the vector database and the graph database for historical response information corresponding to the question information;

[0139] The historical response information and the question information are input into the second language model for processing to obtain the target response information corresponding to the question information.

[0140] In this embodiment, the terminal device may input the target information into the first language model for processing to obtain question information corresponding to the target information, wherein the question information is used to represent the specific question corresponding to the target information.

[0141] The training process for the large language model consists of two stages: pre-training and fine-tuning. In the pre-training stage, the large language model uses a large amount of text corpus (the unanswered sample information in the sample dataset) for unsupervised learning, learning the linguistic patterns and semantic relationships in the text sequence. In the fine-tuning stage, the large language model uses a small amount of labeled data (such as the sample question information corresponding to the unanswered sample information) for supervised learning to optimize the model parameters.

[0142] Based on this, in this embodiment, when training a pre-built large language model, the terminal device can input a sample data set into the large language model for pre-training and fine-tuning to obtain a trained first large language model.

[0143] It should be noted that each vectorized data point stored in the vector database carries its corresponding question information, and each graph data point stored in the graph database carries its corresponding question information. Therefore, the terminal device can search the vector database and graph database for the question information corresponding to the unanswered target information and the corresponding historical response information.

[0144] Afterwards, the terminal device may input the historical response information and the question information corresponding to the unanswered target information into the second language information for processing, and obtain the target response information corresponding to the question information.

[0145] From the above, it can be seen that the data processing method provided by this embodiment, by determining the response results corresponding to each information, specifically selects those target information that have not received a response during the interaction between the intelligent agent and the user from a large number of information, and accurately positions the focus on the "missing" part of the interaction, highlighting the importance of interaction integrity and user experience, reflecting a strong pertinence, and aims to make up for possible interaction deficiencies. The target information is analyzed to obtain target response information, thereby generating reasonable and user-friendly response content, and striving to enable subsequent responses to effectively make up for the missing links in the previous interaction. The data of the target response information is stored in the corresponding database, which realizes the dynamic update of the database content and continuously enriches the information resources in the database, especially supplements the data of the missing response parts in the previous interaction, so that the database can more comprehensively reflect the actual situation of the interaction between the intelligent agent and the user. As time goes by and more such data is accumulated, the integrity and value of the vector database and the graph database will continue to improve.

[0146] In practical applications, after storing interaction records in vector and graph databases, during subsequent interactions between the user and the agent, the terminal device can retrieve information corresponding to the user's input from these databases, thus utilizing the stored information. The specific process of retrieving information from these databases using user input can be found in [reference needed]. Figure 4 The steps S401 to S404 are shown.

[0147] Based on this, see Figure 4 , Figure 4 This is a flowchart illustrating the implementation of a data processing method provided in another embodiment of this application. Compared to... Figure 1 In a corresponding embodiment, this embodiment may further include S401 to S404 after S104, as detailed below:

[0148] In S401, when it is detected that the user interacts with the agent, content information input by the user is obtained.

[0149] In S402, the first content corresponding to the content information is filtered out from the vector database.

[0150] In this embodiment, the terminal device may perform vector conversion processing on the content information input by the user to obtain first vectorized data corresponding to the content information. The terminal device then compares the first vectorized data with each vectorized data in the vector database one by one to obtain a similarity score between the first vectorized data and each vectorized data in the vector database. The terminal device then determines the target vectorized data with a similarity score greater than or equal to a set score as the first content corresponding to the content information. The set score can be determined based on actual needs and is not limited here.

[0151] In one embodiment of the present application, when each vectorized data in the vector database carries a score and a generation time, in order to further improve the accuracy of determining the first content, the terminal device can specifically obtain the first content through the following steps, which are detailed as follows:

[0152] Performing vector conversion processing on the content information to obtain first vectorized data corresponding to the content information;

[0153] The similarity between each vectorized data in the vector database and the first vectorized data is calculated to obtain each similarity score;

[0154] Obtain the first number of backup vectorized data in descending order of similarity scores;

[0155] evaluating each of the spare vectorized data based on the similarity score, the score, and the generation time of each of the spare vectorized data to obtain a target score for each of the spare vectorized data;

[0156] A second quantity of candidate vectorized data is obtained in descending order of target scores, and the second quantity of candidate vectorized data is determined as the first content; the second quantity is smaller than the first quantity.

[0157] In this embodiment, the terminal device may perform vector conversion processing on the content information input by the user to obtain first vectorized data corresponding to the content information.

[0158] Specifically, the terminal device may perform text preprocessing on the content information, wherein the text preprocessing includes but is not limited to text cleaning, text segmentation, and removal of stop words.

[0159] Text cleaning specifically refers to removing punctuation marks, numbers, special characters, etc. from text.

[0160] Text segmentation specifically refers to dividing text into words or characters, such as dividing the sentence "I love natural language processing" into "I", "love", and "natural language processing".

[0161] Removing stop words specifically refers to removing words that contribute little to semantics, such as "de", "shi", "zai", etc.

[0162] After that, the terminal device can perform feature extraction processing on the second text information after text preprocessing, so as to obtain the vectorized data corresponding to the content information.

[0163] It should be noted that the above feature extraction processing method can be the Bag of Words model.

[0164] Specifically, the terminal device can count the number of times each word appears in the second text information, and generate vectorized data according to the number of times each word appears in the second text information. Exemplarily, assuming there are two texts "I like cats" and "I like dogs", and the vocabulary is [I, like, cat, dog], then the first vectorized data is [1, 1, 1, 0], and the second vectorized data is [1, 1, 0, 1].

[0165] After that, the terminal device can traverse and calculate the similarity between each vectorized data in the vector database and the first vectorized data, and obtain the similarity scores between the first vectorized data and each vectorized data in the vector database.

[0166] In this embodiment, the terminal device can obtain the first quantity of backup vectorized data in descending order of the similarity scores. The first quantity can be determined according to actual needs and is not limited here.

[0167] After that, the terminal device can evaluate each backup vectorized data based on the similarity scores, ratings, and generation times of each backup vectorized data, and obtain the target scores of each backup vectorized data.

[0168] In some possible embodiments, the terminal device can input the similarity scores, ratings, and generation times of each backup vectorized data into a trained score evaluation model for processing, and obtain the target scores of each backup vectorized data. The score evaluation model is trained by a pre-constructed first deep learning model.

[0169] It should be noted that the score evaluation model can be obtained by training a pre-built first deep learning model based on a preset sample set. Wherein, each sample data in the preset sample set includes sample information (including the similarity score, score and generation time of each sample vectorized data) and the sample score corresponding to the sample information. When training the pre-built first deep learning model, the sample information in each sample data is used as the input of the first deep learning model, and the sample score corresponding to the sample information in each sample data is used as the output of the first deep learning model. Through training, the first deep learning model can learn the correspondence between all possible sample information and sample scores, and the trained first deep learning model is used as the score evaluation model.

[0170] In this embodiment, after obtaining the target scores for each piece of candidate vectorized data, the terminal device may obtain a second number of candidate vectorized data in descending order of the target scores, and directly determine the second number of candidate vectorized data as the first content. The second number is smaller than the first number. For example, the second number may be 10 pieces.

[0171] In S403, second content corresponding to the content information is filtered out from the graph database.

[0172] In this embodiment, the terminal device can parse the content information input by the user to obtain the first graph data corresponding to the content information. The terminal device then compares each node in the first graph data with each node in each graph data in the graph database to obtain a similarity score between each node in the first graph data and each node in the graph database. The terminal device then determines the actual content corresponding to the target node with a similarity score greater than or equal to a set score as the second content corresponding to the content information.

[0173] In one embodiment of the present application, since each node is used to describe an entity, and each edge in each graph data can represent the connection and dependency relationship between each entity, in order to obtain a richer and more comprehensive second content, the terminal device can specifically perform the following steps: Figure 5 Steps S501 to S503 shown in FIG. 5 obtain the second content, as detailed below:

[0174] In S501 , the similarity between each of the nodes in the graph database and the content information is traversed and calculated to obtain similar nodes corresponding to the content information.

[0175] In S502, each node connected to the similar node is determined as an associated node.

[0176] In S503, the content corresponding to the similar node and the associated node is determined as the second content.

[0177] In this embodiment, the terminal device may parse the content information to obtain the first image data corresponding to the content information.

[0178] Specifically, the terminal device may first perform text preprocessing on the content information to obtain processed third text information, wherein the text preprocessing includes text cleaning, part-of-speech tagging, and named entity recognition.

[0179] It should be noted that text cleaning specifically refers to removing irrelevant punctuation, numbers, special characters, etc. For example, removing URLs and extra spaces in the text.

[0180] Part-of-speech tagging and named entity recognition specifically refer to determining the parts of speech of nouns, verbs, etc. in text, as well as entities such as personal names, place names, and organization names, which may become nodes in graph data.

[0181] Then, the terminal device can extract entities and relationships from the third-party text information.

[0182] Specifically, the terminal device may identify a key entity from the third text information according to various pre-stored preset entities, and use the key entity as a node of the graph data.

[0183] At the same time, the terminal device can input the third text information into a semantic analysis model to perform text semantic analysis to determine the relationship between the above entities. The semantic analysis model can be trained by a pre-built large language model.

[0184] The terminal device can then create nodes based on the extracted entities, assigning each entity to a corresponding node. The terminal device can then add edges based on the relationships between the determined entities, connecting the related nodes with edges and adding labels to the edges to indicate the relationship type, thereby obtaining first graph data corresponding to the content information.

[0185] In this embodiment, after obtaining the first graph data corresponding to the content information, the terminal device may traverse and calculate the similarity between each node in the graph database and each node in the first graph data. The terminal device may then determine, among the various similarities, the target node whose similarity exceeds a set similarity as a similar node corresponding to the content information. The set similarity can be determined based on actual needs and is not limited here.

[0186] It should be noted that the above-mentioned similar nodes may include multiple nodes.

[0187] In this embodiment, since the nodes in the graph database are connected by edges, and each edge can represent the connection and dependency between entities, the terminal device can identify the nodes that are directly connected to each similar node as associated nodes.

[0188] Afterwards, the terminal device can directly determine the content corresponding to the similar nodes and related nodes as the second content.

[0189] As can be seen from the above, the data processing method provided in this embodiment fully leverages the characteristic of graph databases that construct data relationships using nodes and edges. By traversing and calculating the similarity between each node and the content information, the analysis of text content is integrated into the graph structure. This approach does not view data in isolation, but instead considers the interrelationships between the data within the graph structure and the degree of association with the input content information, laying the foundation for mining deeper information. After determining similar nodes, the nodes connected to them are further identified as associated nodes. By expanding this graph-based connection relationship, a network of related information surrounding similar nodes can be mined. This means that the focus is not only on the parts directly similar to the content information, but also extends along the edges of the graph to other elements that are logically related to it, broadening the scope of retrieval and analysis and helping to discover more potential related content hidden in the graph structure. Finally, the content corresponding to the similar nodes and associated nodes is determined as secondary content, clarifying that the content selected from the graph database is centered around parts that have similarities and associations with the content information. This targeted content extraction method avoids indiscriminately selecting a large amount of irrelevant information from the graph database, ensuring that the extracted secondary content closely revolves around the content information entered by the user, providing more valuable data for subsequent operations such as combined use.

[0190] In S404, the first content and the second content are combined to obtain the target retrieval content corresponding to the content information.

[0191] In this embodiment, after obtaining the first content and the second content, the terminal device can combine the first content and the second content to obtain the target search content corresponding to the content information.

[0192] In some possible embodiments, the terminal device can splice the first content and the second content to achieve a combination of the first content and the second content.

[0193] For example, assuming the first content is X1 and the second content is X2, the target search content can be: [X1, X2].

[0194] In one embodiment of this application, in order to obtain richer and more comprehensive target retrieval content, the terminal device may specifically implement step S404 according to the following steps, as detailed below:

[0195] Filtering out third content corresponding to the second content from the vector database;

[0196] The first content, the second content, and the third content are combined to obtain the target search content.

[0197] In this embodiment, combined with Figure 7 After obtaining the second content, the terminal device can compare the second content with each vectorized data in the vector database one by one to obtain a similarity score between the first content and each vectorized data in the vector database. The terminal device can then determine the target vectorized data with a similarity score greater than or equal to a set score as the third content corresponding to the second content.

[0198] Afterwards, the terminal device may combine the first content, the second content, and the third content to obtain target search content corresponding to the content information.

[0199] In some possible embodiments, the terminal device may splice the first content, the second content, and the third content to achieve a combination of the first content, the second content, and the third content.

[0200] For example, assuming that the first content is X1, the second content is X2, and the third content is X3, the target search content may be: [X1, X2, X3].

[0201] As can be seen from the above, the data processing method provided in this embodiment fully utilizes the resources of two different types of databases, vector databases and graph databases, and is not limited to data search in a single database. By filtering the first content from the vector database and the second content from the graph database, respectively, a cross-database collaborative working model is implemented, capable of integrating data stored in different modalities to meet search requirements. Based on the content information entered by the user, targeted screening is performed in both databases. This content-relevance-based screening method accurately finds content closely related to the user's current input. Whether it is the first content corresponding to semantically similar vectorized data in the vector database or the second content with structurally consistent associations in the graph database, both are matched based on the user input, ensuring the relevance and accuracy of the search content. Finally, the first and second content filtered from different databases are combined, achieving multi-source data fusion. This integrates the semantic features contained in the vectorized data and the structural relationships reflected in the graph data to form a more comprehensive and complete target search content. This avoids the one-sidedness that may exist in single-type data and provides richer material for subsequent analysis or use of search results.

[0202] In some possible embodiments, the above method can be implemented in the agent of the terminal device, where the agent refers to an artificial intelligence program that performs user tasks.

[0203] Specifically, the agent has the ability to acquire and manage interaction records. For example, by setting up corresponding interaction interfaces (such as chat windows, voice interaction channels, etc.), the intelligent agent can listen to the information input by the user in real time and record the complete process of each interaction, including the timestamp of the interaction, the specific content of the user's input, the content of its own reply, and other detailed information, forming an interaction record text. These records can be temporarily stored in a local cache or directly transferred to a dedicated storage area for subsequent processing.

[0204] Agents possess data transformation and storage capabilities. For example, agents can perform vector transformation on interaction records to obtain vectorized data of the interaction records. Simultaneously, agents can perform semantic analysis on the interaction records, extracting key elements such as different entities (people, things, etc.) and topics as nodes in graph data, and analyzing the logical relationships between these elements, such as causal relationships and ownership relationships. Related nodes are then connected through edges to obtain graph data. Afterward, agents can establish connections with vector databases, using the database's provided interfaces (such as database driver libraries in Python) to insert the generated vectorized data into the corresponding tables or data structures in the vector database according to a specific format (such as vector arrays, with possible accompanying metadata such as interaction record numbers), ensuring accurate data storage and subsequent retrieval. Furthermore, agents can also establish connections with graph databases, storing the constructed graph data in a format supported by the graph database (such as node and edge definitions, potentially involving node attributes, edge weights, etc.). Graph data entry and storage are completed through database operation statements or related APIs, ensuring the integrity and relevance of the graph data within the database.

[0205] In summary, combined with the powerful performance of the above-mentioned Agent, executing the method of the present application in the Agent can enable the method of the present application to be completed more quickly and accurately.

[0206] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0207] Corresponding to a data processing method described in the above embodiment, Figure 6 A schematic diagram of a data processing apparatus according to an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown. (Refer to...) Figure 6The data processing device 600 includes: a first acquisition unit 61, a first processing unit 62, a parsing unit 63, and a first storage unit 64. Wherein:

[0208] The first acquiring unit 61 is used to acquire the interaction record between the agent and the user.

[0209] The first processing unit 62 is used to perform vector conversion processing on the interaction record to obtain vectorized data corresponding to the interaction record.

[0210] The parsing unit 63 is used to parse the interaction record to obtain the graph data corresponding to the interaction record.

[0211] The first storage unit 64 is used to store the vectorized data in a vector database, and store the graph data in a graph database; each graph data in the graph database is associated through edges.

[0212] In one embodiment of the present application, the interaction record includes multiple pieces of information; the data processing device 600 further includes: a first evaluation unit, a second acquisition unit, an adjustment unit, and a first determination unit; accordingly, the first storage unit 64 specifically includes: a second storage unit.

[0213] The first evaluation unit is configured to evaluate the plurality of dialogue information based on a set rule to obtain an initial weight of each of the plurality of dialogue information.

[0214] The second acquiring unit is configured to acquire the preference information of the user.

[0215] The adjusting unit is configured to adjust the initial weights of the plurality of dialogue information according to the preference information to obtain target weights of the plurality of dialogue information.

[0216] The first determining unit is used to determine a target score for each of the plurality of dialogue messages based on the target weight; the target score for each message is used to describe the importance of each dialogue message.

[0217] The second storage unit is used to associate the target scores of each of the multiple conversation messages with the vectorized data corresponding to each of the multiple conversation messages and store them in the vector database, and associate the target scores of each of the multiple conversation messages with the graph data corresponding to each of the multiple conversation messages and store them in the graph database.

[0218] In one embodiment of the present application, the interaction record includes multiple conversation information between the user and the agent; the data processing device 600 further includes: a second determination unit, a first screening unit, and an analysis unit; and correspondingly, the first storage unit 64 specifically includes: a third storage unit.

[0219] The second determining unit is used to determine the response results corresponding to each of the plurality of dialogue messages; the response results are used to describe whether the agent responds to the user's information during the interaction process with the user.

[0220] The first filtering unit is used to filter out target information from multiple dialogue messages whose response result is no response.

[0221] The analysis unit is used to analyze the target information to obtain the target response information corresponding to the target information.

[0222] The third storage unit is used to store the vectorized data corresponding to the target response information into the vector database, and to store the graph data corresponding to the response information into the graph database.

[0223] In one embodiment of this application, the analysis unit specifically includes: a second processing unit, a search unit, and a third processing unit. Wherein:

[0224] The second processing unit is used to input the target information into the first large language model for processing to obtain the question information corresponding to the target information.

[0225] The search unit is used to search for historical response information corresponding to the question information from the vector database and the graph database.

[0226] The third processing unit is used to input the historical response information and the question information into the second language model for processing, so as to obtain the target response information corresponding to the question information.

[0227] In one embodiment of this application, the data processing apparatus 600 further includes: a third acquisition unit, a second filtering unit, a third filtering unit, and a first combination unit. Wherein:

[0228] The third acquisition unit is used to acquire the content information input by the user when it is detected that the user is interacting with the intelligent agent.

[0229] The second filtering unit is used to filter out the first content corresponding to the content information from the vector database.

[0230] The third filtering unit is used to filter out the second content corresponding to the content information from the graph database.

[0231] The first combination unit is used to combine the first content and the second content to obtain the target retrieval content corresponding to the content information.

[0232] In one embodiment of this application, each vectorized data in the vector database carries a score and a generation time, and the target score of each vectorized data is used to describe the importance of each vectorized data; the first content includes multiple items; the second filtering unit specifically includes: a fourth processing unit, a first calculation unit, a fourth acquisition unit, a second evaluation unit, and a third determination unit. Wherein:

[0233] The fourth processing unit is configured to perform vector conversion processing on the content information to obtain first vectorized data corresponding to the content information.

[0234] The first calculation unit is used to traverse and calculate the similarity between each vectorized data in the vector database and the first vectorized data to obtain each similarity score.

[0235] The fourth acquiring unit is configured to acquire a first quantity of spare vectorized data in descending order of similarity scores.

[0236] The second evaluation unit is used to evaluate each of the backup vectorized data based on the similarity score, the score, and the generation time, and to obtain the target score of each of the backup vectorized data.

[0237] The third determining unit is used to obtain a second number of candidate vectorized data in descending order of target scores, and to determine the second number of candidate vectorized data as the first content; the second number is less than the first number.

[0238] In one embodiment of the present application, the graph database includes a plurality of graph data; the third screening unit specifically includes: a second calculation unit, a fourth determination unit, and a fifth determination unit.

[0239] The second calculation unit is used to traverse and calculate the similarity between each of the nodes in the graph database and the content information, and obtain similar nodes corresponding to the content information.

[0240] The fourth determining unit is configured to determine each node connected to the similar node as an associated node.

[0241] The fifth determining unit is configured to determine the content corresponding to each of the similar node and the associated node as the second content.

[0242] In one embodiment of the present application, the first combination unit specifically includes: a fourth screening unit and a second combination unit.

[0243] The fourth screening unit is configured to screen out third content corresponding to the second content from the vector database.

[0244] The second combining unit is used to combine the first content, the second content and the third content to obtain the target search content.

[0245] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0246] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0247] Figure 7 This is a schematic diagram of the structure of a terminal device provided in one embodiment of the present application. Figure 7 As shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Figure 7 (Only one is shown) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above-described data processing method embodiments.

[0248] The terminal device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that Figure 7 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0249] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0250] In some embodiments, the memory 71 may be an internal storage unit of the terminal device 7, such as the memory of the terminal device 7. In other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Furthermore, the memory 71 may include both an internal storage unit of the terminal device 7 and an external storage device. The memory 71 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 71 may also be used to temporarily store data that has been output or is about to be output.

[0251] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0252] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0253] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable medium cannot be an electric carrier signal or a telecommunication signal.

[0254] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0255] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Obtain the interaction records between the agent and the user; Performing vector conversion processing on the interaction record to obtain vectorized data corresponding to the interaction record; Parsing the interaction record to obtain graph data corresponding to the interaction record; The vectorized data is stored in a vector database, and the graph data is stored in a graph database; each graph data in the graph database is associated through an edge.

2. The data processing method according to claim 1, characterized in that: The interaction record includes the conversation information between the plurality of users and the agent; and further includes: Evaluate the plurality of dialogue information based on a set rule to obtain initial weights of the plurality of dialogue information; Obtaining preference information of the user; Adjusting the initial weights of the plurality of dialogue information according to the preference information to obtain target weights of the plurality of dialogue information; Determine a target score for each of the plurality of dialogue messages based on the target weight; the target score for each message is used to describe the importance of each dialogue message; Accordingly, storing the vectorized data in a vector database, and storing the graph data in a graph database, includes: The target scores of each of the plurality of dialogue information are associated with the vectorized data corresponding to each of the plurality of dialogue information and stored in the vector database; the target scores of each of the plurality of dialogue information are associated with the graph data corresponding to each of the plurality of dialogue information and stored in the graph database.

3. The data processing method according to claim 1, characterized in that: The interaction record includes the conversation information between the plurality of users and the agent; and further includes: Determine a response result corresponding to each of the plurality of dialogue messages; the response result is used to describe whether the agent responds to the user's message during the interaction with the user; Filtering target information whose response result is unresponded from the plurality of conversation information; Analyze the target information to obtain target response information corresponding to the target information; Accordingly, storing the vectorized data in a vector database, and storing the graph data in a graph database, includes: The vectorized data corresponding to the target response information is stored in the vector database, and the graph data corresponding to the response information is stored in the graph database.

4. The data processing method according to claim 3, characterized in that: The analyzing the target information to obtain target response information corresponding to the target information includes: Inputting the target information into the first language model for processing to obtain question information corresponding to the target information; Searching for historical response information corresponding to the question information from the vector database and the graph database; The historical response information and the question information are input into the second largest language model for processing to obtain target response information corresponding to the question information.

5. The data processing method according to any one of claims 1 to 4, characterized in that: Also includes: When detecting that the user interacts with the agent, obtaining content information input by the user; Filtering out first content corresponding to the content information from the vector database; Filtering out second content corresponding to the content information from the graph database; The first content and the second content are combined to obtain target search content corresponding to the content information.

6. The data processing method according to claim 5, characterized in that: Each vectorized data in the vector database carries a score and a generation time, and the target score of each vectorized data is used to describe the importance of each vectorized data; the first content includes multiple items; and filtering out the first content corresponding to the content information from the vector database includes: Performing vector conversion processing on the content information to obtain first vectorized data corresponding to the content information; Traversing and calculating the similarity between each vectorized data in the vector database and the first vectorized data to obtain each similarity score; Acquire a first amount of spare vectorized data in descending order of similarity scores; Based on the similarity score of each of the spare vectorized data, the score and the generation time, each of the spare vectorized data is evaluated to obtain a target score of each of the spare vectorized data; A second quantity of candidate vectorized data is acquired in descending order of target scores, and the second quantity of candidate vectorized data is determined as the first content; the second quantity is smaller than the first quantity.

7. The data processing method according to claim 5, characterized in that: The graph database includes a plurality of graph data; each of the graph data includes an edge and a node; the edge is used to describe the relationship between any two entities; The step of selecting the second content corresponding to the content information from the graph database includes: Traversing and calculating the similarity between each of the nodes in the graph database and the content information to obtain similar nodes corresponding to the content information; Determine each node connected to the similar node as an associated node; The content corresponding to each of the similar node and the associated node is determined as the second content.

8. The data processing method according to claim 7, characterized in that: The combining the first content and the second content to obtain target search content corresponding to the content information includes: Filtering out third content corresponding to the second content from the vector database; The first content, the second content and the third content are combined to obtain the target search content.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data processing method according to any one of claims 1 to 8 is implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the data processing method according to any one of claims 1 to 8 when being executed.

Images