Information generation method and device

Abstract

The invention discloses an information generation method and device, electronic equipment and a readable storage medium, and belongs to the technical field of natural languages. The method comprises the steps that first question information is input into a first translation layer of a generative question and answer model, the first translation layer comprises at least two first memory parameters, and the at least two first memory parameters are obtained by conducting feature extraction on first user historical behavior data; based on the at least two first memory parameters, processing the first question information through a first translation layer to obtain first response information output by the generative question and answer model, the first response information being response information corresponding to the first question information; wherein the first user historical behavior data comprises at least one of the following items: historical browsing records of the user, historical preference data of the user and historical problem information of the user.

Description

Technical Field

[0001] The present application belongs to the field of natural language technology, and specifically relates to an information generation method, device, electronic device and readable storage medium. Background Art

[0002] With the continuous advancement of the field of artificial intelligence, large language models in electronic devices have made great progress. Through their ability to understand and generate natural language, they can understand user intentions and summarize and integrate original user questions to generate more appropriate answers.

[0003] At present, in the question-and-answer scenario based on the large language model, the technical solutions for making personalized answers by combining the characteristics of different users include: the electronic device needs to build a user history behavior database in advance, and then generate answer information based on the question information input by the user after the large language model is used, and then search the user history behavior database for text information that is highly relevant to the answer information, and after combining the user's answer information with the text information in the user history behavior database, provide it to the user for viewing.

[0004] However, on the one hand, since the user history behavior database is usually a large number of text feature vectors, it takes up a large amount of storage space of the electronic device to store the user behavior data, which leads to insufficient storage space of the electronic device. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an information generation method, device, electronic device and readable storage medium, which can reduce the storage space of the electronic device for storing the user's historical behavior data while ensuring that accurate response information can be generated in combination with the user's historical behavior data.

[0006] In a first aspect, an embodiment of the present application provides an information generation method, which includes: inputting first question information into a first translation layer of a generative question-answering model, the first translation layer including at least two first memory parameters, and the at least two first memory parameters are obtained by performing feature extraction on first user historical behavior data; based on the at least two first memory parameters, processing the first question information through the first translation layer to obtain first response information output by the generative question-answering model, the first response information being response information corresponding to the first question information; wherein the first user historical behavior data includes at least one of the following: the user's historical browsing records, the user's historical preference data, and the user's historical question information.

[0007] In a second aspect, an embodiment of the present application provides an information generating device, which includes: a processing module; the processing module is used to input first question information into a first translation layer of a generative question-answering model, the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by performing feature extraction on the first user's historical behavior data; the processing module is also used to process the first question information through the first translation layer based on the at least two first memory parameters to obtain first response information output by the generative question-answering model, the first response information being response information corresponding to the first question information; wherein the first user's historical behavior data includes at least one of the following: the user's historical browsing records, the user's historical preference data, and the user's historical question information.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, the first question information is input into the first translation layer of the generative question-answering model, the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by feature extraction of the first user's historical behavior data; based on the at least two first memory parameters, the first question information is processed by the first translation layer to obtain the first response information output by the generative question-answering model, and the first response information is the response information corresponding to the first question information; wherein the first user's historical behavior data includes at least one of the following: the user's historical browsing record, the user's historical preference data, and the user's historical question information. In this scheme, the memory parameters are obtained by feature extraction of the user's historical behavior data, and the memory parameters are combined with the first translation layer in the generative question-answering model to obtain a generative question-answering model that can generate response information based on the user's historical behavior data. Since the memory parameters in this embodiment are obtained by feature extraction, it is not necessary to store the complete user's historical behavior data, so that while ensuring that accurate response information can be generated in combination with the user's historical behavior data, the storage space of the electronic device storing the user's historical behavior data can be reduced, thereby reducing the storage space occupied by the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of an information generation method provided in an embodiment of the present application;

[0014] Figure 2 It is a structural diagram of a generative question-answering model provided in an embodiment of the present application;

[0015] Figure 3 It is a structural diagram of a generative question-answering model provided in an embodiment of the present application;

[0016] Figure 4 It is a structural diagram of a generative question-answering model provided in an embodiment of the present application;

[0017] Figure 5 It is a structural diagram of a generative question-answering model provided in an embodiment of the present application;

[0018] Figure 6 It is a structural schematic diagram of an information generating device provided in an embodiment of the present application;

[0019] Figure 7 It is a structural schematic diagram of an information generating device provided in an embodiment of the present application;

[0020] Figure 8 It is a structural schematic diagram of an information generating device provided in an embodiment of the present application;

[0021] Fig. 9 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0022] Fig.10 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0024] Some terms and nouns involved in the embodiments of the present application are explained below.

[0025] Smart assistant: An assistant program with AI functions in electronic devices, which allows users to complete certain operations by operating or manually entering shortcut commands.

[0026] Specifically, the above-mentioned assistant program with AI function is an application in an electronic device, and the application is associated with an AI model, which can be deployed in the electronic device or in the cloud. If the user wants to ask a question, he can trigger the interface of the intelligent assistant and enter the question information in the interface, so that the intelligent assistant can input the question information entered by the user into the corresponding language model, generate the corresponding answer information, and output the information content that the user wants.

[0027] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] The terms "at least one (item)", "at least one of" and the like in the specification and claims of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0029] The logo in this application refers to text, symbols, images, etc. used to indicate information. Logos or other containers can be used as carriers to display information, including but not limited to text logos, image logos, symbol logos, etc.

[0030] It should be noted that the information generation method provided in the embodiments of the present application can be executed by electronic devices such as mobile phones, tablet computers, laptop computers, PDAs, and vehicle-mounted electronic devices. In some embodiments of the present application, the information generation method provided in the embodiments of the present application is described by taking the electronic device as the execution subject to execute the information generation method.

[0031] The information generation method, device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0032] At present, the intelligent assistants in electronic devices are mainly built based on large language models, relying on the powerful logical reasoning and generation capabilities of large language models. For example, in the scenario where a user asks a question to the intelligent assistant in an electronic device, the user inputs the question information "What is the weather today?" into the intelligent assistant in the electronic device. At this time, the intelligent assistant inputs the question information into the large language model for processing and generates the corresponding response information "Today's weather is sunny, and the temperature is 20℃ to 25℃".

[0033] Specifically, after the intelligent assistant inputs the question information into the large language model, the electronic device can generally process it in the following two ways to generate response information corresponding to the question information input by the user.

[0034] Method 1: Based on Retrieval-Augmented Generation (RAG) method.

[0035] The electronic device constructs the user's historical behavior data, that is, the user behavior memory, into the form of a knowledge database. When receiving the question information input by the user, it calculates the relevance between the question information and the content in the knowledge database, recalls the text information that is highly relevant to the question information, and then concatenates the question information input by the user and the text information recalled from the knowledge database, and then inputs them into the large language model for personalized generation. This method is mainly suitable for scenarios where the context length supported by the large language model is limited, or where the user has high requirements for response time.

[0036] Specifically, the steps include:

[0037] Step 1: Construct the user's historical behavior data, that is, the user behavior memory, into a document form, represented by D.

[0038] Step 2: Input the document D of historical behavior data into the vector database. The vector database will divide the document data D into n small paragraphs d1, d2...dn, and each small paragraph corresponds to a vector e1, e2...en.

[0039] Step 3: Vectorize the question information Q input by the user to obtain er, then input er into the vector database, calculate the similarity with the vectors e1, e2, ...en corresponding to each small paragraph of D, and return the small paragraph dx with the highest similarity.

[0040] Step 4: Concatenate the question information Q input by the user and the small paragraph dx, and then input them into the large language model to output the answer information corresponding to the question information input by the user.

[0041] Based on the above steps, we can see that only part of the memory dx can be obtained through RAG search enhancement, so the effect of large language model generation is not optimal, especially in scenarios where a summary needs to be made based on the full text.

[0042] Method 2: Implementation based on long text reasoning.

[0043] The electronic device directly constructs the user's historical behavior data, that is, the user behavior memory, into the form of a document. When receiving question information input by the user, the electronic device needs to splice the question information input by the user and the user's historical behavior data, and then input them into the large language model for personalized generation. This method is suitable for scenarios where the large language model supports long text contexts and the user is not sensitive to response time.

[0044] Specifically, the steps include:

[0045] Step 1: Construct the user's historical behavior data, that is, the user behavior memory, into a document form, represented by D.

[0046] Step 2: Input document D into the large language model, and the large language model generates the KVcache intermediate representation corresponding to the large language model from document D.

[0047] Step 3: Input the user request into the large language model, and the large language model finds the corresponding KVcache based on the user ID.

[0048] Step 4: The large language model generates response information based on the question information entered by the user and the corresponding KVcache, and then outputs and displays the response information.

[0049] Based on the above steps, we can see that all historical behavior data can be represented as KVcache through long text, and the large language model can give accurate and personalized answers to user questions based on the global historical behavior data. However, KVcache requires a lot of storage space for storage. 128K text data requires about 100G storage space to store KVcache. In addition, when the KVcache volume is large, the generation performance of the large language model is greatly reduced. Therefore, this method not only has huge operating costs, but also poor user experience.

[0050] In the information generation method provided in the embodiment of the present application, the electronic device adjusts the generative question-answering model M, that is, the transformer network structure in the above-mentioned large language model, and adds two memory parameters to each multilayer perceptron (MLP, Multilayer Perceptron, MLP) module in each translation (transformer) layer, and the memory parameters are used to indicate the user's historical behavior data.

[0051] Each transformer layer is followed by two linear layers, such as Wn0 and Wn1, where n represents the nth transformer layer, and the shapes of Wn0 and Wn1 are [h, 4h] and [4h, h], respectively. Memory parameters An0 and Bn0 are added to Wn0, and their dimensions are [h, d] and [d, 4h], respectively. Memory parameters An1 and Bn1 are added to Wn1, and their dimensions are [4h, d] and [d, h], respectively. It should be noted that d is much smaller than h. An0, Bn0, An1, and Bn1 of all transformer layers together constitute the representation of the user's historical behavior data. During reasoning, An0, Bn0, An1, and Bn1, i.e., memory parameters, are added to the corresponding MLP modules so that the generative question-answering model can generate personalized response information based on the user's historical behavior data. That is, the response information output by the generative question-answering model is: Hn0=(Wn0+An0.Bn0)Xn0, Hn1=(Wn1+An1.Bn1)Xn1.

[0052] Among them, An0.Bn0 and An1.Bn1 are the memory parameters corresponding to the nth transformer layer.

[0053] In this embodiment, the electronic device adds memory parameters to each transformer layer in the generative question-answering model M to represent the long-term memory of the user's historical behavior. Even if the parameter scale of the generative question-answering model M is more than 100B, through the feature extraction and processing of the memory parameters, the size of all newly added memory parameters can be controlled within 30M, and will not increase with the increase of historical data. In this way, this embodiment not only ensures that the intelligent assistant can generate personalized response information effects in the long-term memory scenario, but also greatly reduces the operating costs, while improving the generation performance of the model and greatly improving the user experience.

[0054] The execution subject of the information generation method provided in the embodiment of the present application may be an information generation device. Exemplarily, the information generation device may be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The information generation method provided in the embodiment of the present application will be exemplarily described below using an electronic device as an example.

[0055] The present application embodiment provides a method for generating information. Figure 1 FIG. 1 is a flow chart of an information generation method provided by an embodiment of the present application, which can be applied to electronic devices. Figure 1 As shown, the information generation method provided in the embodiment of the present application may include the following steps 201 to 202.

[0056] Step 201: The electronic device obtains first question information.

[0057] In some embodiments of the present application, the first question information is question information input by a user.

[0058] In some embodiments of the present application, the first question information may be text information or voice information.

[0059] Exemplarily, the electronic device may obtain the first question information by receiving text information manually input by the user, or may obtain the first question information by receiving voice information input by the user.

[0060] For example, the first question information may be “How is the weather today?”, “What should I eat today?”, or “What movie is showing today?”, etc.

[0061] In some embodiments of the present application, a user can wake up an intelligent assistant in an electronic device and, by inputting question information into the intelligent assistant, instruct the intelligent assistant in the electronic device to generate personalized response information based on the question information.

[0062] Step 202: The electronic device inputs the first question information into the first translation layer of the generative question answering model.

[0063] In some embodiments of the present application, the above-mentioned generative question-answering model is used to generate corresponding answer information based on question information.

[0064] For example, if the question information input into the generative question answering model is "What's the weather like today", the answer information output by the generative question answering model is "The weather is sunny today, and the temperature is 20℃ to 25℃"; if the question information input into the generative question answering model is "What to eat today", the answer information output by the generative question answering model is "Based on your taste, I recommend spicy hot pot or malatang to you", etc.

[0065] In some embodiments of the present application, the above-mentioned generative question answering model includes at least one translation layer and a fully connected layer.

[0066] Exemplarily, the first translation layer is used to extract text feature information of the question information.

[0067] Exemplarily, the fully connected layer is used to determine the first response information.

[0068] Exemplarily, the first translation layer includes at least two first memory parameters.

[0069] Exemplarily, the at least two first memory parameters are obtained by extracting features from historical behavior data of the first user.

[0070] Exemplarily, the first memory parameter is a matrix parameter of a linear layer in an MLP module in the first translation layer.

[0071] Exemplarily, the above-mentioned first user historical behavior data includes at least one of the following: user's browsing history, user's preference data, and user's historical question information.

[0072] It should be noted that different users correspond to different user historical behavior data.

[0073] In a possible embodiment, Figure 2 As shown, the above-mentioned generative question-answering model M' is composed of a base large model M, such as a GPT model, a natural language model, etc., and memory parameters generated based on user historical behavior data.

[0074] The base model M consists of a total of n transformer layers. At the end of each transformer layer are two MLP linear modules Wn0 and Wn1, where n represents the nth transformer layer, and the shapes of Wn0 and Wn1 are [h, 4h] and [4h, h] respectively.

[0075] The generative question-answering model M' adds memory parameters An0 and Bn0 to Wn0 based on the base model M, with dimensions [h, d] and [d, 4h] respectively, and adds memory parameters An1 and Bn1 to Wn1, with dimensions [4h, d] and [d, h] respectively. It should be noted that d is much smaller than h. In this way, An0, Bn0, An1, and Bn1 in all transformer layers together constitute the representation of user historical behavior data.

[0076] Since d is much smaller than h, usually less than 10, and h is even greater than 10,000, the volume of the memory parameter L composed of An0, Bn0, An1, and Bn1 of all transformer layers is much smaller than the volume of the base large model M.

[0077] For example, taking the 70B base model M as an example, n=80, h=8192, d=10, the volume of the large model M is about 140B, and its corresponding memory parameter L is only about 62M. In addition, the user historical behavior data in the traditional base model M is represented by KVcache value. If the text data size of the user historical behavior data is about 100K, the KVcache data volume generated by the base model M is about 250G, which is also much larger than 62M in this embodiment.

[0078] The generative question-answering model M' in this embodiment only needs 62M memory parameters to fully represent the user's historical behavior data, and it will not increase with the increase of historical data. That is, regardless of 100K historical behavior data or 200K historical behavior data, the corresponding memory parameter L size is only 62M.

[0079] Specifically, as follows Figure 2 As shown in the figure, the transformer module is the core structure of the base large model M, layer1...layern represents a specific transformer layer, and layer1 is used as an example for expansion in the figure. The left figure KVcache1...KVcachex has a shape of [s,h], s represents the length of the historical behavior data, which is the mapping of the user's historical behavior data in the above method 2. During the operation of the system, it is necessary to first store KVcache1...KVcachex of all layers in the memory or hard disk. Taking the 70B large model as an example, h=8192, x=8, n=80, assuming s=100k, then the total size of KVcache of all layers is 250G, which not only requires 250G storage space to store such a large KVcache memory parameter, but also needs to transmit and load these 250G storage parameters to the inference chip in real time during the actual reasoning process. The network transmission cost is high and the delay is large, which greatly affects the user experience.

[0080] The right picture is the generative question-answering module M' in this embodiment. Each transformer layer only needs to use An0, Bn0, An1, and Bn1 to represent the memory parameters. Taking the 70B large model as an example, h=8192, assuming d=10, the total size of An0, Bn0, An1, and Bn1 of all layers is 62M. In the actual operation process, the storage space required for storage, the bandwidth required for network transmission, and the cost are all smaller than the KVcache solution, and the delay is also smaller, which greatly improves the user experience.

[0081] Optionally, in some embodiments of the present application, the generative question-answering model needs to be deployed in advance before the electronic device uses the generative question-answering model. Since the generative question-answering model is composed of a large base model and memory parameters, when deploying the model, the large base model M and the memory parameter L need to be deployed separately, and the backend service can find the corresponding memory parameter Li according to the user's ID, and then combine the large base model M for personalized reasoning.

[0082] Step 203: The electronic device processes the first question information through the first translation layer based on at least two first memory parameters to obtain first response information output by the generative question-answering model, where the first response information is response information corresponding to the first question information.

[0083] In some embodiments of the present application, the electronic device extracts text feature information of the first question information through the first translation layer, and concatenates and fuses at least two first memory parameters with the text feature information to obtain new text feature information, converts the text feature information into text, i.e., the above-mentioned first response information, and outputs a generative question-answering model.

[0084] Exemplarily, the electronic device generates a personalized answer, namely the first response information mentioned above, based on the memory parameter Li corresponding to the user's historical behavior data in the generative question-answering model, and finally returns the response information to the user for review.

[0085] Exemplarily, the text feature information corresponding to the first response information can be expressed as Hn0=(Wn0+An0.Bn0)Xn0, Hn1=(Wn1+An1.Bn1)Xn1.

[0086] Among them, An0.Bn0 and An1.Bn1 represent the memory parameters of the nth transformer layer, Xn0 and Xn1 are the text feature information of the question information input by the user, Wn0.Xn0 is the result of the transformer layer processing the question information input by the user, and An0.Bn0.Xn0 is the result of the memory parameters processing the question information input by the user. The overall understanding can be understood as obtaining memory information from the memory parameters according to the user's input, and then integrating them together to allow the base model M to generate logical reasoning, so that the generated answers are more inclined to the user's personalized needs.

[0087] For example, Figure 3 As shown, assuming that the user ID is 1, when the generative question-answering model processes the question information of user 1, it will calculate the relevant parameters in the memory parameter L1 in the first layer, that is, (W00+A00.B00)X00, and (W11+A11.Bn1)X11. The calculation logic of other layers is the same.

[0088] Optionally, the above step 203 can be specifically implemented through the following steps 203a to 203c.

[0089] Step 203a: The electronic device extracts first text feature information of the first question information through the first translation layer.

[0090] In some embodiments of the present application, the electronic device extracts features of the first question information through a first translation layer, and vectorizes the extracted features to obtain first text feature information.

[0091] Step 203b: The electronic device concatenates the first text feature information with the first memory parameter through the first translation layer to obtain second text feature information.

[0092] In some embodiments of the present application, the electronic device concatenates the first text feature information extracted by the first translation layer with the first memory parameter included in the first translation layer to obtain the second text feature information.

[0093] Step 203c: The electronic device determines first response information based on the second text feature information through the first translation layer.

[0094] In some embodiments of the present application, the electronic device calculates the degree of match between the response information corresponding to the second text feature information and the first question information through a fully connected layer. If the degree of match is high, the second text feature information is considered to be the first response information corresponding to the first question information; if the degree of match is low, the response information corresponding to the second text feature information is considered to not match the first question information. At this time, the electronic device will detect the degree of match between the response information corresponding to the text feature information output by other translation layers and the first question information until the first response information with a high degree of match is found.

[0095] Optionally, the above step 203c can be specifically implemented through the following steps 203c1 to 203c3.

[0096] Step 203c1: The electronic device inputs the second text feature information into a fully connected layer of the generative question answering model.

[0097] In some embodiments of the present application, the fully connected layer is used to map the text feature information to the text feature information corresponding to different response information, and then calculate the degree of matching with the text feature information of the first question information.

[0098] Step 203c2: The electronic device calculates prediction parameters corresponding to the third text feature information through a fully connected layer.

[0099] In some embodiments of the present application, the above prediction parameter is used to indicate the degree of matching between the response information corresponding to the third text feature information and the first question information.

[0100] Exemplarily, the third text feature information is input into the fully connected layer, and response information having a mapping relationship with the third text feature information is obtained to obtain the prediction parameter.

[0101] Optionally, in an embodiment of the present application, after acquiring the third text feature information, the electronic device may map the third text feature information to the same dimension as the text feature information of the response information, and then predict the mapped third text feature information to obtain a prediction parameter.

[0102] Step 203c3: When the prediction parameter is greater than or equal to a preset threshold, the electronic device determines the response information corresponding to the third text feature information as the first response information through a fully connected layer.

[0103] In some embodiments of the present application, the above-mentioned preset threshold may be a default setting of the electronic device or may be user-defined.

[0104] It can be understood that if the prediction parameter is greater than or equal to the preset threshold, it indicates that the matching degree between the answer information corresponding to the current third text feature information and the first question information meets the user's requirements.

[0105] In this way, the electronic device can combine the text feature information of the question information with the memory parameters used to characterize the user's historical behavior in the generative question-answering model, and finally generate answer information with user characteristics, thereby ensuring the efficiency of generating personalized answer information.

[0106] In the information generation method provided in the embodiment of the present application, the electronic device inputs the first question information into the first translation layer of the generative question-answering model, the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by feature extraction of the first user historical behavior data; based on the at least two first memory parameters, the first question information is processed by the first translation layer to obtain the first answer information output by the generative question-answering model, and the first answer information is the answer information corresponding to the first question information; wherein the first user historical behavior data includes at least one of the following: the user's historical browsing record, the user's historical preference data, and the user's historical question information. In this scheme, the memory parameters are obtained by feature extraction of the user's historical behavior data, and the memory parameters are combined with the first translation layer in the generative question-answering model to obtain a generative question-answering model that can generate answer information based on the user's historical behavior data. Since the memory parameters in this embodiment are obtained by feature extraction, it is not necessary to store the complete user's historical behavior data, so that the storage space of the electronic device storing the user's historical behavior data can be reduced while ensuring that accurate answer information can be generated in combination with the user's historical behavior data, thereby reducing the storage space occupied by the electronic device.

[0107] Optionally, before the above step 201, the information generating method provided in the embodiment of the present application may further include the following steps 301 to 303.

[0108] Step 301: The electronic device obtains a user identifier corresponding to the first question information.

[0109] In some embodiments of the present application, different users correspond to different user identifiers.

[0110] Exemplarily, the user identification may be an account ID used by the user to log into the electronic device.

[0111] Step 302: The electronic device determines at least two first memory parameters corresponding to the user identifier from at least two second memory parameters stored in the electronic device.

[0112] In some embodiments of the present application, the above-mentioned one user identification corresponds to at least two memory parameters.

[0113] In some embodiments of the present application, the memory parameters corresponding to the above different user identifiers can be stored in different storage areas. In other words, the memory parameters generated by different user historical behavior data corresponding to different users can be stored in partitions in the storage area of ​​the electronic device.

[0114] Exemplarily, after the electronic device obtains the user identification, it retrieves the storage area corresponding to the memory parameter through the user identification and reads it.

[0115] For example, assume that user i starts to interact with the corresponding smart assistant, for example, inputs a question. After the electronic device receives the question information of user i, it first finds the corresponding memory parameter Li from the cloud storage according to the user ID, and then loads the memory parameter Li into the generative question-answering model, which corresponds one-to-one with each transformer layer of the base model M.

[0116] For example, Figure 4 As shown in the figure, assuming that the user ID is 1, a question information request is sent to the generative question answering model. The model searches the cloud storage according to the user ID 1, that is, the storage area mentioned above, and finds the corresponding memory parameter L1. Then L1 is loaded into the chip video memory and aligned with the parameters of each transformer layer in the transformer module, corresponding to the upper right part of the figure.

[0117] Step 303: The electronic device generates a first linear layer based on at least two first memory parameters, and concatenates the first linear layer with the first translation layer to construct a generative question-answering model.

[0118] In some embodiments of the present application, the matrix parameters in the first linear layer are the at least two first memory parameters.

[0119] In some embodiments of the present application, the first linear layer may be a linear layer in an MLP module.

[0120] It is understandable that the first translation layer can be directly spliced ​​with the first linear layer, or can be spliced ​​with the MLP module having the first linear layer, and this application does not limit this.

[0121] In this way, the electronic device can select the memory parameters generated by the user historical behavior data corresponding to different users according to different user identifiers to construct the corresponding generative question and answer model. Since the electronic device in this embodiment does not need to store the user historical behavior data of all users in the model, but adds the memory parameters corresponding to the user identifier into the generative question and answer model, the size of the model is reduced, thereby reducing the storage space occupied by the electronic device.

[0122] Optionally, before the above step 302, the information generating method provided in the embodiment of the present application may further include the following steps 401 to 403.

[0123] Step 401: The electronic device obtains historical behavior data of a second user corresponding to a user identifier.

[0124] In some embodiments of the present application, the above-mentioned second user historical behavior data includes at least one of the following: user's browsing history, user's preference data, and user's historical question information.

[0125] In some embodiments of the present application, the electronic device can obtain the user's historical behavior data by means of data embedding, or by collecting all the user's historical question information in the smart assistant to obtain the user's historical behavior data.

[0126] For example, the above data tracking is to collect some information in a specific process of the application to track the application usage, which is then used to further optimize the product or provide operational data support, including visits, visitors, time on site, page views, and bounce rate. Such information collection can be roughly divided into two types: page statistics (track this virtual page view) and statistical operation behavior (track this button by an event).

[0127] Step 402: The electronic device extracts first behavior feature information from historical behavior data of the second user.

[0128] In some embodiments of the present application, the electronic device may use a self-attention mechanism to extract first behavior feature information from the second user's historical behavior data.

[0129] Exemplarily, the self-attention mechanism is to input the second user behavior data into the self-attention module, extract feature data from the second user's historical behavior data, and convert it into feature vector information, namely, the first behavior feature information.

[0130] Step 403: The electronic device determines each first behavior characteristic information as a second memory parameter to obtain at least two second memory parameters corresponding to the user identifier.

[0131] In this way, the electronic device can select memory parameters generated by user historical behavior data corresponding to different users according to different user identifiers to build a corresponding generative question-answering model, thereby ensuring that the model can generate corresponding personalized response information for different users.

[0132] Optionally, before the above-mentioned step 203 "the electronic device processes the first question information based on at least two first memory parameters through the first translation layer to obtain first response information output by the generative question and answer model", the information generation method provided in the embodiment of the present application may also include the following steps 501 to 504.

[0133] Step 501: The electronic device updates the historical behavior data of the first user based on the first response information.

[0134] Exemplarily, the electronic device will actively record the first response information after a preset time period, and record the first response information in the user's historical behavior data.

[0135] For example, at 23:00 every evening, the electronic device will obtain the user's historical behavior data generated by the user's use of the electronic device within 24 hours, and after obtaining the user's historical behavior data, the user's historical behavior data will be further updated.

[0136] Step 502: The electronic device extracts second behavior feature information from the updated historical behavior data of the first user.

[0137] In some embodiments of the present application, the electronic device may use a self-attention mechanism to extract the second behavior feature information of the updated historical behavior data of the first user.

[0138] Exemplarily, the self-attention mechanism is to input the updated historical behavior data of the first user into the self-attention module, extract the feature data in the updated historical behavior data of the first user, and convert it into feature vector information, namely, the second behavior feature information.

[0139] Step 503: The electronic device determines each second behavior characteristic information as a third memory parameter to obtain at least two third memory parameters corresponding to the historical behavior data of the first user.

[0140] Step 504: The electronic device generates a second linear layer based on at least two third memory parameters, and concatenates the second linear layer with the first translation layer to obtain an updated generative question-answering model.

[0141] In some embodiments of the present application, the model parameters of the first translation layer remain unchanged, and only the first linear layer is updated to the second linear layer.

[0142] Exemplarily, the above-mentioned generative question-answering model is composed of a large base model M and a memory parameter L, wherein the large base model M is shared by all users, and the memory parameter L is independent for each user. When the training starts, that is, when the memory parameters are updated, the electronic device will first freeze all the basic model parameters of the large base model M, that is, the basic model parameters of the large base model M will not be updated during the training process, and only the memory parameters L will be updated, that is, the parameters composed of An0, Bn0, An1, and Bn1 of all transformer layers. Then, during the training process, it is necessary to initialize the memory parameters An0, Bn0, An1, and Bn1 of different users according to the historical behavior data Di of different users. And in the process of training, the user's historical behavior data Di will only update the memory parameters An0, Bn0, An1, and Bn1 corresponding to user i.

[0143] For example, Figure 5 As shown, taking the user with id 1 as an example, in the figure, L11 represents the memory parameters of the first layer with user id 1. When training the historical behavior data D1 of the user with id 1, A00, B00, A01, and B01 of L11 will be loaded into layer1 and the parameters will be updated. The memory parameters L1n corresponding to other layers layern are the same as the operation of layer 1. In this way, all memory parameters L1=L11+L12+…+L1n of user id 1 are obtained. The training process of the memory parameters Li of other user ids is the same as that of user id 1. Repeat the above process to obtain the memory parameters of all users.

[0144] Finally, after the training is completed, the memory parameters corresponding to each user need to be saved, assuming that they are stored in cloud storage.

[0145] In this way, the electronic device can reduce the operating cost of the model by updating the training memory parameters only according to the updated user historical behavior data while ensuring that the basic model parameters of the generative question-answering model remain unchanged.

[0146] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0147] The information generation method provided in the embodiment of the present application can be executed by an electronic device or an information generation device. In the embodiment of the present application, the information generation device provided in the embodiment of the present application is described by taking the information generation method executed by the information generation device as an example.

[0148] Figure 6A possible structural diagram of the information generating device involved in the embodiment of the present application is shown. Figure 6 As shown, the information generating device 700 may include: a processing module 701.

[0149] Among them, the above-mentioned processing module 701 is used to input the first question information into the first translation layer of the generative question and answer model, and the first translation layer includes at least two first memory parameters, and the above-mentioned at least two first memory parameters are obtained by feature extraction of the first user historical behavior data; the above-mentioned processing module 701 is also used to process the first question information through the first translation layer based on at least two first memory parameters to obtain the first response information output by the generative question and answer model, and the first response information is the response information corresponding to the first question information; wherein, the first user historical behavior data includes at least one of the following: user browsing history, user preference data, and user historical question information.

[0150] Optionally, in some embodiments of the present application, the processing module 701 is specifically used to:

[0151] Extracting first text feature information of the first question information through the first translation layer;

[0152] The first text feature information is concatenated with the first memory parameter through the first translation layer to obtain the second text feature information;

[0153] The first response information is determined based on the second text feature information through the first translation layer.

[0154] Optionally, in some embodiments of the present application, the processing module 701 is specifically used to:

[0155] Inputting the second text feature information into the fully connected layer of the generative question answering model;

[0156] Calculate the prediction parameter corresponding to the third text feature information through the fully connected layer, where the prediction parameter is used to indicate the matching degree between the answer information corresponding to the third text feature and the first question information;

[0157] When the prediction parameter is greater than or equal to the preset threshold, the response information corresponding to the third text feature information is determined as the first response information through the fully connected layer.

[0158] Optionally, in some embodiments of the present application, combined with Figure 6 ,like Figure 7As shown, the above-mentioned device 700 also includes an acquisition module 702 and a determination module 703; the above-mentioned acquisition module 702 is used to obtain the user identifier corresponding to the first question information before the above-mentioned processing module 701 inputs the first question information into the first translation layer of the generative question answering model; the above-mentioned determination module 703 is used to determine at least two first memory parameters corresponding to the user identifier from at least two second memory parameters stored in the electronic device, and one user identifier corresponds to at least two memory parameters; the above-mentioned processing module 701 is also used to generate a first linear layer based on at least two first memory parameters, and splice the first linear layer with the first translation layer to construct a generative question answering model.

[0159] Optionally, in some embodiments of the present application, combined with Figure 6 ,like Figure 8 As shown, the above-mentioned device 700 also includes: an updating module 704, and the above-mentioned updating module 704 is used to update the first user historical behavior data based on the first answer information after the above-mentioned processing module 701 processes the first question information through the first translation layer based on at least two first memory parameters to obtain the first answer information output by the generative question and answer model; the above-mentioned processing module 701 is also used to extract the second behavior feature information of the updated first user historical behavior data; the above-mentioned determination module 703 is also used to determine each second behavior feature information as a third memory parameter to obtain at least two third memory parameters corresponding to the first user historical behavior data; the above-mentioned processing module 701 is also used to generate a second linear layer based on at least two third memory parameters, and splice the second linear layer with the first translation layer to obtain an updated generative question and answer model.

[0160] In the information generation device provided in the embodiment of the present application, the first question information is input into the first translation layer of the generative question-answering model, the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by feature extraction of the first user historical behavior data; based on the at least two first memory parameters, the first question information is processed by the first translation layer to obtain the first answer information output by the generative question-answering model, and the first answer information is the answer information corresponding to the first question information; wherein the first user historical behavior data includes at least one of the following: the user's historical browsing record, the user's historical preference data, and the user's historical question information. In this scheme, the memory parameters are obtained by feature extraction of the user's historical behavior data, and the memory parameters are combined with the first translation layer in the generative question-answering model to obtain a generative question-answering model that can generate answer information based on the user's historical behavior data. Since the memory parameters in this embodiment are obtained by feature extraction, it is not necessary to store the complete user historical behavior data, so that the storage space of the information generation device storing the user's historical behavior data can be reduced while ensuring that accurate answer information can be generated in combination with the user's historical behavior data, thereby reducing the storage space occupied by the information generation device.

[0161] The information generating device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0162] The information generating device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0163] The information generating device provided in the embodiment of the present application can implement each process implemented in the embodiment of the information generating method and achieve the same technical effect. To avoid repetition, it will not be described here.

[0164] Alternatively, if Fig. 9 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned information generation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0165] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0166] Fig.10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0167] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0168] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0169] Among them, the above-mentioned processor 110 is used to input the first question information into the first translation layer of the generative question and answer model, and the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by feature extraction of the first user historical behavior data; the above-mentioned processor 110 is also used to process the first question information through the first translation layer based on the at least two first memory parameters to obtain the first response information output by the generative question and answer model, and the first response information is the first question information and the corresponding response information; wherein the first user historical behavior data includes at least one of the following: user browsing history, user preference data, and user historical question information.

[0170] Optionally, in some embodiments of the present application, the processor 110 is specifically configured to:

[0171] Extracting first text feature information of the first question information through the first translation layer;

[0172] The first text feature information is concatenated with the first memory parameter through the first translation layer to obtain the second text feature information;

[0173] The first response information is determined based on the second text feature information through the first translation layer.

[0174] Optionally, in some embodiments of the present application, the processor 110 is specifically configured to:

[0175] Inputting the second text feature information into the fully connected layer of the generative question answering model;

[0176] Calculate the prediction parameter corresponding to the third text feature information through the fully connected layer, where the prediction parameter is used to indicate the matching degree between the answer information corresponding to the third text feature and the first question information;

[0177] When the prediction parameter is greater than or equal to the preset threshold, the response information corresponding to the third text feature information is determined as the first response information through the fully connected layer.

[0178] Optionally, in some embodiments of the present application, the processor 110 is further used to obtain a user identifier corresponding to the first question information before inputting the first question information into the first translation layer of the generative question answering model; the processor 110 is further used to determine at least two first memory parameters corresponding to the user identifier from at least two second memory parameters stored in the electronic device, and one user identifier corresponds to at least two memory parameters; the processor 110 is further used to generate a first linear layer based on at least two first memory parameters, and concatenate the first linear layer with the first translation layer to construct a generative question answering model.

[0179] Optionally, in some embodiments of the present application, the processor 110 is further used to update the first user historical behavior data based on the first answer information after processing the first question information through the first translation layer based on at least two first memory parameters to obtain the first answer information output by the generative question and answer model; the processor 110 is further used to extract the second behavior feature information of the updated first user historical behavior data; the processor 110 is further used to determine each second behavior feature information as a third memory parameter to obtain at least two third memory parameters corresponding to the first user historical behavior data; the processor 110 is further used to generate a second linear layer based on at least two third memory parameters, and concatenate the second linear layer with the first translation layer to obtain an updated generative question and answer model.

[0180] In the electronic device provided in the embodiment of the present application, the first question information is input into the first translation layer of the generative question-answering model, the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by feature extraction of the first user historical behavior data; based on the at least two first memory parameters, the first question information is processed by the first translation layer to obtain the first response information output by the generative question-answering model, and the first response information is the response information corresponding to the first question information; wherein the first user historical behavior data includes at least one of the following: the user's historical browsing record, the user's historical preference data, and the user's historical question information. In this scheme, the memory parameters are obtained by feature extraction of the user's historical behavior data, and the memory parameters are combined with the first translation layer in the generative question-answering model to obtain a generative question-answering model that can generate response information based on the user's historical behavior data. Since the memory parameters in this embodiment are obtained by feature extraction, it is not necessary to store the complete user historical behavior data, so that the storage space of the electronic device storing the user's historical behavior data can be reduced while ensuring that accurate response information can be generated in combination with the user's historical behavior data, thereby reducing the storage space occupied by the electronic device.

[0181] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0182] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0183] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0184] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned information generation method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0185] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0186] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned information generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0187] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0188] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned information generation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0189] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0190] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0191] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating information, characterized in that: The method comprises: Inputting the first question information into a first translation layer of a generative question-answering model, wherein the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by extracting features from historical behavior data of the first user; Based on the at least two first memory parameters, the first question information is processed by the first translation layer to obtain first response information output by the generative question-answering model, where the first response information is response information corresponding to the first question information; The first user historical behavior data includes at least one of the following: the user's historical browsing records, the user's historical preference data, and the user's historical question information.

2. The method according to claim 1, characterized in that The step of processing the first question information by the first translation layer based on the at least two first memory parameters to obtain first response information output by the generative question-answering model includes: Extracting first text feature information of the first question information through the first translation layer; The first text feature information is concatenated with the first memory parameter through the first translation layer to obtain second text feature information; The first response information is determined by the first translation layer based on the second text feature information.

3. The method according to claim 2, characterized in that The determining the first response information based on the second text feature information by the first translation layer includes: Inputting the second text feature information into a fully connected layer of the generative question answering model; Calculating, by the fully connected layer, a prediction parameter corresponding to the third text feature information, the prediction parameter being used to indicate a matching degree between the answer information corresponding to the third text feature and the first question information; When the prediction parameter is greater than or equal to a preset threshold, the response information corresponding to the third text feature information is determined as the first response information through the fully connected layer.

4. The method according to claim 1, characterized in that: Before inputting the first question information into the first translation layer of the generative question answering model, the method further includes: Obtaining a user identifier corresponding to the first question information; Determine the at least two first memory parameters corresponding to the user identifier from at least two second memory parameters stored in the electronic device, one user identifier corresponds to at least two memory parameters; A first linear layer is generated based on the at least two first memory parameters, and the first linear layer is concatenated with the first translation layer to construct the generative question-answering model.

5. The method according to claim 1, characterized in that: After the first question information is processed by the first translation layer based on the at least two first memory parameters to obtain first response information output by the generative question-answering model, the method further includes: Updating the first user historical behavior data based on the first response information; By extracting the updated second behavior feature information of the first user's historical behavior data; Determine each second behavior characteristic information as a third memory parameter to obtain at least two third memory parameters corresponding to the first user historical behavior data; A second linear layer is generated based on the at least two third memory parameters, and the second linear layer is concatenated with the first translation layer to obtain the updated generative question-answering model.

6. An information generating device, characterized in that: The information generating device comprises: a processing module; The processing module is used to input the first question information into a first translation layer of a generative question-answering model, wherein the first translation layer includes at least two first memory parameters, and the at least two first memory parameters are obtained by extracting features from the first user's historical behavior data; The processing module is further configured to process the first question information through the first translation layer based on the at least two first memory parameters to obtain first response information output by the generative question-answering model, wherein the first response information is response information corresponding to the first question information; The first user historical behavior data includes at least one of the following: user browsing history, user preference data, and user historical question information.

7. The device according to claim 6, characterized in that The processing module is specifically used for: Extracting first text feature information of the first question information through the first translation layer; The first text feature information is concatenated with the first memory parameter through the first translation layer to obtain second text feature information; The first response information is determined by the first translation layer based on the second text feature information.

8. The device according to claim 7, characterized in that The processing module is specifically used for: Inputting the second text feature information into a fully connected layer of the generative question answering model; Calculating, by the fully connected layer, a prediction parameter corresponding to the third text feature information, the prediction parameter being used to indicate a matching degree between the answer information corresponding to the third text feature and the first question information; When the prediction parameter is greater than or equal to a preset threshold, the response information corresponding to the third text feature information is determined as the first response information through the fully connected layer.

9. The device according to claim 6, characterized in that The device also includes an acquisition module and a determination module; The acquisition module is configured to acquire a user identifier corresponding to the first question information before the processing module inputs the first question information into the first translation layer of the generative question answering model; The determination module is used to determine the at least two first memory parameters corresponding to the user identifier from the at least two second memory parameters stored in the electronic device, and one user identifier corresponds to at least two memory parameters; The processing module is further used to generate a first linear layer based on the at least two first memory parameters, and concatenate the first linear layer with the first translation layer to construct the generative question-answering model.

10. The device according to claim 6, characterized in that The device further includes: an updating module, configured to update the first user historical behavior data based on the first answer information after the processing module processes the first question information through the first translation layer based on the at least two first memory parameters to obtain the first answer information output by the generative question-answering model; The processing module is further used to extract the second behavior feature information of the updated historical behavior data of the first user; The determination module is further configured to determine each second behavior characteristic information as a third memory parameter, so as to obtain at least two third memory parameters corresponding to the first user historical behavior data; The processing module is further used to generate a second linear layer based on the at least two third memory parameters, and concatenate the second linear layer with the first translation layer to obtain the updated generative question-answering model.

Images