Dialogue data generation method and related device thereof

Abstract

The present application discloses a dialogue data generation method and a related device, which can effectively improve the generation efficiency of dialogue data and reduce the generation cost of dialogue data. The method of the present application comprises: when dialogue data between a user and a dialogue model needs to be generated, acquire user data, the user data comprising attributes of the user and items that the user is interested in; processing the user data to obtain recommendable items, the recommendable items comprising the items that the user is interested in and items associated with the items that the user is interested in; updating a preset data generation rule on the basis of the user data and the recommendable items, so as to obtain a new data generation rule; and generating the dialogue data between the user and the dialogue model according to the new data generation rule, wherein the dialogue data comprises items recommended by the dialogue model to the user, and the items recommended by the dialogue model to the user are from the recommendable items.

Description

A method for generating conversation data and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 16, 2023, with application number 202310724012.8 and invention name “A method for generating conversation data and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of artificial intelligence (AI) technology, and in particular to a method for generating conversation data and related devices. Background Art

[0003] In AI-generated content (AIGC), the conversational model (language model) with item recommendation function has demonstrated strong natural language understanding and communication capabilities. The model can recommend items to users by having conversations with them, thereby meeting their item recommendation needs.

[0004] To train a conversational model capable of item recommendation, crowdsourcing can be used to obtain conversational data for model training. Specifically, crowdsourcing staff can play the roles of users and conversational models, respectively. Using information from a pre-built knowledge graph, they generate conversational data between the user and the conversational model. The staff extracts relevant content from the conversational data, as well as the items recommended to the user by the conversational model, from the knowledge graph. This conversational data can then be used to train a recommendation model. The resulting trained recommendation model can then learn from the user's interests and preferences, thereby recommending items to the user.

[0005] In the above process, the entire process of generating dialogue data needs to be the responsibility of the crowdsourcing project staff, which takes a lot of time, that is, the labor cost is too high.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a method for generating conversation data and related devices, which can effectively improve the efficiency of conversation data generation and reduce the cost of conversation data generation.

[0008] A first aspect of an embodiment of the present application provides a method for generating conversation data, the method comprising:

[0009] When conversation data needs to be obtained, user data can be collected first. User data generally includes user attributes and items that the user is interested in. User attributes may include the user's name, gender, and age, etc. Items that the user is interested in may include items that the user has clicked on, items that the user has browsed, items that the user has commented on, etc.

[0010] After obtaining the user data, a series of processing can be performed on the user data to obtain items that can be recommended to the user. It should be noted that the items that can be recommended to the user include not only the items that the user is interested in in the user data, but also other items related to the items that the user is interested in.

[0011] After obtaining items that can be recommended to the user, the pre-set data generation rules can be updated using the user data and the recommended items, thereby generating new data generation rules. It should be noted that the new data generation rules not only constrain the format of the conversation data between the user and the conversation model, but also constrain the content of the conversation data between the user and the conversation model.

[0012] After obtaining the new data generation rules, the conversation data between the user and the dialogue model can be generated according to the new data generation rules. It should be noted that the conversation data between the user and the dialogue model typically includes multiple rounds of interaction, which include multiple rounds of normal conversation and multiple rounds of question and answer. Each normal conversation round includes a non-question utterance from the user and a response from the dialogue model (or, each normal conversation round includes a non-question utterance from the dialogue model and a response from the user), and each question and answer round includes a question from the user and an answer from the dialogue model (or, each question and answer round includes a question from the dialogue model and an answer from the user).

[0013] The above method demonstrates that when generating conversation data between a user and a conversation model, user data can first be acquired. This user data includes the user's attributes and items of interest to the user. Next, the user data can be processed to obtain recommended items, which include items of interest to the user and items related to those items of interest. The user data and recommended items can then be used to update the pre-set data generation rules, thereby generating new data generation rules. Finally, the new data generation rules can be used to generate conversation data between the user and the conversation model. This conversation data includes items recommended to the user by the conversation model, which are selected from the recommended items. The above process provides a conversation data generation framework that utilizes pre-set data generation rules from the collected user data set to obtain new data generation rules. The new data generation rules are then used to generate conversation data between the user and the conversation model. This framework can automatically generate conversation data without requiring human intervention, effectively improving the efficiency and reducing the cost of conversation data generation.

[0014] In one possible implementation, the method further includes: determining non-recommended items based on user data; and updating preset data generation rules based on user data and recommended items to obtain new data generation rules, including: updating preset data generation rules based on user data, recommended items, and non-recommended items to obtain new data generation rules. In the aforementioned implementation, after a series of processing of user data, not only recommended items can be obtained, but also items that cannot be recommended to the user (that is, items that the user is not interested in) can be obtained. Then, the preset data generation rules can be updated using user data, items that can be recommended to the user, and items that cannot be recommended to the user, thereby obtaining new data generation rules. In this way, the new data generation rules can make the conversation data between the user and the conversation model include the following multiple contents: the user accepts the items recommended by the conversation model and the user rejects the items recommended by the conversation model, so that the final generated conversation data can be more realistic.

[0015] In one possible implementation, determining recommendable items based on user data includes: extracting features from the user data and multiple candidate items to obtain features of the user data and the multiple candidate items; calculating the features of the user data and the multiple candidate items to obtain a degree of match between the user data and the multiple candidate items; and determining candidate items with a degree of match greater than or equal to a first threshold as recommendable items. In the aforementioned implementation, after obtaining the user data, multiple candidate items may be obtained. Feature extraction may then be performed on the user data and the multiple candidate items to obtain features of the user data and the multiple candidate items. After obtaining the features of the user data and the multiple candidate items, a series of calculations may be performed on the features of the user data and the multiple candidate items to obtain a degree of match between the user data and the multiple candidate items, namely, a degree of match between the user data and the first candidate item, a degree of match between the user data and the second candidate item, ..., and a degree of match between the user data and the last candidate item. After obtaining the degree of match between the user data and the multiple candidate items, candidate items with a degree of match greater than or equal to the first threshold may be determined as recommendable items from the multiple candidate items. In this way, the items that the user is interested in and the items related to the items that the user is interested in can be accurately obtained and used as items that can be recommended to the user.

[0016] In one possible implementation, determining non-recommended items based on user data includes determining candidate items with a matching degree less than or equal to a second threshold as non-recommended items, where the second threshold is less than the first threshold. In this implementation, after determining the matching degree between the user data and multiple candidate items, candidate items with a matching degree less than or equal to the second threshold can be determined as non-recommended items from the multiple candidate items. This allows accurate identification of items that the user is not interested in, and these items can be designated as non-recommended items.

[0017] In one possible implementation, the preset data generation rules include rules that do not need to be supplemented and rules to be supplemented. Based on user data, recommended items, and non-recommended items, updating the preset data generation rules to obtain new data generation rules includes: populating the user data, recommended items, and non-recommended items into the rules to be supplemented, thereby obtaining supplemented rules. The rules that do not need to be supplemented and the supplemented rules constitute the new data generation rules, where the rules that do not need to be supplemented are used to set the format of the conversation data, and the supplemented rules are used to set the content of the conversation data. In the aforementioned implementation, the preset data generation rules may include two parts: one part is the rules that do not need to be supplemented, and the other part is the rules that are to be supplemented. The rules that are to be supplemented have slots that can be filled with content. Then, the user data, recommended items, and non-recommended items can be added to the slots of the rules that are to be supplemented, thereby obtaining supplemented rules. Thus, the rules that do not need to be supplemented and the supplemented rules constitute the new data generation rules, where the rules that do not need to be supplemented are used to set the format of the conversation data between the user and the conversation model, and the supplemented rules are used to set the content of the conversation data between the user and the conversation model. Then, the dialogue data obtained based on the new data generation rules has a certain format, and its content includes not only the content of the chat between the user and the dialogue model, but also the content of the user accepting the items recommended by the dialogue model, and even the content of the user rejecting the items recommended by the dialogue model, so that the dialogue data finally generated is closer to reality.

[0018] In one possible implementation, the method further includes obtaining target conversation data from the conversation data, where the target conversation data satisfies at least one of the following: all text contained in the target conversation data is within a preset text set; the descriptions of items contained in the target conversation data are true descriptions; the items contained in the target conversation data are authentic items; and the richness of the content contained in the target conversation data is greater than or equal to a third threshold. In the aforementioned implementation, after obtaining the conversation data between the user and the conversation model, several rounds of interactions that do not meet the requirements can be removed from the multiple rounds of interactions contained in the conversation data, and several rounds of interactions that meet the requirements can be selected. These rounds of interactions constitute the target conversation data. At this point, the data can be completely cleaned and evaluated, and the resulting target conversation data can be used as training data.

[0019] In one possible implementation, the method further includes training the conversation model based on the target conversation data to obtain a trained conversation model. In the aforementioned implementation, since the target conversation data can serve as training data, the conversation model can be trained using the target conversation data to obtain a trained conversation model, i.e., a conversation model with item recommendation functionality.

[0020] The second aspect of an embodiment of the present application provides a data processing method, which is implemented through the trained dialogue model involved in the first aspect. The method includes: obtaining a user's question, which is used to describe the user's item recommendation needs; inputting the user's question into the trained dialogue model to obtain an answer corresponding to the question, which is used to describe the item recommendation results for the user.

[0021] A third aspect of an embodiment of the present application provides a conversation data generation device, which includes: an acquisition module for acquiring user data, the user data including user attributes and items of interest to the user; a determination module for determining recommendable items based on the user data, the recommendable items including items of interest to the user and items associated with the items of interest to the user; an update module for updating preset data generation rules based on the user data and the recommendable items to obtain new data generation rules; and a generation module for generating conversation data between a user and a conversation model based on the new data generation rules, the conversation data including items recommended to the user by the conversation model, and the items recommended to the user by the conversation model are from the recommendable items.

[0022] As can be seen from the above apparatus, when generating conversation data between a user and a conversation model, user data can first be acquired. This user data includes the user's attributes and items of interest to the user. Next, the user data can be processed to obtain recommended items. Recommended items include items of interest to the user and items related to the items of interest to the user. Then, the user data and the recommended items can be used to update the pre-set data generation rules, thereby generating new data generation rules. Finally, conversation data between the user and the conversation model can be generated according to the new data generation rules. The conversation data includes items recommended to the user by the conversation model, where the items recommended by the conversation model are selected from the recommended items. The above process provides a conversation data generation framework that utilizes pre-set data generation rules from the collected user data set to obtain new data generation rules. The new data generation rules are then used to generate conversation data between the user and the conversation model. This framework can automatically generate conversation data without requiring human intervention, effectively improving the efficiency of conversation data generation and reducing the cost of conversation data generation.

[0023] In one possible implementation, the determination module is further used to determine non-recommended items based on user data; the update module is used to update the preset data generation rules based on the user data, recommendable items and non-recommended items to obtain new data generation rules.

[0024] In one possible implementation, the determination module is configured to: extract features from user data and multiple candidate items to obtain features of the user data and features of the multiple candidate items; calculate the features of the user data and features of the multiple candidate items to obtain a degree of match between the user data and the multiple candidate items; and determine candidate items having a degree of match greater than or equal to a first threshold as recommendable items.

[0025] In one possible implementation, the determination module is configured to determine candidate items having a matching degree less than or equal to a second threshold as non-recommended items, where the second threshold is less than the first threshold.

[0026] In one possible implementation, the preset data generation rules include rules that do not need to be supplemented and rules to be supplemented. The update module is used to: fill user data, recommended items and non-recommended items into the rules to be supplemented to obtain the supplemented rules. The rules that do not need to be supplemented and the supplemented rules constitute new data generation rules. The rules that do not need to be supplemented are used to set the format of the conversation data, and the supplemented rules are used to set the content of the conversation data.

[0027] In one possible implementation, the device also includes: a screening module for obtaining target conversation data from the conversation data, where the target conversation data satisfies at least one of the following: all text contained in the target conversation data is located in a preset text set; the description of the object contained in the target conversation data is consistent with the real description; the object contained in the target conversation data is a real object; the richness of the content contained in the target conversation data is greater than or equal to a third threshold.

[0028] In one possible implementation, the device further includes: a training module, configured to train the dialogue model based on target dialogue data to obtain a trained dialogue model.

[0029] The fourth aspect of an embodiment of the present application provides a data processing device, which includes the trained dialogue model involved in the third aspect. The device includes: an acquisition module for acquiring a user's question, which is used to describe the user's item recommendation needs; a processing module for inputting the user's question into the trained dialogue model to obtain an answer corresponding to the question, which is used to describe the item recommendation results for the user.

[0030] A fifth aspect of an embodiment of the present application provides a conversation data generation device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the conversation data generation device performs the method described in the first aspect or any possible implementation of the first aspect.

[0031] A sixth aspect of an embodiment of the present application provides a data processing device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code. When the code is executed, the data processing device executes the method described in the second aspect or any possible implementation method of the second aspect.

[0032] A seventh aspect of an embodiment of the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0033] An eighth aspect of an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0034] In a possible implementation, the processor is coupled to the memory through an interface.

[0035] In one possible implementation, the chip system further includes a memory in which computer programs or computer instructions are stored.

[0036] A ninth aspect of an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, enables the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0037] A tenth aspect of an embodiment of the present application provides a computer program product, which stores instructions that, when executed by a computer, enable the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect.

[0038] In an embodiment of the present application, when generating conversation data between a user and a conversation model, user data can first be acquired. The user data includes user attributes and items of interest to the user. Next, the user data can be processed to obtain recommended items. Recommended items include items of interest to the user and items related to the items of interest to the user. Pre-set data generation rules can then be updated using the user data and the recommended items, thereby generating new data generation rules. Finally, conversation data between the user and the conversation model can be generated according to the new data generation rules. The conversation data includes items recommended to the user by the conversation model, where the items recommended by the conversation model are selected from the recommended items. The aforementioned process provides a conversation data generation framework that utilizes pre-set data generation rules from a collected set of user data to obtain new data generation rules. The new data generation rules are then used to generate conversation data between the user and the conversation model. This framework can automatically generate conversation data without requiring human intervention, effectively improving the efficiency of conversation data generation and reducing the cost of conversation data generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a schematic diagram of the structure of the artificial intelligence main framework;

[0040] FIG2a is a schematic structural diagram of a data processing system provided in an embodiment of the present application;

[0041] FIG2 b is another schematic diagram of the structure of the data processing system provided in an embodiment of the present application;

[0042] FIG2c is a schematic diagram of a data processing device according to an embodiment of the present application;

[0043] FIG3 is a schematic diagram of the architecture of the system 100 provided in an embodiment of the present application;

[0044] FIG4 is a schematic diagram of a structure of a conversation data generation architecture provided in an embodiment of the present application;

[0045] FIG5 is a flow chart of a method for generating conversation data according to an embodiment of the present application;

[0046] FIG6 is a schematic diagram of a user data conversion process provided by an embodiment of the present application;

[0047] FIG7 is a schematic diagram of a process for obtaining recommended items according to an embodiment of the present application;

[0048] FIG8 is a schematic diagram of a process for obtaining data generation rules according to an embodiment of the present application;

[0049] FIG9 is a schematic diagram of a process for acquiring conversation data according to an embodiment of the present application;

[0050] FIG10 is a schematic diagram of a process for acquiring conversation data according to an embodiment of the present application;

[0051] FIG11 is a schematic diagram of the scoring results provided in an embodiment of the present application;

[0052] FIG12 is another schematic diagram of the scoring results provided in an embodiment of the present application;

[0053] FIG13 is another schematic diagram of the scoring results provided in an embodiment of the present application;

[0054] FIG14 is another schematic diagram of the scoring results provided in an embodiment of the present application;

[0055] FIG15 is a flow chart of a data processing method provided in an embodiment of the present application;

[0056] FIG16 is a schematic structural diagram of a conversation data generating device provided in an embodiment of the present application;

[0057] FIG17 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0058] FIG18 is a schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0059] FIG19 is a schematic structural diagram of a training device provided in an embodiment of the present application;

[0060] FIG20 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The embodiments of the present application provide a method for generating conversation data and related devices, which can effectively improve the efficiency of conversation data generation and reduce the cost of conversation data generation.

[0062] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0063] In AIGC, the conversational model with item recommendation function (also known as the language large model with item recommendation function) demonstrates strong natural language understanding and communication capabilities. The model can recommend items to users by having conversations with them, thereby meeting users' item recommendation needs.

[0064] To train a conversational model capable of item recommendation, crowdsourcing can be used to obtain conversational data for model training. Specifically, crowdsourcing staff can play the roles of users and conversational models, respectively. Using information from a pre-built knowledge graph, they generate conversational data between the user and the conversational model. The staff extracts relevant content from the conversational data, as well as the items recommended to the user by the conversational model, from the knowledge graph. This conversational data can then be used to train a recommendation model. The resulting trained recommendation model can then learn from the user's interests and preferences, thereby recommending items to the user.

[0065] In the above process, the entire process of generating conversation data needs to be the responsibility of the crowdsourcing project staff. The staff training time is long, and extracting information also takes a lot of staff time. It can be seen that the labor cost required to generate conversation data in this way is too high.

[0066] Furthermore, due to the differences in educational backgrounds among the staff of the crowdsourcing project, different staff members have different understandings of the knowledge graph, which will lead to deviations in the quality of the generated dialogue data. This will not only make subsequent data cleaning and evaluation difficult, but also lead to poor performance of the dialogue model obtained through subsequent training.

[0067] In order to solve the above problems, an embodiment of the present application provides a method for generating dialogue data, which can be implemented in combination with artificial intelligence (AI) technology. AI technology is a technical discipline that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence. AI technology achieves the best results by sensing the environment, acquiring knowledge and using knowledge. In other words, artificial intelligence technology is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Using artificial intelligence for data processing is a common application of artificial intelligence.

[0068] First, we will describe the overall workflow of an AI system. See Figure 1, which illustrates a schematic diagram of the AI ​​framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0069] (1) Infrastructure

[0070] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0071] (2) Data

[0072] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0073] (3) Data processing

[0074] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0075] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0076] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0077] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0078] (4) General ability

[0079] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0080] (5) Smart products and industry applications

[0081] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0082] Next, several application scenarios of this application are introduced.

[0083] Figure 2a is a schematic diagram of the structure of a data processing system provided in an embodiment of the present application. The data processing system includes a user device and a data processing device. The user device includes an intelligent terminal such as a mobile phone, a personal computer, or an information processing center. The user device is the initiator of data processing and is the initiator of a data processing request, typically initiated by a user through the user device.

[0084] The aforementioned data processing devices can be devices or servers with data processing capabilities, such as cloud servers, network servers, application servers, and management servers. The data processing devices receive data processing requests from smart terminals via interactive interfaces and then perform data processing methods such as machine learning, deep learning, search, reasoning, and decision-making through the memory used to store data and the processors used for data processing. The memory in the data processing device is a general term that includes local storage and databases that store historical data. The database can be located on the data processing device or on other network servers.

[0085] In the data processing system shown in FIG2a, a user device can receive user instructions. For example, the user device can obtain a question input / selected by the user and then initiate a request to the data processing device, causing the data processing device to perform a series of processing on the question from the user device, thereby obtaining a processing result for the question. For example, the user device can obtain a question input by the user (for example, the question can describe the user's item recommendation needs), and then the user device can initiate a data processing request to the data processing device, causing the data processing device to perform a series of processing on the question based on the data processing request, thereby obtaining a processing result for the question, i.e., a corresponding answer (for example, the answer can describe the item recommendation result for the user, i.e., the item recommended to the user).

[0086] In FIG. 2 a , the data processing device may execute the data processing method according to the embodiment of the present application.

[0087] Figure 2b is another structural diagram of the data processing system provided in an embodiment of the present application. In Figure 2b, the user device directly serves as a data processing device. The user device can directly obtain input from the user and directly process it by the hardware of the user device itself. The specific process is similar to that of Figure 2a. Please refer to the above description and will not be repeated here.

[0088] In the data processing system shown in Figure 2b, the user device can receive instructions from the user. For example, the user device can obtain a question input by the user (for example, the question can describe the user's item recommendation needs), and then the user device can perform a series of processing on the question to obtain the processing result of the question, that is, the corresponding answer (for example, the answer can describe the item recommendation result for the user, that is, the item recommended to the user).

[0089] In FIG2b , the user equipment itself can execute the data processing method of the embodiment of the present application.

[0090] FIG2c is a schematic diagram of data processing related equipment provided in an embodiment of the present application.

[0091] The user device in the above Figures 2a and 2b can specifically be the local device 301 or the local device 302 in Figure 2c, and the data processing device in Figure 2a can specifically be the execution device 210 in Figure 2c, wherein the data storage system 250 can store the data to be processed of the execution device 210, and the data storage system 250 can be integrated on the execution device 210, or it can be set on the cloud or other network servers.

[0092] The processors in Figures 2a and 2b can perform data training / machine learning / deep learning through a neural network model or other models (for example, a model based on a support vector machine), and use the model finally trained or learned from the data to execute data processing applications on the image, thereby obtaining corresponding processing results.

[0093] Figure 3 is a schematic diagram of the system 100 architecture provided in an embodiment of the present application. In Figure 3, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with an external device. The user can input data to the I / O interface 112 through the client device 140. The input data can include: various tasks to be scheduled, callable resources and other parameters in the embodiment of the present application.

[0094] When the execution device 110 preprocesses the input data, or when the computing module 111 of the execution device 110 performs calculations and other related processing (such as implementing the functions of the neural network model in this application), the execution device 110 can call the data, code, etc. in the data storage system 150 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 150.

[0095] Finally, the I / O interface 112 returns the processing result to the client device 140 so as to provide it to the user.

[0096] It is worth noting that the training device 120 can generate corresponding target models (e.g., the trained dialogue model provided in the embodiments of the present application) / rules based on different training data for different goals or tasks. The corresponding target models / rules can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results. The training data can be stored in the database 130 and come from the training samples collected by the data collection device 160.

[0097] In the scenario shown in FIG3 , the user can manually input data, which can be performed through the interface provided by I / O interface 112. Alternatively, client device 140 can automatically send input data to I / O interface 112. If user authorization is required for client device 140 to automatically send input data, the user can set the corresponding permissions in client device 140. The user can view the output of execution device 110 on client device 140, which can be presented in a display, sound, action, or other specific form. Client device 140 can also serve as a data acquisition terminal, collecting input data and output results from I / O interface 112 as new sample data and storing them in database 130. Of course, collection can also be performed without client device 140, with I / O interface 112 directly storing the input data and output results from I / O interface 112 as new sample data in database 130.

[0098] It is worth noting that FIG3 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in FIG3, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 can also be placed in the execution device 110. As shown in FIG3, a neural network can be trained according to the training device 120.

[0099] The present application also provides a chip including a neural network processor (NPU). The chip can be provided in the execution device 110 shown in FIG3 to complete the computational work of the computation module 111. The chip can also be provided in the training device 120 shown in FIG3 to complete the training work of the training device 120 and output the target model / rule.

[0100] The neural network processor (NPU) is a coprocessor mounted on the host central processing unit (CPU), which assigns tasks to it. The core of the NPU is the arithmetic circuit, which is controlled by a controller to extract data from memory (weight memory or input memory) and perform calculations.

[0101] In some implementations, the arithmetic circuit includes multiple processing engines (PEs). In some implementations, the arithmetic circuit is a two-dimensional systolic array. The arithmetic circuit can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit is a general-purpose matrix processor.

[0102] For example, consider an input matrix A, a weight matrix B, and an output matrix C. The computational circuit retrieves the corresponding data for matrix B from the weight memory and caches it on each PE within the computational circuit. The computational circuit then performs a matrix operation on the matrix A data from the input memory and matrix B, storing the partial or final matrix results in the accumulator.

[0103] The vector computation unit can further process the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector computation unit can be used for network calculations in non-convolutional / non-FC layers in neural networks, such as pooling, batch normalization, and local response normalization.

[0104] In some implementations, the vector computation unit can store the processed output vectors in a unified buffer. For example, the vector computation unit can apply a nonlinear function to the output of the computation circuit, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit generates normalized values, merged values, or both. In some implementations, the processed output vectors can be used as activation inputs to the computation circuit, such as for use in subsequent layers in a neural network.

[0105] The unified memory is used to store input data and output data.

[0106] The weight data is directly transferred from the external memory to the input memory and / or unified memory through the direct memory access controller (DMAC), the weight data in the external memory is stored in the weight memory, and the data in the unified memory is stored in the external memory.

[0107] The bus interface unit (BIU) is used to implement interaction between the main CPU, DMAC and instruction fetch memory through the bus.

[0108] An instruction fetch buffer connected to the controller, used to store instructions used by the controller;

[0109] The controller is used to call the instructions cached in the memory to control the working process of the computing accelerator.

[0110] Generally, unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories, and external memory is memory outside the NPU, which can be double data rate synchronous dynamic random access memory (DDR SDRAM), high bandwidth memory (HBM) or other readable and writable memory.

[0111] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0112] (1) Neural Network

[0113] A neural network can be composed of neural units. A neural unit can refer to an operation unit with xs and intercept 1 as input. The output of the operation unit can be:

[0114] Where s = 1, 2, ... n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0115] The operation of each layer in a neural network can be described by the mathematical expression y = a(Wx + b). From a physical perspective, the operation of each layer in a neural network can be understood as transforming the input space (a set of input vectors) into the output space (i.e., from the row space to the column space of a matrix) through five operations. These operations include: 1. Dimensionality increase / decrease; 2. Scaling / reduction; 3. Rotation; 4. Translation; and 5. "Bending." Operations 1, 2, and 3 are performed by Wx, 4 by +b, and 5 by a(). The word "space" is used here because the objects being classified are not individual things, but rather a class of things, and space refers to the collection of all individuals within that class. W is the weight vector, each value in which represents the weight of a neuron in that layer of the neural network. This vector W determines the spatial transformation from input space to output space described above. That is, the weights W of each layer control how the space is transformed. The goal of training a neural network is to ultimately obtain the weight matrix for all layers of the trained neural network (a weight matrix formed by the vectors W of many layers). Therefore, the training process of a neural network is essentially about learning how to control spatial transformations, and more specifically, about learning the weight matrix.

[0116] Because we want the output of the neural network to be as close as possible to the desired value, we can compare the current network's prediction with the desired target value, and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the neural network.) For example, if the network's prediction is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the neural network can predict the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Therefore, training the neural network becomes a process of minimizing this loss as much as possible.

[0117] (2) Backpropagation algorithm

[0118] Neural networks can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during training, reducing the reconstruction error loss of the neural network model. Specifically, the forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backward propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0119] The method provided in this application is described below from the training side of the neural network and the application side of the neural network.

[0120] The conversation data generation method provided in the embodiments of the present application can generate conversation data between a user and a conversation model, and use the conversation data as training data to train the conversation model, thereby obtaining a trained conversation model. During the model training process, data sequence processing is involved, which can be specifically applied to methods such as data training, machine learning, and deep learning. The training data (for example, the conversation data in the conversation data generation method provided in the embodiments of the present application) is symbolized and formalized for intelligent information modeling, extraction, preprocessing, and training, ultimately obtaining a trained neural network (for example, the trained conversation model in the conversation data generation method provided in the embodiments of the present application). In addition, the data processing method provided in the embodiments of the present application can use the above-mentioned trained neural network to input input data (for example, the question in the data processing method provided in the embodiments of the present application) into the trained neural network to obtain output data (for example, the answer in the data processing method provided in the embodiments of the present application). It should be noted that the conversation data generation method and data processing method provided in the embodiments of the present application are inventions based on the same concept and can also be understood as two parts of a system, or two stages of an overall process: such as the model training stage and the model application stage.

[0121] The conversation data generation method provided in the embodiment of the present application can be applied to the conversation data generation architecture shown in Figure 4. Figure 4 is a structural diagram of the conversation data generation architecture provided in the embodiment of the present application. As shown in Figure 4, the architecture includes: obtaining user data in non-text form and converting it into user data in text form. Then, new data generation rules can be generated based on the user data in text form combined with preset data generation rules. Then, the new data generation rules can be used to obtain conversation data. Finally, the target conversation data is filtered out from the conversation data, and the target conversation data can be used as training data to train the final model. In order to further understand the workflow of the conversation data generation architecture, the workflow is further introduced below in conjunction with Figure 5. Figure 5 is a flow chart of the conversation data generation method provided in the embodiment of the present application. As shown in Figure 5, the method includes:

[0122] 501. Obtain user data, where the user data includes user attributes and items of interest to the user.

[0123] In this embodiment, when conversation data needs to be obtained, user data can be collected first. User data generally includes user attributes (for example, user's name, user's gender, user's age, etc.) and items that the user is interested in (preferred) (for example, items that the user has clicked on, items that the user has browsed, items that the user has commented on, etc.).

[0124] Specifically, user data can be obtained in the following ways:

[0125] When collecting user data, the user data that can be directly collected is usually in non-text form. Therefore, the collected non-text form user data can be converted into text form user data to facilitate subsequent processing.

[0126] User attributes are generally located in their personal information. Therefore, the user's personal information can be collected first (usually in the form of an image or a table). In this case, the user's attributes are presented in a non-textual form, such as an image or table. Then, the user's attributes presented in non-textual form can be converted from the user's personal information into textual attributes.

[0127] Items of interest to users are typically found in their comments, browsing history, or click history. Therefore, user comments, browsing history, or click history can be collected (usually in the form of a table, etc.). In this case, the items of interest to users are presented in a non-textual format, such as a table. Then, the items of interest presented in non-textual format can be converted from the user comments, browsing history, or click history into textual format.

[0128] For example, as shown in Figure 6 (Figure 6 is a schematic diagram of the user data conversion process provided in an embodiment of the present application), taking the movie recommendation scenario as an example, the user's profile image, user comments, user browsing history, and user click history can be collected first.

[0129] The user's profile image presents the following user attribute information: (1) User: Li XX; (2) Age: 22; (3) Location: Shenzhen; (4) Interests: Movies and Music, etc. Then, the following user attribute information in text form can be identified from the user's profile image: Li XX, a 22-year-old male user living in Shenzhen, whose hobbies are watching movies and listening to music, etc.

[0130] The user's comments include the following user's movie review information: (1) Movie 1 is the most interesting movie I have ever seen; (2) Movies 2 and 3 are boring, and their plots and performances are not outstanding, etc. The user's browsing / clicking records include the following user's movie browsing information / movie clicking information: (1) Movies watched: Movie 1 and Movie 4, etc.; (2) Movies you want to watch: Movie 5, Movie 6, and Movie 7, etc.; (3) Movies you have browsed: Movie 8 and Movie 9, etc. Then, the following textual user movie preference information can be extracted from the user's comments and the user's viewing / clicking records: User Zhang XX likes the following movies: Movie 5, Movie 6, Movie 7, Movie 8, and Movie 9, etc., and User Zhang XX has watched the following movies: Movie 1 and Movie 4.

[0131] 502. Determine recommendable items based on user data, where the recommendable items include items that the user is interested in and items related to the items that the user is interested in.

[0132] After obtaining user data, the user data can be input into an item sampling model (a trained neural network model) to perform a series of processing on the user data, thereby obtaining items that can be recommended to the user. It should be noted that the recommended items can be presented in the form of an item list, which includes not only the items of interest to the user in the user data, but also other items related to the items of interest to the user.

[0133] Furthermore, after a series of processing of user data by the item sampling model, not only recommended items can be obtained, but also items that cannot be recommended to the user (that is, items that the user is not interested in). It should be noted that items that cannot be recommended to the user can also be presented in the form of an item list.

[0134] Specifically, the item sampling model can obtain items that can be recommended to the user and items that cannot be recommended to the user in the following ways:

[0135] (1) After obtaining the user data, a preset candidate item pool can be obtained first. The candidate item pool contains multiple candidate items. Therefore, the user data and the multiple candidate items can be input into the item sampling model, so that the item sampling model can perform feature extraction (for example, convolution operation, etc.) on the user data and the multiple candidate items respectively, thereby obtaining the features of the user data and the features of the multiple candidate items accordingly.

[0136] (2) After obtaining the features of the user data and the features of the multiple candidate items, the item sampling model can calculate the features of the user data and the features of the multiple candidate items, thereby obtaining the matching degree between the user data and the multiple candidate items, including the matching degree between the user data and the first candidate item, the matching degree between the user data and the second candidate item, ..., the matching degree between the user data and the last candidate item.

[0137] (3) After obtaining the matching degree between the user data and multiple candidate items, the item sampling model can determine the candidate items with a matching degree greater than or equal to a first threshold (the size of the first threshold can be set according to actual needs and is not limited here) among the multiple candidate items as items that can be recommended to the user. It should be noted that among the multiple candidate items, some candidate items are the same as the items that the user is interested in contained in the user data, so the matching degree between this part of the candidate items and the user data is the largest (significantly greater than the first threshold), and can be determined as items that can be recommended to the user. Moreover, among the multiple candidate items, some candidate items are related to the items that the user is interested in contained in the user data, so the matching degree between this part of the candidate items and the user data is relatively large (greater than or equal to the first threshold), and can be determined as items that can be recommended to the user. It can be seen that the items that can be recommended to the user include not only the items that the user is interested in in the user data, but also the items that are related to the items that the user is interested in.

[0138] (4) After obtaining the matching degree between the user data and multiple candidate items, the item sampling model can determine, among the multiple candidate items, candidate items whose matching degree is less than or equal to a second threshold (the second threshold is less than the first threshold, and the size of the second threshold can be set according to actual needs and is not limited here) as items that cannot be recommended to the user. It should be noted that among the multiple candidate items, some candidate items are completely unrelated to the items of interest to the user included in the user data. Therefore, the matching degree between these candidate items and the user data is relatively small (less than or equal to the second threshold), and they can be determined as items that cannot be recommended to the user, that is, items that the user is not interested in.

[0139] Still taking the above example, as shown in FIG7 (FIG. 7 is a schematic diagram of the process of obtaining recommended items provided in an embodiment of the present application, and FIG7 is drawn based on FIG6), after obtaining the user attribute information and user preference information, a candidate movie pool can be obtained. The candidate movie pool includes 50 candidate movies: Movie 1, Movie 2, Movie 3, ..., Movie 50. After inputting the user attribute information, user preference information, and candidate movie pool into the item sampling model, a recommended movie list and a non-recommended movie list can be obtained. The recommended movie list includes: Movie 1, Movie 4, Movie 5, ..., Movie 20, and the non-recommended movie list includes: Movie 2, Movie 3, Movie 21, ..., Movie 50.

[0140] 503. Based on the user data and the recommended items, the preset data generation rules are updated to obtain new data generation rules.

[0141] After obtaining items that can be recommended to the user, the preset data generation rules can be updated using the user data and the items that can be recommended to the user, thereby obtaining new data generation rules.

[0142] Furthermore, if not only items that can be recommended to users but also items that cannot be recommended to users are obtained, the preset data generation rules (also called preset data generation instructions) can be updated using user data, items that can be recommended to users, and items that cannot be recommended to users, thereby obtaining new data generation rules (also called new data generation instructions).

[0143] Specifically, new data generation rules can be obtained in the following ways:

[0144] It is worth noting that the preset data generation rules can include two parts: one part is the rules that do not need to be supplemented (i.e., complete rules), and the other part is the rules to be supplemented (i.e., incomplete rules). The rules to be supplemented have slots that can be filled with content. Then, user data, recommended items, and non-recommended items can be filled into the slots of the rules to be supplemented, thereby obtaining the supplemented rules. In this way, the rules that do not need to be supplemented and the supplemented rules constitute the new data generation rules, wherein the rules that do not need to be supplemented are used to set the format of the dialogue data between the user and the dialogue model (for example, the number of question and answer rounds in the dialogue data, the way the dialogue data is started (i.e., the dialogue data is started in the form of small talk), the language style of the dialogue data, the behavioral norms of the user and the dialogue model in the dialogue data, the dialogue data must include descriptions of items, etc.), and the supplemented rules are used to set the content of the dialogue data between the user and the dialogue model (for example, the dialogue data must take into account the user's personal information, the dialogue data must present the user's recommendation needs, the range of item recommendations that the dialogue model must meet in the dialogue data, etc.).

[0145] Still taking the above example, as shown in FIG8 (FIG. 8 is a schematic diagram of the data generation rule acquisition process provided by the embodiment of the present application, and FIG8 is obtained based on FIG7), after obtaining the recommended movie list and the non-recommended movie list, 13 preset data generation instructions can be obtained, among which data generation instruction (5), data generation instruction (6), data generation instruction (7), and data generation instruction (10) are all incomplete rules, and the remaining data generation instructions are all complete rules. Then, the preset data generation instructions can be used to supplement the incomplete rules using user attribute information, user preference information, the recommended movie list, and the non-recommended movie list, thereby obtaining 13 new data generation instructions:

[0146] (1) The dialogue data must contain 10 rounds of questions and answers, but not more than 20 rounds.

[0147] (2) The dialogue data must start with a small talk between the user (questioner) and the dialogue model (answerer).

[0148] (3) The dialogue model can only recommend one movie to the user in each round and cannot recommend movies to the user outside the recommended movie list.

[0149] (4) In the conversation data, the conversation model must explore the user's taste and interest in movies and adjust the recommendation strategy based on the user's feedback.

[0150] (5) The user is a 22-year-old male user living in Shenzhen. His hobbies include watching movies and listening to music, etc.

[0151] (6) The user’s favorite movies are: Movie 1, Movie 4, Movie 5, Movie 6, Movie 7, Movie 8, Movie 9, etc.

[0152] (7) The movies that the user dislikes are: Movie 2 and Movie 3.

[0153] (8) Users should express their interests gradually and refuse to watch a movie if it does not suit their preferences or if they have already seen it.

[0154] (9) Users can ask for more details, such as actors, directors, plot, or introduction about the movie.

[0155] (10) Recommendable movies: Movie 1, Movie 4, Movie 5, ..., Movie 20; unrecommended movies: Movie 2, Movie 3, Movie 21, ..., Movie 50.

[0156] (11) Movies mentioned in the conversation data should be accompanied by the symbol “<>”.

[0157] (12) All movie-related entities mentioned in the conversation data, such as director, actor, genre, actor, etc., should be enclosed by the symbol “[]”.

[0158] (13) All descriptions or explanations of a film should be enclosed by the symbols “{}”.

[0159] 504. Based on the new data generation rule, generate dialogue data between the user and the dialogue model, where the dialogue data includes items recommended by the dialogue model to the user, and the items recommended by the dialogue model to the user are from the recommended items.

[0160] After obtaining the new data generation rules, they can be input into the dialogue model (the neural network model to be trained). The dialogue model can then generate conversation data between the user and the dialogue model according to the new data generation rules. It should be noted that conversation data between the user and the dialogue model typically consists of multiple rounds of interaction, which include multiple rounds of regular conversation and multiple rounds of question-and-answer sessions. Each regular conversation round consists of a non-question utterance from the user and a response from the dialogue model (or, alternatively, each regular conversation round consists of a non-question utterance from the dialogue model and a response from the user), and each question-and-answer round consists of a question from the user and an answer from the dialogue model (or, alternatively, each question-and-answer round consists of a question from the dialogue model and an answer from the user).

[0161] It's worth noting that in a single round of multi-round Q&A, the user's question can be a recommendation from the dialogue model, and the dialogue model's answer can be the recommended item. The recommended items can be either recommended items (which simulates the user accepting a recommendation) or unrecommended items (which simulates the user rejecting a recommendation). Of course, in a single round of a multi-round conversation, the user's words can be small talk initiated by the user to the dialogue model, and the dialogue model's reply can be its response to the user's small talk.

[0162] In this way, the content of the dialogue data finally generated includes not only the content of the chat between the user and the dialogue model, but also the content of the user accepting the items recommended by the dialogue model, and even the content of the user rejecting the items recommended by the dialogue model, making the finally generated dialogue data closer to reality.

[0163] Still using the above example, as shown in FIG9 ( FIG9 is a schematic diagram of the process of acquiring dialogue data provided in an embodiment of the present application, and FIG9 is drawn based on FIG8 ), after obtaining the new 13 data generation instructions, the new 13 data generation instructions can be input into the dialogue model, so that the dialogue model generates the following dialogue data (including 10 rounds of interaction):

[0164] User: Hi, do you have any movie recommendations for me?

[0165] Dialogue Model: Of course, what kind of movies do you like?

[0166] User: I love science fiction and dystopian movies.

[0167] Dialogue model: How about <Movie 4>? This is a classic [science fiction] movie.

[0168] User: I've already seen that one. Do you have any other recommendations?

[0169] Dialogue Model: How about Movie 21? It's a suspenseful thriller with a great cast.

[0170] User: I'm not a big fan of horror movies. Do you have more sci-fi or dystopian themes?

[0171] Dialogue model: <Movie 15> How is this movie? It's a science fiction movie set on a spaceship.

[0172] User: Sounds interesting! Could you please elaborate?

[0173] Dialogue Model: The protagonists are [G] and [J], {They travel to a new planet on a spaceship. But they wake up from hibernation 90 years early and have to find a way to survive together}.

[0174] User: I seem to have heard of it. I haven’t watched it yet. I will watch it later.

[0175] Dialogue Model: Great! My other recommendation is Movie 1. It's a dystopian film about a teenage deathmatch.

[0176] User: I actually saw that one and I loved it. Do you have any other recommendations?

[0177] Dialogue Model: How about <Movie 35>? This is another [dystopian] movie about {a society where people are divided into factions based on their personality traits}.

[0178] User: I don't really like this type of movie. Do you have any other options?

[0179] Dialogue Model: How about Movie 50? It's the sequel to the classic sci-fi movie Movie 49.

[0180] User: I haven't seen the original Movie 49, so I don't think I'd understand the sequel. Do you have any other ideas?

[0181] Dialogue Model: How about <Movie 30>, which is {a story about a thief who impersonates a warlord}.

[0182] User: That's not my cup of tea. I think I'll stick with Movie 15 for now. Thanks for the recommendation!

[0183] 505. Obtain target dialogue data from the dialogue data. The target dialogue data is used to train the dialogue model to obtain a trained dialogue model.

[0184] After obtaining the conversation data between the user and the conversation model, the conversation data can be input into the data screening model (trained neural network model) to screen out several rounds of interactions from the multiple rounds of interactions contained in the conversation data. These rounds of interactions are the target conversation data. It should be noted that the target conversation data satisfies at least one of the following: (1) All the texts contained in the target conversation data are in a preset text set (all the texts contained in the preset text set are non-violation texts, which can also be understood as non-discriminatory, non-prejudiced, and non-abusive texts). (2) The descriptions of the items contained in the target conversation data are consistent with the real descriptions. (3) The items contained in the target conversation data are all real items. (4) The richness of the content contained in the target conversation data is greater than or equal to a third threshold (the size of the third threshold can be set according to actual needs and is not limited here). The richness of the content contained in the target conversation data can be calculated based on information such as the number of interaction rounds of the target conversation data, the length of the target conversation data, and the number of items contained in the target conversation data.

[0185] Still as in the above example, as shown in Figure 10 (Figure 10 is a schematic diagram of the process of obtaining conversation data provided in an embodiment of the present application, and Figure 10 is drawn based on Figure 9), after obtaining the conversation data, the conversation data can be input into the data screening model. The data screening model includes a violation detection module, an entity authenticity detection module, a description authenticity detection module and a richness detection module. Among them, in the multiple rounds of interactions contained in the dialogue data, the violation detection module can eliminate interactions containing illegal information such as discrimination, prejudice, and insults. The entity authenticity detection module can (with the help of knowledge graphs or information databases that record various information such as real movies, real entities, and the relationship between real movies and real entities, etc.) eliminate interactions containing unreal information such as unreal movies and unreal entities (directors, actors, plots, etc.). The description authenticity detection module can (with the help of public search platforms that record the real descriptions (introductions) of each movie, etc.) eliminate interactions containing unreal information such as unreal movie descriptions. The richness detection module can eliminate interactions with insufficient richness (calculated based on various indicators such as the distinctiveness of the question and answer, the length of the question and answer, the number of rounds of the question and answer, the number of movies and entities included in the question and answer), and the remaining interactions constitute the target dialogue data.

[0186] Furthermore, after obtaining the target conversation data, the target conversation data can be used as training data to train the conversation model, thereby obtaining a trained conversation model (trained neural network model), that is, a conversation model with item recommendation function.

[0187] It should be understood that step 505 is optional. In actual applications, step 505 may not be performed. Instead, the dialogue data obtained in step 504 may be directly used as training data to train the dialogue model and obtain a trained dialogue model.

[0188] In addition, the conversation data generated by the embodiment of the present application can be compared with the conversation data obtained (manually annotated) by the staff of the crowdsourcing project. The comparison results are shown in Table 1:

[0189] Based on Table 1, it can be seen that the quality of the conversation data obtained in the embodiment of the present application is better than the conversation data obtained by manual annotation.

[0190] Furthermore, the conversation data generated by the embodiment of the present application and the conversation data obtained (manually annotated) by the staff of the crowdsourcing project can be scored by four evaluators on four additional indicators: role consistency, fluency, informativeness, and interestingness (the scores of these four indicators are all between 1 and 5, where 1 is the worst and 5 is the best). The scoring results are shown in Figures 11 to 14 (Figure 11 is a schematic diagram of the scoring results provided by the embodiment of the present application, Figure 12 is another schematic diagram of the scoring results provided by the embodiment of the present application, Figure 13 is another schematic diagram of the scoring results provided by the embodiment of the present application, and Figure 14 is another schematic diagram of the scoring results provided by the embodiment of the present application).

[0191] Based on Figures 11 to 14, it can be seen that the dialogue data obtained in the embodiment of the present application scores better than the dialogue data obtained by manual annotation in terms of indicators such as role consistency, fluency, knowledge and fun, that is, the quality of the dialogue data obtained in the embodiment of the present application is better.

[0192] In an embodiment of the present application, when generating conversation data between a user and a conversation model, user data can first be acquired. The user data includes user attributes and items of interest to the user. Next, the user data can be processed to obtain recommended items. Recommended items include items of interest to the user and items related to the items of interest to the user. Pre-set data generation rules can then be updated using the user data and the recommended items, thereby generating new data generation rules. Finally, conversation data between the user and the conversation model can be generated according to the new data generation rules. The conversation data includes items recommended to the user by the conversation model, where the items recommended by the conversation model are selected from the recommended items. The aforementioned process provides a conversation data generation framework that utilizes pre-set data generation rules from a collected set of user data to obtain new data generation rules. The new data generation rules are then used to generate conversation data between the user and the conversation model. This framework can automatically generate conversation data without requiring human intervention, effectively improving the efficiency of conversation data generation and reducing the cost of conversation data generation.

[0193] Furthermore, in the dialogue data generation framework provided in the embodiment of the present application, a neural network model (item collection model, dialogue model, and data screening model) is used to replace manually generated dialogue data. Since the neural network model itself has strong learning and understanding capabilities, it can accurately generate high-quality dialogue data according to data generation rules, which is beneficial to subsequent data cleaning and evaluation, and can enable the dialogue model obtained by subsequent training to have better performance.

[0194] The above is a detailed description of the conversation data generation method provided by the embodiment of the present application. The following is an introduction to the data processing method provided by the embodiment of the present application. Figure 15 is a flow chart of the data processing method provided by the embodiment of the present application. As shown in Figure 15, the method includes:

[0195] 1501. Obtain the user's question, which is used to describe the user's item recommendation needs.

[0196] In this embodiment, when a user has an item recommendation demand, a textual question from the user may be obtained, where the question is used to describe the user's item recommendation demand.

[0197] 1502. Input the user's question into the trained dialogue model to obtain an answer corresponding to the question. The answer is used to describe the item recommendation result for the user.

[0198] After receiving the user's question, the user's question can be input into the trained dialogue model in the embodiment shown in Figure 5, so that the user can be processed through the trained dialogue model to obtain an answer corresponding to the question. The answer is used to describe the item recommendation result for the user, that is, the item recommended to the user.

[0199] The above is a detailed description of the data processing method provided by the embodiment of the present application. The following describes the conversation data generation device and data processing device provided by the embodiment of the present application. Figure 16 is a schematic diagram of the structure of the conversation data generation device provided by the embodiment of the present application. As shown in Figure 16, the device includes:

[0200] Acquisition module 1601, used to acquire user data, which includes user attributes and items of interest to the user;

[0201] A determination module 1602 is configured to determine recommendable items based on the user data, where the recommendable items include items that the user is interested in and items related to the items that the user is interested in;

[0202] An updating module 1603 is configured to update the preset data generation rules based on the user data and the recommended items to obtain new data generation rules;

[0203] The generation module 1604 is used to generate dialogue data between the user and the dialogue model based on the new data generation rule. The dialogue data includes items recommended by the dialogue model to the user. The items recommended by the dialogue model to the user are from the recommended items.

[0204] In an embodiment of the present application, when generating conversation data between a user and a conversation model, user data can first be acquired. The user data includes user attributes and items of interest to the user. Next, the user data can be processed to obtain recommended items. Recommended items include items of interest to the user and items related to the items of interest to the user. Pre-set data generation rules can then be updated using the user data and the recommended items, thereby generating new data generation rules. Finally, conversation data between the user and the conversation model can be generated according to the new data generation rules. The conversation data includes items recommended to the user by the conversation model, where the items recommended by the conversation model are selected from the recommended items. The aforementioned process provides a conversation data generation framework that utilizes pre-set data generation rules from a collected set of user data to obtain new data generation rules. The new data generation rules are then used to generate conversation data between the user and the conversation model. This framework can automatically generate conversation data without requiring human intervention, effectively improving the efficiency of conversation data generation and reducing the cost of conversation data generation.

[0205] In one possible implementation, the determination module is further used to determine non-recommended items based on user data; the update module is used to update the preset data generation rules based on the user data, recommendable items and non-recommended items to obtain new data generation rules.

[0206] In one possible implementation, the determination module is configured to: extract features from user data and multiple candidate items to obtain features of the user data and features of the multiple candidate items; calculate the features of the user data and features of the multiple candidate items to obtain a degree of match between the user data and the multiple candidate items; and determine candidate items having a degree of match greater than or equal to a first threshold as recommendable items.

[0207] In one possible implementation, the determination module is configured to determine candidate items having a matching degree less than or equal to a second threshold as non-recommended items, where the second threshold is less than the first threshold.

[0208] In one possible implementation, the preset data generation rules include rules that do not need to be supplemented and rules to be supplemented. The update module is used to: fill user data, recommended items and non-recommended items into the rules to be supplemented to obtain the supplemented rules. The rules that do not need to be supplemented and the supplemented rules constitute new data generation rules. The rules that do not need to be supplemented are used to set the format of the conversation data, and the supplemented rules are used to set the content of the conversation data.

[0209] In one possible implementation, the device also includes: a screening module for obtaining target conversation data from the conversation data, where the target conversation data satisfies at least one of the following: all text contained in the target conversation data is located in a preset text set; the description of the object contained in the target conversation data is consistent with the real description; the object contained in the target conversation data is a real object; the richness of the content contained in the target conversation data is greater than or equal to a third threshold.

[0210] In one possible implementation, the device further includes: a training module, configured to train the dialogue model based on target dialogue data to obtain a trained dialogue model.

[0211] FIG17 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. As shown in FIG17 , the device includes:

[0212] The acquisition module 1701 is used to acquire the user's question, which is used to describe the user's item recommendation needs.

[0213] The processing module 1702 is used to input the user's question into the trained dialogue model to obtain an answer corresponding to the question, and the answer is used to describe the item recommendation result for the user.

[0214] It should be noted that the information interaction, execution process, etc. between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the embodiment of the present application, and no further details will be given here.

[0215] The embodiments of the present application also relate to an execution device, and FIG18 is a structural diagram of the execution device provided by the embodiments of the present application. As shown in FIG18 , the execution device 1800 can be specifically manifested as a mobile phone, a tablet, a laptop computer, a smart wearable device, a server, etc., which are not limited here. Among them, the data processing device described in the embodiment corresponding to FIG17 can be deployed on the execution device 1800 to implement the data processing function in the embodiment corresponding to FIG15 . Specifically, the execution device 1800 includes: a receiver 1801, a transmitter 1802, a processor 1803 and a memory 1804 (wherein the number of processors 1803 in the execution device 1800 can be one or more, and FIG18 takes one processor as an example), wherein the processor 1803 may include an application processor 18031 and a communication processor 18032. In some embodiments of the present application, the receiver 1801, the transmitter 1802, the processor 1803 and the memory 1804 may be connected via a bus or other means.

[0216] Memory 1804 may include read-only memory and random access memory, and provides instructions and data to processor 1803. A portion of memory 1804 may also include non-volatile random access memory (NVRAM). Memory 1804 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0217] Processor 1803 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0218] The methods disclosed in the above embodiments of the present application can be applied to or implemented by processor 1803. Processor 1803 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 1803. The above processor 1803 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1803 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1804, and processor 1803 reads information from memory 1804 and, in conjunction with its hardware, completes the steps of the above method.

[0219] Receiver 1801 can be used to receive input digital or character information and generate signal input related to executing device-related settings and function control. Transmitter 1802 can be used to output digital or character information through the first interface. Transmitter 1802 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack. Transmitter 1802 can also include a display device such as a display screen.

[0220] In an embodiment of the present application, in one case, the processor 1803 is used to obtain an answer corresponding to the user's question through the trained dialogue model in the embodiment corresponding to Figure 15.

[0221] The present application also relates to a training device. FIG19 is a schematic diagram of the structure of the training device provided by the present application. As shown in FIG19 , the training device 1900 is implemented by one or more servers. The training device 1900 may vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1919 (e.g., one or more processors) and memory 1932, and one or more storage media 1930 (e.g., one or more mass storage devices) storing application programs 1942 or data 1944. The memory 1932 and storage medium 1930 may be temporary storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown), each module may include a series of instruction operations in the training device. Furthermore, the CPU 1919 may be configured to communicate with the storage medium 1930 to execute the series of instruction operations in the storage medium 1930 on the training device 1900.

[0222] The training device 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958; or, one or more operating systems 1941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0223] Specifically, the training device can execute the dialogue data generation method in the embodiment corresponding to Figure 5 to obtain dialogue data between the user and the dialogue model, and then train the dialogue model based on the dialogue data to obtain a trained dialogue model.

[0224] An embodiment of the present application also relates to a computer storage medium, which stores a program for signal processing. When the computer storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0225] An embodiment of the present application also relates to a computer program product, which stores instructions that, when executed by a computer, enable the computer to execute the steps executed by the aforementioned execution device, or enable the computer to execute the steps executed by the aforementioned training device.

[0226] The execution device, training device or terminal device provided in the embodiments of the present application can specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute the computer execution instructions stored in the storage unit, so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0227] Specifically, see Figure 20, which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 2000. NPU 2000 is mounted on the host CPU (host CPU) as a coprocessor and is assigned tasks by the host CPU. The core of the NPU is arithmetic circuit 2003, which is controlled by controller 2004 to extract matrix data from memory and perform multiplication operations.

[0228] In some implementations, the arithmetic circuit 2003 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 2003 is a two-dimensional systolic array. The arithmetic circuit 2003 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 2003 is a general-purpose matrix processor.

[0229] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 2002 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 2001 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 2008.

[0230] Unified memory 2006 is used to store input and output data. Weight data is directly transferred to weight memory 2002 through the Direct Memory Access Controller (DMAC) 2005. Input data is also transferred to unified memory 2006 through the DMAC.

[0231] BIU stands for Bus Interface Unit, i.e., bus interface unit 2013 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 2009 .

[0232] The bus interface unit 2013 (BIU) is used for the instruction fetch memory 2009 to obtain instructions from the external memory, and is also used for the storage unit access controller 2005 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0233] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 2006 or transfer weight data to the weight memory 2002 or transfer input data to the input memory 2001.

[0234] The vector calculation unit 2007 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit 2003, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of the predicted label plane.

[0235] In some implementations, the vector calculation unit 2007 can store the processed output vector in the unified memory 2006. For example, the vector calculation unit 2007 can apply a linear function or a nonlinear function to the output of the operation circuit 2003, such as linear interpolation of the predicted label plane extracted by the convolution layer, or another example is a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 2007 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 2003, for example, for use in subsequent layers in a neural network.

[0236] An instruction fetch buffer 2009 connected to the controller 2004 is used to store instructions used by the controller 2004;

[0237] Unified memory 2006, input memory 2001, weight memory 2002, and instruction fetch memory 2009 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0238] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.

[0239] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0240] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0241] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0242] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A method for generating conversation data, characterized in that: The method comprises: Acquire user data, wherein the user data includes attributes of the user and items of interest to the user; Determining recommendable items based on the user data, wherein the recommendable items include items that the user is interested in and items associated with the items that the user is interested in; Based on the user data and the recommendable items, updating a preset data generation rule to obtain a new data generation rule; Based on the new data generation rule, dialogue data between the user and the dialogue model is generated, wherein the dialogue data includes items recommended by the dialogue model to the user, and the items recommended by the dialogue model to the user are from the recommendable items.

2. The method according to claim 1, characterized in that The method further comprises: determining non-recommended items based on the user data; The updating of the preset data generation rules based on the user data and the recommended items to obtain new data generation rules includes: Based on the user data, the recommendable items and the non-recommended items, the preset data generation rules are updated to obtain new data generation rules.

3. The method according to claim 2, characterized in that The determining of the recommendable items based on the user data comprises: Extracting features from the user data and the multiple candidate items to obtain features of the user data and features of the multiple candidate items; Calculating the features of the user data and the features of the multiple candidate items to obtain a matching degree between the user data and the multiple candidate items; The candidate items whose matching degree is greater than or equal to the first threshold are determined as recommendable items.

4. The method according to claim 3, characterized in that The determining of the unrecommended items based on the user data comprises: The candidate items whose matching degree is less than or equal to a second threshold are determined as non-recommended items, where the second threshold is less than the first threshold.

5. The method according to any one of claims 2 to 4, characterized in that: The preset data generation rules include rules that do not need to be supplemented and rules that need to be supplemented. The updating of the preset data generation rules based on the user data, the recommended items and the non-recommended items to obtain new data generation rules includes: The user data, the recommendable items and the non-recommended items are filled into the rules to be supplemented to obtain the supplemented rules. The rules that do not need to be supplemented and the supplemented rules constitute new data generation rules. The rules that do not need to be supplemented are used to set the format of the conversation data, and the supplemented rules are used to set the content of the conversation data.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Target conversation data is obtained from the conversation data, where the target conversation data satisfies at least one of the following: All the characters included in the target dialogue data are in a preset character set; The description of the object contained in the target conversation data conforms to the real description; The objects included in the target conversation data are real objects; The richness of the content contained in the target conversation data is greater than or equal to a third threshold.

7. The method according to claim 6, characterized in that The method further comprises: Based on the target dialogue data, the dialogue model is trained to obtain a trained dialogue model.

8. A conversation data generating device, characterized in that: The device comprises: An acquisition module, used to acquire user data, wherein the user data includes attributes of the user and items of interest to the user; A determination module, configured to determine recommendable items based on the user data, wherein the recommendable items include items that the user is interested in and items associated with the items that the user is interested in; An updating module, configured to update a preset data generation rule based on the user data and the recommended items to obtain a new data generation rule; A generation module is used to generate dialogue data between the user and the dialogue model based on the new data generation rule, wherein the dialogue data includes items recommended by the dialogue model to the user, and the items recommended by the dialogue model to the user are from the recommendable items.

9. The device according to claim 8, characterized in that The determination module is further used to determine non-recommended items based on the user data; The updating module is used to update the preset data generation rules based on the user data, the recommendable items and the non-recommended items to obtain new data generation rules.

10. The device according to claim 9, characterized in that The determining module is used to: Extracting features from the user data and the multiple candidate items to obtain features of the user data and features of the multiple candidate items; Calculating the features of the user data and the features of the multiple candidate items to obtain a matching degree between the user data and the multiple candidate items; The candidate items whose matching degree is greater than or equal to the first threshold are determined as recommendable items.

11. The device according to claim 10, characterized in that The determining module is used to: The candidate items whose matching degree is less than or equal to a second threshold are determined as non-recommended items, where the second threshold is less than the first threshold.

12. The device according to any one of claims 9 to 11, characterized in that The preset data generation rules include rules that do not need to be supplemented and rules that need to be supplemented, and the updating module is used to: The user data, the recommendable items and the non-recommended items are filled into the rules to be supplemented to obtain the supplemented rules. The rules that do not need to be supplemented and the supplemented rules constitute new data generation rules. The rules that do not need to be supplemented are used to set the format of the conversation data, and the supplemented rules are used to set the content of the conversation data.

13. The device according to any one of claims 8 to 12, characterized in that The device also includes: A screening module is used to obtain target conversation data from the conversation data, where the target conversation data satisfies at least one of the following: All the characters included in the target dialogue data are in a preset character set; The description of the object contained in the target conversation data conforms to the real description; The objects included in the target conversation data are real objects; The richness of the content contained in the target conversation data is greater than or equal to a third threshold.

14. The device according to claim 13, characterized in that The device also includes: The training module is used to train the dialogue model based on the target dialogue data to obtain a trained dialogue model.

15. A conversation data generating device, characterized in that: The device includes a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the conversation data generating device executes the method according to any one of claims 1 to 7.

16. A computer storage medium, characterized in that: The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 7.

17. A computer program product, characterized in that The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 7.

Images