A question-answering data generation method and related device

Abstract

The present application discloses a method for generating question-answer data and a related device, which relates to the field of data generation. The method for generating question-answer data includes: obtaining first target data; generating a plurality of questions related to the first target data according to the first target data; combining the plurality of questions with the first target data respectively to obtain a plurality of second target data; using a pre-trained model, based on the plurality of second target data, generating answers corresponding to the plurality of questions respectively, and obtaining a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively. The method for generating question-answer data disclosed in the present application can generate high-quality question-answer pairs.

Description

Technical Field

[0001] The present application relates to the technical field of data generation, and in particular to a question and answer data generation method and related devices. Background Art

[0002] In some application scenarios, it is necessary to generate question-answer pairs for target data (such as images and image-related text).

[0003] With the rapid development of artificial intelligence technology, pre-trained models have been widely used. At present, most solutions for generating question-answer pairs for target data are solutions that directly generate question-answer pairs based on pre-trained models, that is, input the target data into the pre-trained model and let the pre-trained model generate multiple question-answer pairs for the target data.

[0004] The above-mentioned solution of directly generating question-answer pairs based on the pre-trained model is simple and direct, and can quickly generate a large number of question-answer pairs. However, the quality of the generated question-answer pairs is not high. Summary of the invention

[0005] In view of this, the present application provides a question-answer data generation method and related devices to solve the problem that the quality of question-answer pairs generated by the existing question-answer data generation scheme is not high. The technical solution is as follows:

[0006] The first aspect of the present application provides a method for generating question-answer data, comprising:

[0007] Acquiring first target data;

[0008] generating, according to the first target data, a plurality of questions related to the first target data;

[0009] Combining the plurality of questions with the first target data respectively to obtain a plurality of second target data;

[0010] By using the pre-trained model and taking the plurality of second target data as a basis, answers corresponding to the plurality of questions are generated respectively, so as to obtain a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively.

[0011] In a possible implementation, the first target data includes a target text;

[0012] Alternatively, the first target data includes a target image and text related to the target image;

[0013] Alternatively, the first target data includes target audio and text related to the target audio.

[0014] In a possible implementation manner, generating, according to the first target data, a plurality of questions related to the first target data includes:

[0015] In the case where the first target data includes a target image and text related to the target image: the target image and the text related to the target image are encoded respectively to obtain visual features of the target image and text features of the text related to the target image; the visual features of the target image are fused with the text features of the text related to the target image to obtain first fused features; and a number of questions related to the first target data are generated according to the first fused features;

[0016] In the case where the first target data includes target audio and related text of the target audio: the target audio and the related text of the target audio are respectively encoded to obtain auditory features of the target audio and text features of the related text of the target audio; the auditory features of the target audio are fused with the text features of the related text of the target audio to obtain second fused features; and based on the second fused features, a number of questions related to the first target data are generated.

[0017] In a possible implementation, combining the plurality of questions with the first target data respectively to obtain a plurality of second target data includes:

[0018] For each question:

[0019] directly combining the question with the first target data to obtain a second target data;

[0020] Alternatively, data related to the question is extracted from the first target data, and the question is combined with the extracted data to obtain a second target data.

[0021] In a possible implementation, extracting data related to the question from the first target data, and combining the question with the extracted data to obtain second target data includes:

[0022] In the case where the first target data includes a target text, extracting a text segment related to the question from the target text, and concatenating the question with the extracted text segment to obtain a second target data;

[0023] In the case where the first target data includes a target image and text related to the target image, extracting the target image from the first target data, extracting a text segment related to the question from the text related to the target image, splicing the question with the extracted text segment, and forming a second target data with the spliced ​​text and the target image;

[0024] In the case where the first target data includes target audio and related text of the target audio, the target audio is extracted from the first target data, and a text segment related to the question is extracted from the related text of the target audio, and the question is spliced ​​with the extracted text segment to form a second target data with the spliced ​​text and the target audio.

[0025] In a possible implementation, the using of the pre-trained model to generate answers corresponding to the plurality of questions respectively based on the plurality of second target data includes:

[0026] For each second target data:

[0027] The second target data is input into a pre-training model to obtain an answer corresponding to the question in the second target data output by the pre-training model.

[0028] In a possible implementation, the question-answer data generating method further includes:

[0029] Perform quality assessment on each generated question-answer pair according to predefined quality assessment criteria;

[0030] Directly output the question-answer pairs whose quality meets the quality assessment standard;

[0031] Optimize the question-answer pairs whose quality does not meet the quality assessment standard, and output the optimized question-answer pairs.

[0032] A second aspect of the present application provides a question-answer data generating device, comprising: a data acquisition module, a question generating module, a question processing module and an answer generating module;

[0033] The data acquisition module is used to acquire first target data;

[0034] The question generating module is used to generate a plurality of questions related to the first target data according to the first target data;

[0035] The question processing module is used to combine the plurality of questions with the first target data respectively to obtain a plurality of second target data;

[0036] The answer generation module is used to generate answers corresponding to the questions respectively based on the plurality of second target data by using the pre-trained model, and obtain a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively.

[0037] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0038] The memory is used to store computer programs;

[0039] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned question and answer data generation methods.

[0040] In a fourth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned question and answer data generation methods.

[0041] By means of the above technical scheme, the question and answer data generation method provided by the present application first obtains the first target data, then generates several questions related to the first target data according to the first target data, then combines the several questions with the first target data respectively to obtain several second target data, and finally uses the pre-trained model to generate answers corresponding to several questions based on several second target data, and obtains several question and answer pairs consisting of several questions and answers corresponding to several questions respectively. Considering that it is difficult to directly generate questions and answers at the same time, resulting in low quality of generated question and answer pairs, the present application adopts a generation strategy based on question and answer separation (i.e., separating the generation process of question and answer pairs into two independent processes of question generation and answer generation) to generate question and answer pairs. Compared with the generation strategy of directly generating questions and answers at the same time, the generation strategy based on question and answer separation can greatly reduce the difficulty of generating question and answer pairs, and thus can significantly improve the quality of generated question and answer pairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a system architecture involved in this application;

[0044] Figure 2 A schematic diagram of a hardware structure of a terminal provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a hardware structure of a server provided in an embodiment of the present application;

[0046] Figure 4 A flowchart of a method for generating question and answer data provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of the second question generation model provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of the third question generation model provided in an embodiment of the present application;

[0049] Figure 7 A flowchart of another method for generating question and answer data provided in an embodiment of the present application;

[0050] Figure 8 A schematic diagram of the structure of a question and answer data generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] In one possible implementation, Figure 1 As shown, the system architecture involved in the present application may include a terminal 101 and a server 102, and the terminal 101 may interact with the server 102 via a network (wired network or wireless network). The server 102 may include one or more servers ( Figure 1 The example includes a server. The terminal 101 can obtain target data (data based on which question-answer pairs are generated), transmit the obtained target data to the server 102 through the network, and the server 102 uses the question-answer pair generation method provided in the present application to generate a number of question-answer pairs related to the target data based on the target data, and feed back the generated several question-answer pairs to the terminal 101.

[0055] In another possible implementation, the system architecture involved in the present application may include a terminal. The terminal has strong data processing capabilities. The terminal can obtain target data (data based on which question-answer pairs are generated), and use the question-answer pair generation method provided in the present application to generate a number of question-answer pairs related to the target data based on the target data.

[0056] Next, the product form of the above-mentioned terminal is described.

[0057] The above-mentioned terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a robot, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions on this.

[0058] Figure 2 An optional hardware structure diagram of a terminal is shown.

[0059] refer to Figure 2 As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0060] The input unit 230 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 270, and can receive and execute the command sent by the processor 270; the touch signal at least includes the contact point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch screen 231, the input unit 230 can also include other input devices. Specifically, other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0061] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0062] The memory 220 can be used to store instructions and data. The memory 220 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; it provides the processor 270 with hardware, software, and data resources including management of computing and processing equipment, and supports control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0063] The processor 270 is the control center of the terminal. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing instructions stored in the memory 220 and calling data stored in the memory 220, it executes various functions of the terminal and processes data, thereby controlling the terminal as a whole. Optionally, the processor 270 may include one or more processing units; preferably, the processor 270 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 270. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 270 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 220, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0064] Among them, the memory 220 can be used to store software codes related to the question and answer data generation method, the processor 270 can execute the software codes in the memory 220, and can also schedule other units (such as the above-mentioned input unit 230 and display unit 240) to realize corresponding functions.

[0065] The radio frequency unit 210 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 270 for processing; in addition, the designed uplink data is sent to the base station. Generally, the radio frequency unit 210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LowNoiseAmplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 210 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General PacketRadio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0066] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices, and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional, and can be replaced by other communication interfaces, such as a network port.

[0067] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0068] The terminal also includes an external interface 280, which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to communicate with other devices, or to connect a charger to charge the terminal.

[0069] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0070] Next, the product form of the above server is described.

[0071] Figure 3 A structural diagram of the above server is provided, such as Figure 3 As shown, the server may include a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, the memory 304, and the communication interface 303 communicate with each other via the bus 301.

[0072] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0073] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0074] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0075] The memory 304 can be used to store software codes related to the question and answer data generation method. The processor 302 can call the software codes stored in the memory 304, and can also schedule other units to implement corresponding functions.

[0076] The processors in the above-mentioned terminals and servers (for example, processor 270 and processor 302) may be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the processor may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0077] Most of the current question-and-answer data generation solutions are based on pre-trained models to directly generate question-and-answer pairs. Since the pre-trained model needs to generate multiple questions and the answers corresponding to the multiple questions at the same time, and it is difficult for the pre-trained model to generate questions and answers at the same time, the question-and-answer pairs generated by the solution based on the pre-trained model directly are of low quality (for example, the answers are short and incomplete), which is difficult to meet the needs of actual applications.

[0078] In response to the above problems, the inventors of this case conducted research, and the initial idea was to post-process the question-answer pairs generated by the model to improve the quality of the question-answer pairs, for example, by correcting and supplementing the generated answers manually or automatically.

[0079] The inventors of this case studied the above ideas and found that although post-processing the question-answer pairs generated by the model can improve the quality of the question-answer pairs to a certain extent, it adds additional processing steps, has low efficiency, and is difficult to apply on a large scale.

[0080] In view of the many problems with the above ideas, the inventor of this case conducted further research and came up with a solution to generate question-answer pairs based on predefined templates. Specifically, according to the predefined templates, the corresponding content is filled in to generate question-answer pairs. Although the solution to generate question-answer pairs based on predefined templates can ensure the consistency of the structure and format of question-answer pairs, the content of the generated question-answer pairs is relatively fixed and lacks flexibility and diversity.

[0081] In view of the problems existing in the above-mentioned scheme, the inventor of this case continued to conduct research, and finally proposed a question and answer data generation method with better effect through continuous research. Next, the question and answer data generation method provided by this application will be introduced through the following embodiments.

[0082] See also Figure 4 , shows a flow chart of a method for generating question and answer data provided in an embodiment of the present application, and the method for generating question and answer data may include:

[0083] Step S401: Acquire first target data.

[0084] Among them, the first target data is the data based on which the question and answer pairs are generated.

[0085] In some scenarios, it is necessary to obtain question-answer pairs related to text. In view of this, the first target data in this embodiment may include target text.

[0086] In some scenarios, it is necessary to obtain question-answer pairs related to pictures and texts related to the pictures. In view of this, the first target data in this embodiment may include a target picture and texts related to the target picture. It should be noted that the texts related to the target picture provide the background and details required to generate questions. For example, in educational applications, the texts related to the target picture may be a text describing a historical event or a scientific phenomenon, and the target picture shows a moment of the historical event or a scientific phenomenon.

[0087] In some scenarios, it is necessary to obtain question-answer pairs related to audio and audio-related text. In view of this, the first target data in this embodiment may include target audio and target audio-related text. The target audio-related text is text related to the audio content of the target audio. Exemplarily, if the target audio is a reading audio of a classical Chinese article, the target audio-related text may be a translation of the classical Chinese article, a background introduction, etc.

[0088] Step S402: Generate a number of questions related to the first target data according to the first target data.

[0089] Taking into account that it is difficult to directly generate questions and answers at the same time, resulting in low quality of the generated question-answer pairs, the embodiment of the present application proposes a generation strategy based on question-answer separation, that is, separating the generation process of question-answer pairs into two independent processes of question generation and answer generation, that is, first generating several questions related to the first target data based on the first target data, and then generating answers corresponding to the several questions respectively.

[0090] The above content mentioned that the first target data may include a target text. If the first target data includes the target text, a number of questions related to the target text are generated according to the target text.

[0091] The above content mentioned that the first target data may include a target picture and related text of the target picture. If the first target data includes a target picture and related text of the target picture, several questions related to the target picture and related text of the target picture are generated based on the target picture and related text of the target picture.

[0092] The above content mentioned that the first target data may include target audio and related text of the target audio. If the first target data includes target audio and related text of the target audio, several questions related to the target audio and related text of the target audio are generated based on the target audio and related text of the target audio.

[0093] Step S403: Combine a plurality of questions with the first target data respectively to obtain a plurality of second target data.

[0094] There are many ways to combine several questions with the first target data respectively. In one possible implementation, for each question, the question can be directly combined with the first target data. Considering that sometimes the first target data may contain some noise (for example, text data that is irrelevant or not highly relevant to the generated question), in order to avoid interference of noise on subsequent answer generation, in another possible implementation, for each question, data related to the question can be first extracted from the first target data, and then the question can be combined with the extracted data.

[0095] Step S404: using the pre-trained model and taking a number of second target data as a basis, generating answers corresponding to a number of questions, and obtaining a number of question-answer pairs consisting of a number of questions and answers corresponding to the number of questions.

[0096] The pre-trained model in this embodiment can be a deep learning model trained based on a large-scale data set. In the case where the second target data includes data of multiple modalities, the pre-trained model can be a multimodal pre-trained model, which can process multimodal data through a cross-modal attention mechanism.

[0097] In this embodiment, there are multiple implementation methods for using a pre-trained model to generate answers corresponding to a number of questions based on a number of second target data. This embodiment provides the following two implementation methods.

[0098] The first implementation method: input a plurality of second target data into a pre-training model at the same time, and the pre-training model generates answers corresponding to a plurality of questions according to the input plurality of second target data.

[0099] In order to further improve the quality of generated answers, this embodiment provides a second implementation method: for each second target data, the second target data is input into a pre-trained model, so that the pre-trained model generates an answer corresponding to the question in the second target data based on the input second target data.

[0100] The question and answer data generation method provided in the embodiment of the present application first obtains the first target data, then generates several questions related to the first target data according to the first target data, then combines the several questions with the first target data respectively to obtain several second target data, and finally uses the pre-trained model to generate answers corresponding to several questions respectively based on several second target data, and obtains several question and answer pairs consisting of several questions and answers corresponding to several questions respectively. Considering that it is difficult to directly generate questions and answers at the same time, resulting in low quality of generated question and answer pairs, the embodiment of the present application adopts a generation strategy based on question and answer separation (i.e., separating the generation process of question and answer pairs into two independent processes of question generation and answer generation) to generate question and answer pairs. Compared with the generation strategy of directly generating questions and answers at the same time, the generation strategy based on question and answer separation may greatly reduce the difficulty of generating question and answer pairs, and thus can significantly improve the quality of generated question and answer pairs. In addition, compared with the end-to-end generation method, the question and answer data generation method provided in the embodiment of the present application is more flexible and extensible.

[0101] In another embodiment of the present application, the specific implementation process of "step S402: generating a number of questions related to the first target data according to the first target data" in the above embodiment is introduced.

[0102] The above embodiment mentioned that the first target data may include a target text. In the case where the first target data includes the target text, the process of generating a number of questions related to the first target data based on the first target data may include: encoding the target text to obtain text features of the target text; and generating a number of questions related to the target text based on the text features of the target text.

[0103] In a possible implementation, a number of questions related to the target text may be generated based on the first question generation model and the target text, wherein the first question generation model may be trained using a training text with a number of questions marked. Specifically, the first question generation model may include a text encoder and a text-related question generation module, and the target text may be encoded based on the text encoder of the first question generation model to obtain text features of the target text, and then the text-related question generation module of the first question generation model may generate a number of questions related to the target text based on the text features of the target text.

[0104] The above embodiment mentioned that the first target data may include a target picture and related text of the target picture. When the first target data includes the target picture and the related text of the target picture, the process of generating several questions related to the first target data according to the first target data may include: encoding the target picture and the related text of the target picture respectively to obtain the visual features of the target picture and the text features of the related text of the target picture; fusing the visual features of the target picture with the text features of the related text of the target picture to obtain a first fused feature; and generating several questions related to the target picture and the related text of the target picture according to the first fused feature.

[0105] In a possible implementation, the second question generation model can be based on the target image and the text related to the target image to generate several questions related to the target image and the text related to the target image, wherein the second question generation model can be trained using training samples (here, the training samples include training images and the text related to the training images) with several questions marked. Figure 5 As shown, the second question generation model may include a visual encoder, a text encoder and a multimodal feature fusion and question generation module. The target image may be encoded based on the visual encoder of the second question generation model to obtain the visual features of the target image. The relevant text of the target image may be encoded based on the text encoder of the second question generation model to obtain the text features of the relevant text of the target image. The visual features of the target image may be fused with the text features of the relevant text of the target image based on the multimodal feature fusion and question generation module of the second question generation model, and a number of questions related to the target image and the relevant text of the target image may be generated based on the fused features.

[0106] The above embodiment mentioned that the first target data may include target audio and related text of the target audio. When the first target data includes the target audio and related text of the target audio, the process of generating several questions related to the first target data based on the first target data may include: encoding the target audio and the related text of the target audio respectively to obtain the auditory features of the target audio and the text features of the related text of the target audio; fusing the auditory features of the target audio with the text features of the related text of the target audio to obtain second fused features; and generating several questions related to the target audio and the related text of the target audio based on the second fused features.

[0107] In a possible implementation, the third question generation model can be based on the target audio and the text related to the target audio to generate several questions related to the target audio and the text related to the target audio, wherein the third question generation model can be trained using training samples (the training samples here include the training audio and the text related to the training audio) with several questions marked. Figure 6 As shown, the second question generation model may include an auditory encoder, a text encoder, and a multimodal feature fusion and question generation module. The auditory encoder of the third question generation model may encode the target audio to obtain the auditory features of the target audio. The text encoder of the third question generation model may encode the relevant text of the target audio to obtain the text features of the relevant text of the target audio. The multimodal feature fusion and question generation module of the third question generation model may fuse the auditory features of the target audio with the text features of the relevant text of the target audio, and generate a number of questions related to the target audio and the relevant text of the target audio based on the fused features.

[0108] Through the above process, several high-quality questions related to the first target data (questions that can accurately reflect the key information of the first target data and have diversity) can be obtained. These questions serve as the basis for subsequent answer generation.

[0109] In another embodiment of the present application, the implementation process of "step S403: combining a plurality of questions with the first target data respectively to obtain a plurality of second target data" in the above embodiment is introduced.

[0110] The above embodiment mentioned that when a plurality of questions are respectively combined with the first target data, each question can be directly combined with the first target data.

[0111] Specifically, when the first target data includes the target text, several questions can be spliced ​​with the target text to obtain several second target data. Exemplarily, three questions are generated through step S402, namely question 1, question 2 and question 3, then question 1 is spliced ​​with the target text to obtain second target data 1, question 2 is spliced ​​with the target text to obtain second target data 2, question 3 is spliced ​​with the target text to obtain second target data 3, and thus three second target data are obtained.

[0112] Specifically, in the case where the first target data includes a target image and text related to the target image, for each question, the question is spliced ​​with the text related to the target image, and the target image and the spliced ​​text constitute the second target data. Exemplarily, three questions are generated through step S402, namely, question 1, question 2, and question 3, then question 1 is spliced ​​with the text related to the target image to obtain spliced ​​text 1, and the spliced ​​text 1 and the target image constitute the second target data 1, question 2 is spliced ​​with the text related to the target image to obtain spliced ​​text 2, and the spliced ​​text 2 and the target image constitute the second target data 2, question 3 is spliced ​​with the text related to the target image to obtain spliced ​​text 3, and the spliced ​​text 3 and the target image constitute the second target data 3, and thus, three second target data are obtained.

[0113] Specifically, in the case where the first target data includes the target audio and the text related to the target audio, for each question, the question is spliced ​​with the text related to the target audio, and the target audio and the spliced ​​text constitute the second target data. Exemplarily, three questions are generated through step S402, namely question 1, question 2 and question 3, then question 1 is spliced ​​with the text related to the target audio to obtain spliced ​​text 1, and the spliced ​​text 1 and the target audio constitute the second target data 1, question 2 is spliced ​​with the text related to the target audio to obtain spliced ​​text 2, and the spliced ​​text 2 and the target audio constitute the second target data 2, question 3 is spliced ​​with the text related to the target audio to obtain spliced ​​text 3, and the spliced ​​text 3 and the target audio constitute the second target data 3, and in this way, three second target data are obtained.

[0114] When combining several questions with the first target data respectively, in addition to the direct combination method described above, the following combination method can also be used: for each question, first extract data related to the question from the first target data, and then combine the question with the extracted data.

[0115] Specifically, in the case where the first target data includes the target text, for each question, a text segment related to the question can be extracted from the target text, and the question can be combined with the extracted text segment to obtain the second target data. Exemplarily, three questions are generated through step S402, namely question 1, question 2 and question 3, then the text segment related to question 1 is extracted from the target text, question 1 is spliced ​​with the extracted text segment (the text segment related to question 1), and the second target data 1 is obtained, the text segment related to question 2 is extracted from the target text, question 2 is spliced ​​with the extracted text segment (the text segment related to question 2), and the second target data 2 is obtained, the text segment related to question 3 is extracted from the target text, question 3 is spliced ​​with the extracted text segment (the text segment related to question 3), and the second target data 3 is obtained.

[0116] Specifically, in the case where the first target data includes a target image and text related to the target image, for each question, the target image is extracted from the first target data, and a text segment related to the question is extracted from the text related to the target image, and the question is spliced ​​with the extracted text segment, and the target image and the spliced ​​text form the second target data. Exemplarily, three questions are generated through step S402, namely, question 1, question 2, and question 3, then the text segment related to question 1 is extracted from the text related to the target image, question 1 is spliced ​​with the extracted text segment (the text segment related to question 1), and the target image and the spliced ​​text form the second target data 1, the text segment related to question 2 is extracted from the text related to the target image, question 2 is spliced ​​with the extracted text segment (the text segment related to question 2), and the target image and the spliced ​​text form the second target data 2, the text segment related to question 3 is extracted from the text related to the target image, question 3 is spliced ​​with the extracted text segment (the text segment related to question 3), and the target image and the spliced ​​text form the second target data 3.

[0117] Specifically, in the case where the first target data includes the target audio and the text related to the target audio, for each question, the target audio is extracted from the first target data, and the text segment related to the question is extracted from the text related to the target audio, and the question is spliced ​​with the extracted text segment, and the target audio and the spliced ​​text form the second target data. Exemplarily, three questions are generated through step S402, namely, question 1, question 2 and question 3, then the text segment related to question 1 is extracted from the text related to the target audio, question 1 is spliced ​​with the extracted text segment (the text segment related to question 1), and the target audio and the spliced ​​text form the second target data 1, the text segment related to question 2 is extracted from the text related to the target audio, question 2 is spliced ​​with the extracted text segment (the text segment related to question 2), and the target audio and the spliced ​​text form the second target data 2, the text segment related to question 3 is extracted from the text related to the target audio, question 3 is spliced ​​with the extracted text segment (the text segment related to question 3), and the target audio and the spliced ​​text form the second target data 3.

[0118] Through the above process, a number of second target data can be obtained. The second target data provides more and more useful information for subsequent answer generation, which helps to generate more detailed and accurate answers.

[0119] In another embodiment of the present application, the process of "Step S404: using a pre-trained model to generate answers corresponding to a number of questions based on a number of second target data, and obtaining a number of question-answer pairs consisting of a number of questions and answers corresponding to the number of questions" in the above embodiment is introduced.

[0120] The process of generating answers corresponding to several questions based on several second target data using a pre-trained model may include: inputting several second target data into the pre-trained model respectively to obtain the answers generated by the pre-trained model for the questions contained in the input second target data.

[0121] In order to generate high-quality answers, each second target data can be used as an independent input, that is, only one second target data is input into the pre-trained model each time, so that the pre-trained model generates an answer for only one question each time. It should be noted that before the second target data is input into the pre-trained model, it can be pre-processed, such as word segmentation, stop word removal, etc., and the pre-processed data is then input into the pre-trained model for answer generation.

[0122] Through the above process, a complete, detailed and accurate answer can be generated (that is, the generated answer can accurately answer the corresponding question and is detailed and complete), and the generated answer is highly relevant to the second target data.

[0123] After generating answers corresponding to several questions, each question and the corresponding answer can be combined into a question-answer pair, so that several question-answer pairs can be obtained.

[0124] In order to further improve the quality of question-answer pairs, in another embodiment of the present application, Figure 7 As shown, the question-answer data generation method may include the following steps in addition to step S401 to step S404:

[0125] Step S405: Perform quality assessment on each generated question-answer pair according to a predefined quality assessment standard.

[0126] The quality assessment criteria may include, but are not limited to, multiple aspects of the assessment criteria, including the accuracy assessment criteria of the answer, the detail assessment criteria of the answer, the relevance assessment criteria of the answer, etc. When performing the quality assessment on each question-answer pair, it is necessary to assess whether each question-answer pair meets the assessment criteria in each aspect.

[0127] Step S406: directly output the question-answer pairs whose quality meets the quality assessment standard, optimize the question-answer pairs whose quality does not meet the quality assessment standard, and output the optimized question-answer pairs.

[0128] Question-answer pairs that meet the quality assessment standards can be directly output, while question-answer pairs that do not meet the quality assessment standards are optimized (such as adding details, correcting errors, etc.) to make them meet the quality assessment standards, and finally the optimized question-answer pairs are output.

[0129] Steps S405 and S406 can ensure that the final output is a question-answer pair that meets the quality evaluation standard, that is, a high-quality question-answer pair.

[0130] The question and answer data generation method provided in the embodiment of the present application reduces the difficulty of generating question and answer data, simplifies the generation process of question and answer data, and improves the quality of question and answer data and the generation efficiency of question and answer data. The question and answer data generated by the question and answer data generation method provided in the embodiment of the present application can be applied to a variety of application scenarios, such as intelligent customer service scenarios, education and training scenarios, knowledge base construction scenarios, etc.

[0131] The above introduces the question and answer data generation method provided in the embodiment of the present application. The following will introduce the device corresponding to the above question and answer data generation method.

[0132] See also Figure 8 , Figure 8 A structural schematic diagram of a question and answer data generating device provided in an embodiment of the present application, the question and answer data generating device may include: a data acquisition module 801, a question generating module 802, a question processing module 803 and an answer generating module 804.

[0133] The data acquisition module 801 is used to acquire first target data.

[0134] The question generating module 802 is used to generate a plurality of questions related to the first target data according to the first target data.

[0135] The question processing module 803 is used to combine the plurality of questions with the first target data respectively to obtain a plurality of second target data.

[0136] The answer generation module 804 is used to use the pre-trained model to generate answers corresponding to the questions respectively based on the plurality of second target data, and obtain a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively.

[0137] In a possible implementation, the first target data includes target text; or, the first target data includes a target image and text related to the target image; or, the first target data includes target audio and text related to the target audio.

[0138] In a possible implementation, when the question generating module 802 generates a plurality of questions related to the first target data according to the first target data, it is specifically configured to:

[0139] In the case where the first target data includes a target image and text related to the target image: the target image and the text related to the target image are encoded respectively to obtain visual features of the target image and text features of the text related to the target image; the visual features of the target image are fused with the text features of the text related to the target image to obtain first fused features; and a number of questions related to the first target data are generated according to the first fused features;

[0140] In the case where the first target data includes target audio and related text of the target audio: the target audio and the related text of the target audio are respectively encoded to obtain auditory features of the target audio and text features of the related text of the target audio; the auditory features of the target audio are fused with the text features of the related text of the target audio to obtain second fused features; and based on the second fused features, a number of questions related to the first target data are generated.

[0141] In a possible implementation, when the question processing module 803 combines the plurality of questions with the first target data respectively to obtain the plurality of second target data, it is specifically configured to:

[0142] For each question:

[0143] directly combining the question with the first target data to obtain a second target data;

[0144] Alternatively, data related to the question is extracted from the first target data, and the question is combined with the extracted data to obtain a second target data.

[0145] In a possible implementation, when the question processing module 803 extracts data related to the question from the first target data and combines the question with the extracted data to obtain a second target data, it is specifically configured to:

[0146] In the case where the first target data includes a target text, extracting a text segment related to the question from the target text, and concatenating the question with the extracted text segment to obtain a second target data;

[0147] In the case where the first target data includes a target image and text related to the target image, extracting the target image from the first target data, extracting a text segment related to the question from the text related to the target image, splicing the question with the extracted text segment, and forming a second target data with the spliced ​​text and the target image;

[0148] In the case where the first target data includes target audio and related text of the target audio, the target audio is extracted from the first target data, and a text segment related to the question is extracted from the related text of the target audio, and the question is spliced ​​with the extracted text segment to form a second target data with the spliced ​​text and the target audio.

[0149] In a possible implementation, when the answer generation module 804 generates answers corresponding to the plurality of questions respectively based on the plurality of second target data using the pre-trained model, it is specifically configured to:

[0150] For each second target data:

[0151] The second target data is input into a pre-training model to obtain an answer corresponding to the question in the second target data output by the pre-training model.

[0152] In a possible implementation, the question-answer data generating device may further include: a question-answer pair post-processing module 805. The question-answer pair post-processing module 805 is used to:

[0153] Perform quality assessment on each generated question-answer pair according to predefined quality assessment criteria;

[0154] Directly output the question-answer pairs whose quality meets the quality assessment standard;

[0155] Optimize the question-answer pairs whose quality does not meet the quality assessment standard, and output the optimized question-answer pairs.

[0156] The question-and-answer data generating device provided in the embodiment of the present application first obtains the first target data, then generates several questions related to the first target data according to the first target data, then combines the several questions with the first target data respectively to obtain several second target data, and finally uses the pre-trained model to generate answers corresponding to several questions respectively based on the several second target data, and obtains several question-and-answer pairs consisting of several questions and answers corresponding to several questions respectively. Considering that it is difficult to directly generate questions and answers at the same time, resulting in low quality of generated question-and-answer pairs, the question-and-answer data generating device provided in the embodiment of the present application adopts a generation strategy based on question-and-answer separation (i.e., separating the generation process of question-and-answer pairs into two independent processes of question generation and answer generation) to generate question-and-answer pairs. Compared with the generation strategy of directly generating questions and answers at the same time, the generation strategy based on question-and-answer separation may greatly reduce the difficulty of generating question-and-answer pairs, and thus can significantly improve the quality of question-and-answer pairs. In addition, the question and answer data generation device provided in the embodiment of the present application includes a question generation module, a question processing module, an answer generation module and a question and answer pair quality assessment module. The above modules are relatively independent of each other and at the same time cooperate with each other. Compared with the end-to-end generation device, the question and answer data generation device provided in the embodiment of the present application is more flexible and scalable, ensuring the reliability and practicality of the device in different application scenarios.

[0157] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0158] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0159] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0160] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0161] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the question and answer data generation method provided in the above embodiment.

[0162] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the question and answer data generation method provided in the above embodiment.

[0163] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the question and answer data generation method provided in the above embodiment.

[0164] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0165] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0166] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0167] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for generating question and answer data, characterized in that: include: Acquiring first target data; generating, according to the first target data, a plurality of questions related to the first target data; Combining the plurality of questions with the first target data respectively to obtain a plurality of second target data; By using the pre-trained model and taking the plurality of second target data as a basis, answers corresponding to the plurality of questions are generated respectively, so as to obtain a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively.

2. The question-answer data generating method according to claim 1, characterized in that: The first target data includes a target text; Alternatively, the first target data includes a target image and text related to the target image; Alternatively, the first target data includes target audio and text related to the target audio.

3. The question-answer data generating method according to claim 2, characterized in that: The step of generating a plurality of questions related to the first target data according to the first target data includes: In the case where the first target data includes a target image and text related to the target image: the target image and the text related to the target image are encoded respectively to obtain visual features of the target image and text features of the text related to the target image; the visual features of the target image are fused with the text features of the text related to the target image to obtain first fused features; and a number of questions related to the first target data are generated according to the first fused features; In the case where the first target data includes target audio and related text of the target audio: the target audio and the related text of the target audio are respectively encoded to obtain auditory features of the target audio and text features of the related text of the target audio; the auditory features of the target audio are fused with the text features of the related text of the target audio to obtain second fused features; and based on the second fused features, a number of questions related to the first target data are generated.

4. The question-answer data generating method according to claim 1, characterized in that: The combining the plurality of questions with the first target data to obtain a plurality of second target data comprises: For each question: directly combining the question with the first target data to obtain a second target data; Alternatively, data related to the question is extracted from the first target data, and the question is combined with the extracted data to obtain a second target data.

5. The question-answer data generating method according to claim 4, characterized in that: The step of extracting data related to the problem from the first target data and combining the problem with the extracted data to obtain a second target data comprises: In the case where the first target data includes a target text, extracting a text segment related to the question from the target text, and concatenating the question with the extracted text segment to obtain a second target data; In the case where the first target data includes a target image and text related to the target image, extracting the target image from the first target data, extracting a text segment related to the question from the text related to the target image, splicing the question with the extracted text segment, and forming a second target data with the spliced ​​text and the target image; In the case where the first target data includes target audio and related text of the target audio, the target audio is extracted from the first target data, and a text segment related to the question is extracted from the related text of the target audio, and the question is spliced ​​with the extracted text segment to form a second target data with the spliced ​​text and the target audio.

6. The question-answer data generating method according to claim 1, characterized in that: The using of the pre-trained model to generate answers corresponding to the plurality of questions respectively based on the plurality of second target data includes: For each second target data: The second target data is input into a pre-training model to obtain an answer corresponding to the question in the second target data output by the pre-training model.

7. The question-answer data generating method according to any one of claims 1 to 6, characterized in that: Also includes: Perform quality assessment on each generated question-answer pair according to predefined quality assessment criteria; Directly output the question-answer pairs whose quality meets the quality assessment standard; Optimize the question-answer pairs whose quality does not meet the quality assessment standard, and output the optimized question-answer pairs.

8. A question-answer data generating device, characterized in that: include: Data acquisition module, question generation module, question processing module and answer generation module; The data acquisition module is used to acquire first target data; The question generating module is used to generate a plurality of questions related to the first target data according to the first target data; The question processing module is used to combine the plurality of questions with the first target data respectively to obtain a plurality of second target data; The answer generation module is used to generate answers corresponding to the questions respectively based on the plurality of second target data by using the pre-trained model, and obtain a plurality of question-answer pairs consisting of the plurality of questions and the answers corresponding to the plurality of questions respectively.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the question and answer data generating method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the question and answer data generating method as described in any one of claims 1 to 7.

Images