Data question answering method, data question answering model training method, device and equipment

Abstract

The present application relates to a data question-and-answer method, a data question-and-answer model training method, a device and equipment, and relates to the field of natural language processing technology. The method includes: obtaining a target question for target multimedia data; determining a target answer type corresponding to the target question based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type; in the case where the target answer type is a reference data query type, determining the target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and determining the answer data corresponding to the target question based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data. The use of this method can simplify the data question-and-answer processing flow and improve the accuracy of data question-and-answer.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a data question and answer method, a data question and answer model training method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the improvement of social level, the subject matter of multimedia videos is becoming more and more abundant, and the requirements for the text quality of multimedia videos are also becoming higher and higher.

[0003] Taking long videos as an example, in traditional technology, in order to shoot high-quality multimedia videos, a large language model is usually used to understand the content of massive candidate data in the form of questions and answers, so as to quickly evaluate the text quality of each candidate data and then determine the optimal video material.

[0004] However, the process of processing candidate data by large language models in traditional technologies is relatively complicated, and there is a problem of low data question-answering accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide a data question and answer method, a data question and answer model training method, an apparatus, a computer device, a storage medium and a computer program product that can improve the accuracy of data question and answer in response to the above technical problems.

[0006] This application provides a data question-answering method. The method comprises:

[0007] Obtaining target questions for target multimedia data;

[0008] Determining a target answer type corresponding to the target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type;

[0009] In the case where the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0010] The present application also provides a data question-answering device. The device comprises:

[0011] A question acquisition module, used for acquiring target questions for target multimedia data;

[0012] A type determination module, configured to determine a target answer type corresponding to the target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type;

[0013] A content determination module is used to determine the target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data when the target answer type is a reference data query type, and to determine the answer data corresponding to the target question based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0014] The present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above data question-and-answer method when executing the computer program.

[0015] The present application also provides a computer-readable storage medium, wherein a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned data question-and-answer method are implemented.

[0016] The present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the data question-and-answer method are implemented.

[0017] The data question-answering method, device, computer equipment, storage medium and computer program product described above obtain a target question for target multimedia data; determine the target answer type corresponding to the target question based on the logical relationship between the question text feature represented by the answer type prompt information corresponding to the target multimedia data and the answer type; when the target answer type is a reference data query type, determine the target reference data of the target question based on the logical relationship between the question text feature represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and determine the answer data corresponding to the target question based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data. In this way, when processing the target question, first determine the target answer type based on the logical relationship between the question text feature and the answer type, preliminarily clarify the method for obtaining the answer content corresponding to the target question, and improve the speed of obtaining the answer content. Then, when the target answer type is a reference data query type, determine the target reference data based on the logical relationship between the question text feature and the reference data, further clarify the search scope of the answer content corresponding to the target question, and improve the search speed of the answer content. Finally, quickly search for the key information corresponding to the target question based on the target reference data, and use the key information as the answer data corresponding to the target question. This is equivalent to using a task decomposition strategy to decompose the processing task into an answer type acquisition task and a reference data search task when faced with the target problem processing task, thereby simplifying the complex problem processing task and facilitating the server to select a suitable answer data acquisition method based on the question text features of the target problem, thereby accurately finding the answer data for the target problem and improving the accuracy of question and answering.

[0018] The present application provides a data question-answering model training method. The method comprises:

[0019] Acquire a data question and answer dataset; the data question and answer dataset includes a question type dataset and a question reference information dataset; the question type dataset includes questions and answer types matching the questions; the question reference information dataset includes questions and reference data matching the questions;

[0020] According to the data question and answer data set, an initial data question and answer model is trained to obtain a corresponding target data question and answer model;

[0021] The target data question-and-answer model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type. When the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0022] The present application also provides a data question-answering model training device. The device comprises:

[0023] A data set acquisition module is used to acquire a data question and answer data set; the data question and answer data set includes a question type data set and a question reference information data set; the question type data set includes questions and answer types matching the questions; the question reference information data set includes questions and answer data matching the questions;

[0024] A model training module, used to train the initial data question answering model according to the data question answering data set to obtain a corresponding target data question answering model;

[0025] The target data question-and-answer model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type. When the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0026] The present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps described in the above-mentioned data question-answering model training method when executing the computer program.

[0027] The present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps described in the above-mentioned data question-answering model training method when executed by a processor.

[0028] The present application also provides a computer program product, which includes a computer program that implements the steps of the data question-answering model training method when executed by a processor.

[0029] The data question-answering model training method, device, computer equipment, storage medium and computer program product obtain a data question-answering data set; the data question-answering data set includes a question type data set and a question reference information data set; the question type data set includes questions and answer types matching the questions; the question reference information data set includes questions and reference data matching the questions; according to the data question-answering data set, the initial data question-answering model is trained to obtain a corresponding target data question-answering model; the target data question-answering model is used to determine the target answer type corresponding to the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type, and when the target answer type is the reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data. In this way, when processing the target question, the target answer type is first determined based on the logical relationship between the question text features and the answer type, and the method for obtaining the answer content corresponding to the target question is preliminarily clarified, thereby improving the speed of obtaining the answer content. Then, when the target answer type is a reference data query type, the target reference data is determined based on the logical relationship between the question text features and the reference data, and the search scope of the answer content corresponding to the target question is further clarified to improve the search speed of the answer content. Finally, the key information corresponding to the target question is quickly searched based on the target reference data, and the key information is used as the answer data corresponding to the target question. This is equivalent to using a task decomposition strategy to decompose the processing task into an answer type acquisition task and a reference data search task when facing the target question processing task, simplifying the complex question processing task so that the server can select the appropriate answer data acquisition method based on the question text features of the target question, and then accurately find the answer data of the target question, thereby improving the accuracy of question and answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a diagram of an application environment of a data question-answering method in one embodiment;

[0031] FIG2 is a schematic diagram of a first flow chart of a data question-and-answer method in one embodiment;

[0032] FIG3 is a flow chart of a data question-and-answer interaction process in one embodiment;

[0033] FIG4 is a schematic diagram of a first process flow of a data question-answering model training method according to an embodiment;

[0034] FIG5 is a schematic diagram of a second process of a data question-answering model training method in one embodiment;

[0035] FIG6 is a schematic flow chart of a step of obtaining a question type data set in one embodiment;

[0036] FIG7 is a flow chart of a step of obtaining a question reference information dataset in one embodiment;

[0037] FIG8 is a schematic flow chart of a problem analysis data set acquisition step in one embodiment;

[0038] FIG9 is a schematic diagram of a flow chart of a data question-answering model prediction method in one embodiment;

[0039] FIG10 is a schematic diagram of a second flow chart of a data question-and-answer method in one embodiment;

[0040] FIG11 is a structural block diagram of a data question-answering device in one embodiment;

[0041] FIG12 is a structural block diagram of a data question-answering model training device in one embodiment;

[0042] FIG13 is a diagram showing the internal structure of a computer device in one embodiment;

[0043] FIG. 14 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The data question-and-answer method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1 . Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, can be integrated on the server 104, or can be placed on the cloud or other servers. The server 104 receives the target question for the target multimedia data sent by the terminal 102, and the server 104 determines the target answer type corresponding to the target question based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type; when the target answer type is a reference data query type, the server 104 determines the target reference data of the target question based on the logical relationship between the question text features represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and determines the answer data corresponding to the target question based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data; the server 104 sends the determined answer data to the terminal 102. The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0046] In one embodiment, as shown in FIG2 , a data question and answer method is provided. The method is described by taking the method applied to the server in FIG1 as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0047] S202: Obtain a target question for target multimedia data.

[0048] The multimedia data is a combination of multiple media, and may include multiple media forms such as text and images. In this embodiment, the target multimedia data includes a large number of details such as storylines, character relationships, theme scenes, etc. The target multimedia data may be a novel, a script, a comic book, etc.

[0049] In practical applications, when evaluating the quality of target multimedia data, in order to ensure the accuracy of the quality evaluation results, it is usually necessary to repeatedly understand the details of the target multimedia data, which consumes a lot of manpower costs. Based on this, the embodiment of the present application adopts the form of human-computer interaction combined with data question and answer, and the server obtains the answer result based on the target question raised by the user for the target multimedia data, so as to help the user better understand and evaluate the target multimedia data, and then obtain the quality evaluation result of the target multimedia data.

[0050] For example, the target questions for the target multimedia data can be questions related to the storyline, such as "What is the content of the first chapter", etc., or questions related to character relationships, such as "Who are the main characters", etc., or questions related to theme scenes, such as "What is the main clue throughout the text", etc.

[0051] In a feasible scenario, the server receives a target question of the target multimedia through a terminal.

[0052] Exemplarily, the server receives a data question and answer request sent by the terminal, the data question and answer request carries a target question of the target multimedia data, and the server parses the data question and answer request to obtain the target question carried in the data question and answer request.

[0053] Exemplarily, a question library is pre-stored on the server. The server receives a data question and answer request sent by the terminal. The data question and answer request carries a question identifier of a target question. The server queries the question library for a target question corresponding to the question identifier based on the question identifier carried in the data question and answer request.

[0054] S204, determining a target answer type corresponding to the target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type.

[0055] The target answer type refers to the method adopted by the server to obtain the answer content corresponding to the target question, including but not limited to: reference data query type, semantic similarity matching type and rejection response type. The reference data query type refers to the method of using database query to determine the answer data of the target question from the database; the semantic similarity matching type refers to the method of using similarity matching to determine the answer data of the target question from the entire text content of the target multimedia data; the rejection response type refers to the inability to obtain the answer data of the target question.

[0056] The answer type prompt information represents the logical relationship between the question text features and the answer type, and may include mapping relationships between multiple sets of question text features and answer types. It should be noted that the answer type in the answer type prompt information is a one-to-many mapping relationship between the question text features, that is, the answer types corresponding to different question text features may be the same.

[0057] When obtaining the target question, the server extracts the question text features of the target question, determines the target logical relationship that matches the question text features of the target question from the logical relationship between the question text features represented by the answer type prompt information and the answer type, and determines the answer type in the target logical relationship as the target answer type.

[0058] Optionally, the data question and answer model can embed answer type prompt information internally. The server inputs the target question into the data question and answer model, and the data question and answer model performs inference analysis on the text features of the target question based on the internally embedded answer type prompt information, and outputs a target answer type that matches the target question.

[0059] Optionally, the server inputs the target question and answer type prompt information into the data question and answer model, parses the target question and answer type prompt information through the data question and answer model, extracts the question text features of the target question, and then the data question and answer model determines the target logical relationship corresponding to the question text features of the target question from the answer type prompt information, and outputs the answer type in the target logical relationship. The server determines the answer type output by the data question and answer model as the target answer type.

[0060] S206, when the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data.

[0061] The target reference data is the identifier of the answer content corresponding to the target question in the multimedia database, which is used to tell the server how to search the multimedia database for the target question, including but not limited to: event table search, summary table search, and character setting table search. The event table search method refers to disassembling the event table from the target multimedia database and determining the answer data of the target question by parsing the event table; the summary table search method refers to disassembling the summary table from the target multimedia database and determining the answer data of the target question by parsing the summary table; the character setting table search method refers to disassembling the character setting table from the target multimedia database and determining the answer data of the target question by parsing the character setting table.

[0062] The element disassembly prompt information represents the logical relationship between the question text features and the reference data, and may include mapping relationships between multiple sets of question text features and reference data. It should be noted that the mapping relationship between the reference data and the question text features in the element disassembly prompt information is a one-to-many mapping relationship, that is, the reference data corresponding to different question text features may be the same target reference data.

[0063] When obtaining the target answer type and the target answer type is a reference data query type, the server determines the target logical relationship that matches the question text features of the target question from the logical relationship between the question text features represented by the element decomposition prompt information and the reference data based on the question text features of the target question, and determines the reference data in the target logical relationship as the target reference data.

[0064] Optionally, the data question and answer model can embed element decomposition prompt information internally. The server inputs the target question into the data question and answer model, and the data question and answer model performs inference analysis on the question text features of the target question based on the internally embedded element decomposition prompt information, and outputs target reference data that matches the target question.

[0065] After determining the target reference data, the server can query the answer data corresponding to the target reference data from the target multimedia database. For example, the server inputs the target reference data into the answer data acquisition model, and queries the answer data corresponding to the target reference data from the target multimedia database through the answer data acquisition model.

[0066] In another scenario, the server may also send the target reference data to the terminal to instruct the terminal to query answer data corresponding to the target reference data from a target multimedia database according to the target reference data and display the answer data.

[0067] It should be noted that, when the target answer type is other than the reference data query type, the server may return the content of the target answer type to the terminal to instruct the terminal to display the result of the server processing the target question.

[0068] Exemplarily, the target answer type is a rejection reply type, the server returns the rejection reply type to the terminal, and the terminal displays rejection reply type indication information to instruct the user to change the target question;

[0069] Exemplarily, the target answer type is a semantic similarity matching type, the server returns the semantic similarity matching type to the terminal, and the terminal displays the semantic similarity matching type indication information to instruct the terminal to calculate the similarity between the target question and each data content in the target multimedia data, and use the data content with the highest similarity as the answer data for the target question.

[0070] In the embodiment of the present application, a target question for target multimedia data is obtained; based on the logical relationship between the question text feature represented by the answer type prompt information corresponding to the target multimedia data and the answer type, the target answer type corresponding to the target question is determined; in the case where the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text feature represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data. In this way, when processing the target question, the target answer type is first determined according to the logical relationship between the question text feature and the answer type, and the method for obtaining the answer content corresponding to the target question is preliminarily clarified, and the speed of obtaining the answer content is improved. Then, in the case where the target answer type is a reference data query type, the target reference data is determined according to the logical relationship between the question text feature and the reference data, and the search scope of the answer content corresponding to the target question is further clarified, and the search speed of the answer content is improved. Finally, the key information corresponding to the target question is quickly searched according to the target reference data, and the key information is used as the answer data corresponding to the target question. This is equivalent to using a task decomposition strategy to decompose the processing task into an answer type acquisition task and a reference data search task when faced with the target problem processing task, thereby simplifying the complex problem processing task and facilitating the server to select a suitable answer data acquisition method based on the question text features of the target problem, thereby accurately finding the answer data for the target problem and improving the accuracy of question and answering.

[0071] It can be seen from the above embodiments that the data question answering method can be divided into two stages. The first stage is to obtain the target answer type of the target question, and the second stage is to obtain the target reference data of the target question when the target answer type is determined to be the reference data query type, and then determine the answer data. Based on this, the following describes the achievable processes of the above two stages through different embodiments.

[0072] In one embodiment, determining the target answer type corresponding to the target question based on the logical relationship between the question text feature represented by the answer type prompt information corresponding to the target multimedia data and the answer type includes:

[0073] Concatenate the answer type prompt information and the target question to obtain a first question text; input the first question text into a first data question-answering model, extract the target question text features of the target question in the first question text through the first data question-answering model, and determine the target answer type that matches the target question based on the logical relationship between the question text features and the answer type.

[0074] The answer type prompt information is used to guide and stimulate the reasoning process of the first data question-answering model to output the answer type according to the target question, so that the first data question-answering model can output an accurate target answer type. For example, the answer type prompt information sets a virtual role scene, and allows the first data question-answering model to substitute into the scene for creation, and then outputs the method of obtaining the target answer to the target question, that is, the target answer type.

[0075] The first data question-answering model may be a natural language processing model obtained by training and fine-tuning based on a question type dataset and a question analysis dataset, and is used to characterize the logical relationship between the first question text and the answer type, such as a large language model.

[0076] In an embodiment of the present application, before the target question is input into the first data question-answering model, the answer type prompt information and the target question are combined according to a preset splicing method to obtain a first question text.

[0077] In one scenario, the splicing method is that the answer type prompt information is before the target question. In this case, the content of the answer type prompt information and the content of the target question can be kept unchanged, and the target question can be copied and pasted after the answer type prompt information, or the answer type prompt information can be copied and pasted before the target question to form the first question text.

[0078] In another scenario, the splicing method can also be to extract text features of the answer type prompt information and the target question, and then combine the extracted answer type feature information and the target question feature information to obtain the first question text. In this case, the answer type feature information of the answer type prompt information is extracted, and the target question feature information of the target question is extracted, and then the answer type feature information and the target question feature information are combined to construct the first question text.

[0079] When the first question text is obtained, the first question text is input into the first data question and answer model, and semantic recognition is performed through the first question and answer model to determine the content corresponding to the target question and the content corresponding to the answer type prompt information; then, the target question text features of the target question are extracted, and based on the logical relationship between the question text features and the answer type, the answer type that logically corresponds to the target question text features, that is, the target answer type that matches the target question, is determined.

[0080] In an embodiment of the present application, a first question text obtained by concatenating the answer type prompt information and the target question is input into a first data question and answer model, and the target question text features of the target question in the first question text are extracted through the first data question and answer model. Based on the logical relationship between the question text features and the answer type, the target answer type that matches the target question is determined, which is equivalent to integrating the answer type acquisition process of the target question into the first data question and answer model. In this way, the first data question and answer model can be flexibly called to perform data question and answer processing to support fast and accurate acquisition of the target answer type of the target question, thereby improving the convenience of data question and answer while also improving the efficiency of the data question and answer process.

[0081] In the above embodiment, the answer type prompt information and the target question may be spliced ​​by text combination, text replacement, or splicing the answer type prompt information and the target question. Next, a possible implementation method of the first question text acquisition step is described.

[0082] In one embodiment, the answer type prompt information and the target question are concatenated to obtain the first question text, including:

[0083] According to the question supplement prompt information in the answer type prompt information, a supplement area in the answer type prompt information is determined; and the target question is filled into the supplement area to obtain the first question text.

[0084] The answer type prompt information includes a reserved area for the target question, and the question supplement prompt information is marked at a designated position in the reserved area. In this case, the question supplement prompt information in the answer type prompt information is determined through semantic information recognition, and then the area indicated by the question supplement prompt information is determined as the supplement area, and the content of the target question is filled into the supplement area, completing the splicing of the answer type prompt information and the target question to obtain the first question text.

[0085] Exemplarily, in the answer type prompt information, the question supplement prompt information is "The question is as follows: "%s"". In this case, "%s" is determined to be the supplementary area, and "%s" is replaced with the target question, and the answer type prompt information after replacing the target question is determined to be the first question text.

[0086] Taking the target question "What is covered in Chapter 3" as an example, replace "%s" with "What is covered in Chapter 3" to obtain the first question text.

[0087] In an embodiment of the present application, the question supplement prompt information in the answer type prompt information is used to determine the supplement area in the answer type prompt information, and then the target question is filled into the supplement area to obtain the first question text. This splicing method retains all the contents of the answer type prompt information and the target question, so that the first data question and answer model can fully learn the logic of the answer type prompt information, conduct multi-dimensional comparative analysis of the target question, and obtain a more accurate target answer type.

[0088] Next, after obtaining the first question text, the first data question-answering model processes the first question text to obtain an implementation method of the target answer type is described.

[0089] In one embodiment, the target question text features of the target question in the first question text are extracted by the first data question-answering model, and the target answer type matching the target question is determined according to the logical relationship between the question text features and the answer type, including at least one of the following methods:

[0090] If the target question text feature includes at least one of a chapter text feature, a character attribute text feature, and a character role event text feature, determining that the target question text feature matches the reference data query type, and determining that the target answer type is the reference data query type;

[0091] If the target question text feature includes at least one of an event name text feature, a scene attribute text feature, and a non-character attribute word text feature, then determining that the target question text feature matches the semantic similarity matching type, and determining that the target answer type is the semantic similarity matching type;

[0092] If the matching degree between the target question text feature and the multimedia data is less than a preset matching degree threshold, the target answer type is determined to be a rejection reply type.

[0093] Since the first question text is obtained by concatenating the answer type prompt information and the target question, and the answer type prompt information represents the logical relationship between the question text features and the answer type, the first question text also includes the logical relationship between the question text features and the answer type. Based on this, after receiving the first question text, the first data question-answering model parses the first question text, determines the target question and answer type prompt information, and extracts the target question text features of the target question; then, according to the logical relationship between the question text features and the answer type in the answer type prompt information, determines the mapping relationship between multiple groups of question text features and answer types, and infers and judges the target question text features, determines the target mapping relationship corresponding to the target question text features in the above multiple groups of mapping relationships, and determines the answer type in the target mapping relationship as the target answer type.

[0094] The target answer type refers to the method used to obtain the answer content corresponding to the target question, including: reference data query type, semantic similarity matching type and rejection response type.

[0095] Among them, the reference data query type is applicable to the case where the target question involves chapters, character attributes or character role features, as a way to obtain the answer output by the first data question-answering model. In the embodiment of the present application, the question text features corresponding to the reference data query type include but are not limited to: chapter text features, character attribute text features and character role event text features. Next, the question text features corresponding to the reference data query type are illustrated with examples:

[0096] ① Chapter text features, such as specific scenes, specific chapters, obtaining full text content, analyzing the overall plot, obtaining chapter / episode / scene or full text summaries. Common target questions include but are not limited to: what is the content of the third chapter, what is the content of the novel, what kind of mentality changes the protagonist has experienced during the development of the plot, etc.

[0097] ② Character attribute text features, such as gender, protagonist and antagonist, main and supporting roles, role relationships, etc. Common target questions include but are not limited to: which villain roles are there, and the relationship between role A and role B.

[0098] ③Character event text features. Common target questions include but are not limited to: what did character A do, what did character B do, etc.

[0099] In an embodiment of the present application, when the target question text features include any one or more of the above three types of text features, the target answer type is determined to be a reference data query type.

[0100] Among them, the semantic similarity matching type is applicable to the answer acquisition method adopted when the target question involves a specific plot. In the embodiment of the present application, the question text features corresponding to the semantic similarity matching type include but are not limited to: event name text features, scene attribute text features, and non-character attribute word text features. Next, the question text features corresponding to the semantic similarity matching type are illustrated as follows:

[0101] ① Event name text features. Common target questions include but are not limited to: Role A investigates event B.

[0102] ② Scene attribute text features, such as forest, combat power, realm, house size, etc. Common target questions include but are not limited to: How big is character A's room?

[0103] ③ Text features of non-character attribute words, such as non-character entities (jade pendants), non-character-specific nouns (sect names, book titles), etc.

[0104] In an embodiment of the present application, when the target question text features include any one or more of the above three types of text features, the target answer type is determined to be a semantic similarity matching type.

[0105] Among them, the rejection reply type is applicable to the answer acquisition method adopted when the target question text feature is irrelevant to the target video data. In the embodiment of the present application, by calculating the matching degree between the target question text feature and the multimedia data, when the matching degree is less than a preset matching degree threshold, it is determined that the target question is irrelevant to the target video data, and the target answer type is determined to be a rejection reply type.

[0106] In another scenario, the target question text features may also be compared with the text features corresponding to the reference data query type and the text features corresponding to the semantic similarity matching type. When the target question text features do not match either of the above two types, the target answer type of the target question is directly determined to be a rejection response type.

[0107] In an embodiment of the present application, the target question text features of the target question in the first question text are extracted through the first data question-answering model to eliminate redundant features in the target question and improve the effectiveness of the target question text features. Then, based on the logical relationship between the question text features and the answer type, the target answer type that matches the target question is intuitively and quickly determined. This method of obtaining the target answer type is reliable and improves the accuracy of the target answer type.

[0108] Next, when it is determined in the first stage that the target answer type is the reference data query type, the steps for the server to obtain the target reference data in the second stage are explained.

[0109] In one embodiment, determining target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data includes:

[0110] The target question and the element decomposition prompt information are spliced ​​to obtain a second question text; the second question text is input into a second data question-answering model, the target question text features of the target question in the second question text are extracted through the second data question-answering model, and the target reference data matching the target question text features is determined based on the logical relationship between the question text features and the reference data.

[0111] The target reference data is the location of the answer content corresponding to the target question in the multimedia database. The multimedia database includes multiple sets of data content obtained by extracting key information from the target multimedia data, and has the characteristics of simplicity and comprehensiveness.

[0112] The element disassembly prompt information is used to guide and stimulate the reasoning process of the second data question-answering model to output the target reference data according to the target question, so that the second data question-answering model can output accurate target reference data. For example, the element disassembly prompt information sets a virtual role scene, and allows the second data question-answering model to substitute into the scene for creation, and then outputs the target reference data that matches the target question.

[0113] The second data question-answering model may be a natural language processing model obtained by training and fine-tuning based on the question reference information dataset and the question analysis dataset, and is used to characterize the logical relationship between the second question text and the reference data, such as a large language model.

[0114] In an embodiment of the present application, before the target question is input into the second data question-answering model, the element decomposition prompt information and the target question are combined in a preset splicing method to obtain a second question text.

[0115] In one scenario, the splicing method is combined splicing. In this case, the content of the element decomposition prompt information and the content of the target question can be kept unchanged, and the target question and the element decomposition prompt information are associated and combined to form a second question text.

[0116] In another scenario, the splicing method is feature extraction first and then combination splicing. In this case, the element decomposition feature information of the element decomposition prompt information is extracted separately, and the target problem feature information of the target problem is extracted. Then, the element decomposition feature information and the target problem feature information are combined to obtain the second question text.

[0117] When the second question text is obtained, the second question text is input into the second data question-answering model, and semantic recognition is performed through the second question-answering model to determine the content corresponding to the target question and the content corresponding to the element decomposition prompt information; then, the target question text features of the target question are extracted, and based on the logical relationship between the question text features and the reference data, the target reference data that logically corresponds to the target question text features is determined.

[0118] In an embodiment of the present application, a second question text obtained by splicing the target question and the element decomposition prompt information is input into the second data question and answer model, and the target question text features of the target question in the second question text are extracted through the second data question and answer model, and the target reference data matching the target question text features is determined according to the logical relationship between the question text features and the reference data, which is equivalent to integrating the reference data acquisition process of the target question into the second data question and answer model. In this way, the second data question and answer model can be flexibly called to perform data question and answer processing to support fast and accurate acquisition of the target reference data of the target question, which improves the convenience of data question and answer while also improving the efficiency of the data question and answer process.

[0119] In one embodiment, the first data question-answering model and the second data question-answering model are the same target data question-answering model, and the data question-answering method further includes:

[0120] The initial data question and answer model is trained according to the data question and answer dataset to obtain a target data question and answer model; the data question and answer dataset includes a first dataset for training the initial data question and answer model to determine the logical relationship of the answer type according to the question text and a second dataset for training the initial data question and answer model to determine the logical relationship of the reference data according to the question text.

[0121] The data question and answer dataset includes multiple sets of data pairs, each set of data pairs includes a question and the question and answer result corresponding to the question. Each set of data pairs in the first dataset includes a question and the answer type corresponding to the question, and each set of data pairs in the second dataset includes a question and the reference data corresponding to the question.

[0122] In an embodiment of the present application, the data question and answer dataset covers data pairs in the first dataset and the second dataset, and in the data question and answer dataset, multiple groups of data pairs in the first dataset and multiple groups of data pairs in the second dataset can be randomly mixed together in order.

[0123] When training the initial data question and answer model, the questions in multiple data pairs in the data question and answer dataset and the prompt information corresponding to the question and answer results are spliced ​​to obtain the splicing results of each data pair, for example, the question and answer type prompt information of each data pair in the first dataset are spliced, and the question and element decomposition prompt information of each data pair in the second dataset are spliced; then, according to the preset order and batch training size (batch size), the splicing results are input into the initial data question and answer model in batches, the prediction results output by the initial data question and answer model are obtained, and the loss error between the prediction results and the question and answer results of the input questions is calculated, and the parameters in the initial data question and answer model are updated until the training termination conditions are met, such as the loss value converges or the number of training rounds reaches the preset number of rounds (all data in the data question and answer dataset are input into the initial data question and answer model once for one round of training), and the target data question and answer model is obtained.

[0124] According to the training process of the initial data question-answering model, when the data in the first data set is used to train the initial data question-answering model, the focus is on training the logical reasoning ability of the initial data question-answering model to determine the answer type according to the question text, and when the data in the second data set is used to train the initial data question-answering model, the focus is on training the logical reasoning ability of the initial data question-answering model to determine the reference data according to the question text. Therefore, the target data question-answering model obtained by training the initial data question-answering model with the data question-answering dataset constructed with the first data set and the second data set has both the logical reasoning ability to determine the answer type according to the question text and the logical reasoning ability to determine the reference data according to the question text.

[0125] In an embodiment of the present application, based on the first data set in the data question and answer data set, the initial data question and answer model is trained to have logical reasoning capabilities to determine the answer type based on the question text. Based on the first data set in the data question and answer data set, the initial data question and answer model is trained to have logical reasoning capabilities to determine the reference data based on the question text. In this way, the target data question and answer model trained is endowed with diverse logical reasoning capabilities, which greatly enriches the application scenarios of the target data question and answer model.

[0126] Next, a possible implementation method of the second question text acquisition step is described.

[0127] In one embodiment, the target question and the element decomposition prompt information are concatenated to obtain a second question text, including:

[0128] According to the question supplement prompt information in the element decomposition prompt information, a supplement area in the element decomposition prompt information is determined; the target question is filled into the supplement area to obtain a second question text.

[0129] The element disassembly prompt information includes a reserved area for the target question, and the question supplementary prompt information is marked at a designated position in the reserved area. In this case, the question supplementary prompt information in the element disassembly prompt information is determined through semantic information recognition, and then the area indicated by the question supplementary prompt information is determined as the supplementary area, and the content of the target question is filled into the supplementary area, completing the splicing of the element disassembly prompt information and the target question to obtain the second question text.

[0130] Exemplarily, in the element decomposition prompt information, the question supplement prompt information is "The question is as follows: "%s"". In this case, "%s" is determined to be the supplementary area, and "%s" is replaced with the target question, and the element decomposition prompt information after replacing the target question is determined as the second question text.

[0131] Taking the target question "What is covered in Chapter 3" as an example, replace "%s" with "What is covered in Chapter 3" to obtain a schematic diagram of the second question text.

[0132] In an embodiment of the present application, prompt information is supplemented according to the question in the element decomposition prompt information, a supplementary area in the element decomposition prompt information is determined, and then the target question is filled into the supplementary area to obtain a second question text. This splicing method retains all the contents of the element decomposition prompt information and the target question, so that the second data question and answer model can fully learn the logic of the element decomposition prompt information, conduct multi-dimensional comparative analysis of the target question, and obtain more accurate target reference data.

[0133] Next, after obtaining the second question text, the second data question answering model processes the second question text to obtain the target reference data.

[0134] In one embodiment, the element disassembly prompt information includes: at least one of an event table, a summary table, and a character setting table; according to the logical relationship between the question text feature and the reference data, determining the target reference data matching the target question text feature from the reference data includes at least one of the following methods:

[0135] If the target question text feature includes the role name text feature, determining that the text feature of the target question matches the event table, and determining that the target reference data is the event table;

[0136] If the target question text feature includes a numeric range text feature, determining that the text feature of the target question matches the summary table, and determining that the target reference data is the summary table;

[0137] If the target question text feature includes the character attribute text feature, it is determined that the text feature of the target question matches the character setting table, and the target reference data is determined to be the character setting table.

[0138] The reference data is the second data question-answering model, which judges one or more sections in the multimedia database according to the target question, determines the identifier of the answer data of the target question in the multimedia database, and is used to tell the server how to search from the multimedia database for the target question.

[0139] In the embodiment of the present application, the reference data includes an event table, a summary table, and a character setting table.

[0140] Among them, the event table is suitable for use as reference data output by the second data question-answering model when the target question involves text features of specific character names, including chapter / episode (1, 2, 3...), scene (1, 2, 3...), character (xxx), range episode_start (1, 2, 3...), range episode_end (1, 2, 3...), etc.

[0141] Optionally, if the target question text feature includes a name text feature, such as "Zhang Mou", the target reference data is determined to be an event table.

[0142] The summary table is suitable for use as reference data output by the second data question answering model when the target question involves numeric range text features corresponding to chapters, episodes, and scenes, including chapters / episodes (1, 2, 3...), scenes (1, 2, 3...), characters (xxx), ranges episode_start (1, 2, 3...), ranges episode_end (1, 2, 3...), etc.

[0143] Optionally, if the target question text feature includes a numeric range text feature, such as "Chapter 1", the target reference data is determined to be a summary table.

[0144] The character setting table is used as reference data for the output of the second data question-answering model when the target question involves character setting content. It includes five fields: gender (male / female), role ranking (integer type), faction (good / villain), episode / scene, and starting episode number / ending episode number.

[0145] In an embodiment of the present application, if the target question text includes character setting text features, such as male protagonist, female protagonist, important role, supporting role, good guy, and bad guy, then the target reference data is determined to be a character setting table.

[0146] In practical applications, when it is determined that the target question text features include character setting text features, the corresponding role ranking in the character setting table is further determined according to the role level in the character setting text features.

[0147] Taking the character setting table role ranking as an example, 1 is the protagonist, 6 is the important role, and -6 is the supporting role, if the character setting text feature is the male protagonist or the female protagonist, the target reference data is determined to be the role ranking 1 in the character setting table; if the character setting text feature is "a role that appears more frequently, or a key role, or an important role, the target reference data is determined to be the role ranking 6 in the character setting table; if the character setting text feature is a supporting role, the target reference data is determined to be the role ranking -6 in the character setting table.

[0148] In addition, in order to improve the learning ability of the second data question-answering model for the logical relationship represented by the element decomposition prompt information, the element decomposition prompt information may also include example questions and the field decomposition method corresponding to the example questions.

[0149] In an embodiment of the present application, the target question text features of the target question in the second question text are extracted through the second data question-answering model to eliminate redundant features in the target question and improve the effectiveness of the target question text features. Then, based on the logical relationship between the question text features and the reference data, the target reference data matching the target question is intuitively and quickly determined. This method of obtaining target reference data is reliable and improves the accuracy of the target reference data.

[0150] In one embodiment, applied to a terminal, as shown in FIG3 , the data question and answer method further includes:

[0151] S302, displaying a first interface corresponding to the target multimedia data, where the first interface is used to indicate input of a target question.

[0152] The first interface is a visual editing interface for the terminal to interact with the user, and supports the user to input a target question for the target multimedia data through interactive operations. The interactive operation can be a click operation, a keyboard input operation, or a voice operation. The target question can be a text question represented in text form, or an image text question represented in picture form, etc.

[0153] Taking the target question as a text question as an example, the method of obtaining the target question on the first interface is explained: the first interface may include a target question list, and the target question is determined based on a click operation on any target question in the target question list; the first interface may also include a keyboard interface, and the target question for the target multimedia data is determined by collecting the text content input by the user triggering the keyboard interface; the first interface also includes a voice indication button, and by pressing the voice indication button, the target question is input by voice and converted into text.

[0154] Taking the target question text as image text as an example, the method of obtaining the target question on the first interface is explained: the first interface may include a question upload button, and in response to the triggering operation of the question upload button, a list of multiple target questions cached by the terminal is displayed on the first interface, and then any target question in the target question list is clicked to determine the target question.

[0155] S304: Send a target question for the target multimedia data to the server to instruct the server to obtain answer data corresponding to the target question according to the target question.

[0156] In the case of obtaining the target question input on the first interface, the target question is sent to the server to instruct the server to first determine the target answer type corresponding to the target question based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type; in the case where the target answer type is a reference data query type, the target reference data of the target question is determined from the reference data based on the logical relationship between the question text features represented by the prompt information based on the elements corresponding to the target multimedia data and the reference data; and the answer data corresponding to the target question is determined based on the target reference data.

[0157] Exemplarily, the terminal sends a question and answer request carrying a target question to the server, the server concatenates the target question and answer type prompt information to obtain a first question text, inputs the first question text into the data question and answer model, obtains the target answer type for the target question, then determines the content of the target answer type, and when the target answer type is a reference data query type, concatenates the target question and element decomposition prompt information to obtain a second question text, inputs the second question text into the data question and answer model, obtains the target reference data for the target question, then determines the content of the target reference data in the reference data as the target data content, carries the target data content in the question and answer response to the target question, and returns the question and answer response to the terminal.

[0158] S306, receiving answer data returned by the server, and displaying the answer data on the first interface.

[0159] In an embodiment of the present application, the terminal interacts with the user by displaying a first interface corresponding to the target multimedia data, so that the user can flexibly input the target question based on actual needs. The terminal then interacts with the server, sends the target question to the server, and instructs the server to obtain the answer data corresponding to the target question. The terminal displays the answer data on the first interface. Through the interaction between the terminal and the server, the speed at which the terminal responds to the target question submitted by the user is improved, thereby improving the efficiency of data question and answering.

[0160] In one embodiment, as shown in FIG4 , a data question-answering model training method is provided. The method is applied to the server in FIG1 as an example for explanation. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0161] S402, obtaining a data question and answer dataset; the data question and answer dataset includes a question type dataset and a question reference information dataset; the question type dataset includes questions and answer types matching the questions; the question reference information dataset includes questions and reference data matching the questions.

[0162] Among them, the question type dataset includes questions and answer types that match the questions, which is used to train the model to determine the logical relationship of the answer type based on the question text; the question reference information dataset includes questions and answer types that match the questions, which is used to train the model to determine the logical relationship of the reference data based on the question text.

[0163] It should be noted that there is overlap between the questions in the data type dataset and the questions in the question reference information dataset. For example, questions whose answer types in the data type dataset correspond to the reference data query type are determined as questions in the question reference information dataset.

[0164] Exemplarily, the data question answering dataset can be pre-stored in the storage space of the server or stored in the cloud, and as more and more processing tasks for target questions of target multimedia data are added, the data in the data question answering dataset can also be updated periodically or in real time.

[0165] S404, training the initial data question-answering model according to the data question-answering data set to obtain the corresponding target data question-answering model; the target data question-answering model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type, and when the target answer type is a reference data query type, determine the target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and determine the answer data corresponding to the target question based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0166] The order of questions in the data question and answer dataset is randomly shuffled, and then each question in the data question and answer dataset is input into the initial data question and answer model in batches according to the preset batch training size. The question prediction results output by the initial data question and answer model are obtained, and the loss error between the question prediction results and the actual question and answer results corresponding to the input questions is calculated. The parameters in the initial data question and answer model are updated according to the loss error to complete a round of model training. The above training process is continuously repeated until the loss error converges or the number of model training rounds reaches the preset number of rounds, the training is completed, and the target data question and answer model is obtained.

[0167] From the training process of the above initial data question-answering model, it can be seen that when the question type data set in the data question-answering data set is used to train the initial data question-answering model, the focus is on training the logical reasoning ability of the initial data question-answering model to determine the answer type according to the question text, and when the question reference information data set in the data question-answering data set is used to train the initial data question-answering model, the focus is on training the logical reasoning ability of the initial data question-answering model to determine the reference data according to the question text. Therefore, when the initial data question-answering model is trained according to the question type data set and the question reference information data set, the target data question-answering model obtained in this way has the logical reasoning ability to determine the answer type according to the question text and to determine the reference data according to the question text.

[0168] Next, the application scenario of the target data question and answer model is explained: the server calls the answer prediction instruction to call the target data question and answer model, instructing the target data question and answer model to perform the answer type prediction step: based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type, determine the target answer type corresponding to the target question of the target multimedia data; the server determines the target answer type output by the target data question and answer model, and when the target answer type is the reference data query type, calls the reference data prediction instruction to call the target data question and answer model to instruct the target data question and answer model to perform the reference data prediction step: based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, determine the target reference data of the target question; finally, the server determines the answer data corresponding to the target question based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data.

[0169] In the embodiment of the present application, a data question and answer data set is obtained; the data question and answer data set includes a question type data set and a question reference information data set; the question type data set includes questions and answer types matching the questions; the question reference information data set includes questions and reference data matching the questions; according to the data question and answer data set, an initial data question and answer model is trained to obtain a corresponding target data question and answer model; the target data question and answer model is used to determine the target answer type corresponding to the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type, and in the case where the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data. In this way, when processing the target question, the target answer type is first determined based on the logical relationship between the question text features and the answer type, and the method for obtaining the answer content corresponding to the target question is preliminarily clarified, thereby improving the speed of obtaining the answer content. Then, when the target answer type is a reference data query type, the target reference data is determined based on the logical relationship between the question text features and the reference data, and the search scope of the answer content corresponding to the target question is further clarified to improve the search speed of the answer content. Finally, the key information corresponding to the target question is quickly searched based on the target reference data, and the key information is used as the answer data corresponding to the target question. This is equivalent to using a task decomposition strategy to decompose the processing task into an answer type acquisition task and a reference data search task when facing the target question processing task, simplifying the complex question processing task so that the server can select the appropriate answer data acquisition method based on the question text features of the target question, and then accurately find the answer data of the target question, thereby improving the accuracy of question and answer.

[0170] As a supervised learning data processing model, in the training process of the target data question answering model, the quality of the data question answering dataset is crucial to the output accuracy of the target data question answering model. Based on this, the following describes the steps for obtaining the dataset.

[0171] In one embodiment, the process of obtaining the question type dataset includes:

[0172] The target multimedia data is segmented to obtain a plurality of sub-target multimedia data; each sub-target multimedia data is input into a question generation model respectively, and a question is constructed for each sub-target multimedia data based on a plurality of question construction prompt words by the question generation model to obtain a plurality of first candidate questions for each sub-target multimedia data; each first candidate question and answer type prompt information are spliced, and the spliced ​​results are input into a plurality of large language models in parallel to obtain a plurality of candidate answer types corresponding to each first candidate question; according to the plurality of candidate answer types corresponding to each first candidate question, a first target question that meets a first preset condition is screened out from the plurality of first candidate questions, and an answer type that matches each first target question is determined from the plurality of candidate answer types corresponding to each first target question; each first target question and the answer type that matches each first target question are summarized to obtain a question type data set.

[0173] Taking the target multimedia data as a script as an example, the target multimedia data is segmented according to any segmentation method including single episode, single chapter, single scene, multiple consecutive episodes, multiple consecutive chapters, and multiple consecutive scenes to obtain multiple sub-target multimedia data.

[0174] For each sub-target multimedia data, the sub-target multimedia data and multiple question construction prompt words are input into the question generation model, and questions are constructed for each sub-target multimedia data based on the multiple question construction prompt words through the question generation model, taking into account multiple dimensions such as text structure, storyline, events, details, character relationships, and theme ideas, to ensure that the generated multiple first candidate questions can fully cover actual application scenarios, such as character analysis, plot prediction, theme discussion, language style recognition, etc.

[0175] For any first candidate question corresponding to the above-mentioned sub-target multimedia data, the first candidate question and the answer type prompt information are spliced, and the spliced ​​result is copied into k copies and input into k large language models in parallel to instruct each large language model to output the candidate answer type of the first candidate question, and k candidate answer types corresponding to the first candidate question are obtained. Then, it is determined whether the k candidate answer types meet the first preset condition, and if the third preset condition is met, the first candidate question is determined as the first target question, and the answer type matching the first target question is determined from the k candidate answer types of the first candidate question.

[0176] For example, the first preset condition may be that the candidate answer type with the highest frequency among the k candidate answer types is not less than n times. Based on this, the frequency of occurrence of different types of candidate answer types among the k candidate answer types can be counted, and the maximum frequency f kmax Compared with n, if f kmax>n, determine the first candidate problem as the first target problem, and then set the maximum frequency f kmax The corresponding candidate answer type is determined to be the answer type that matches the first target question. kmax <n, the first candidate question is determined to be invalid, and the first candidate question is no longer aggregated into the question type data set.

[0177] The above steps are repeated to select at least one first target question from the multiple first candidate questions, and an answer type matching each first target question is determined from the multiple candidate answer types corresponding to each first target question.

[0178] Each first target question and the answer type matching each first target question are combined to obtain multiple groups of first target question answer data pairs, and the multiple groups of first target question answer data pairs are summarized to construct a question type data set.

[0179] In an embodiment of the present application, based on the disassembled sub-target multimedia data, candidate questions of different granularities and categories are generated through a question generation model, covering various application scenarios, so as to improve the diversity and depth of the questions. Furthermore, candidate answer types for each first candidate question are automatically generated through multiple large language models, and verification and judgment are performed, which improves the adaptability of the answer type to the question while greatly reducing the cost of manual annotation. In summary, the question type data set constructed in this embodiment includes a large number of rich and accurate first target questions and answer types. The initial data question model training is performed using this question type data set, which not only trains the initial data question model to understand the surface information of the text, but also trains the initial data questions to perform in-depth reasoning and analysis, thereby improving the accuracy of the data question and answer results.

[0180] In one embodiment, according to the multiple candidate answer types corresponding to the first candidate questions, the first target questions that meet the first preset condition are screened out from the multiple first candidate questions, and the answer types that match the first target questions are determined from the multiple candidate answer types corresponding to the first target questions, including:

[0181] For any first candidate question, determine the quantitative proportions of different types of candidate answer types in the multiple candidate answer types corresponding to the first candidate question; if the maximum value among the quantitative proportions is not less than the first proportion threshold, determine that the first candidate question is the first target question, and determine the candidate answer type corresponding to the maximum value among the quantitative proportions as the answer type that matches the first target question.

[0182] For any first candidate question, the corresponding candidate answer type is a reference data query type, a semantic similarity matching type, or a rejection reply type. Based on this, the proportion of the number of candidate answer types that are reference data query types in the multiple candidate answer types corresponding to the first candidate question, the proportion of the number of candidate answer types that are semantic similarity matching types in the multiple candidate answer types corresponding to the first candidate question, and the proportion of the number of candidate answer types that are rejection reply types in the multiple candidate answer types corresponding to the first candidate question are calculated respectively, and the quantity proportion values ​​of the three candidate answer types corresponding to the first candidate question are obtained. The three quantity proportion values ​​are compared to obtain the maximum quantity proportion value.

[0183] Furthermore, the maximum value and the first ratio threshold are compared; if the maximum value is less than the first ratio threshold, the first candidate question is determined to be invalid and the first candidate question is discarded; if the maximum value is not less than the first ratio threshold, the first candidate question is determined to be the first target question, and from the multiple candidate answer types corresponding to the first target question, the candidate answer type corresponding to the largest number proportion value is determined as the answer type matching the first target question; finally, the first target question and the answer type matching the first target question are combined into a group of question type data pairs, which are summarized into a question type data set.

[0184] Taking the first candidate question corresponding to k candidate answer types, k1 candidate answer type is the reference data query type, the number ratio is k1 / k, k2 candidate answer type is the semantic similarity matching type, the number ratio is k2 / k, k3 candidate answer type is the rejection reply type, the number ratio is k3 / k as an example, the method of determining a set of question type data pairs in the question type dataset is explained:

[0185] If k1 / k is the maximum value of the three quantity proportion values ​​and is greater than the first proportion threshold, the first candidate question is determined as the first target question, the reference data query type is used as the answer type matching the first target question, and the first target question and the reference data query type are combined to obtain a set of question type data pairs; if k2 / k is the maximum value of the three quantity proportion values ​​and is greater than the first proportion threshold, the first candidate question is determined as the first target question, the semantic similarity matching type is used as the answer type matching the first target question, and the first target question and the semantic similarity matching type are combined to obtain a set of question type data pairs; if k3 / k is the maximum value of the three quantity proportion values ​​and is greater than the first proportion threshold, the first candidate question is determined as the first target question, the rejection reply type is used as the answer type matching the first target question, and the first target question and the rejection reply type are combined to obtain a set of question type data pairs.

[0186] In an embodiment of the present application, for multiple candidate answer types for the first candidate question, by comparing the quantitative proportions of different types of candidate answer types and a first proportion threshold, when the first proportion threshold is met, the first candidate question is determined to be the first target question, and the candidate answer type corresponding to the largest quantitative proportion is determined as the answer type that matches the first target question, thereby ensuring the validity of the first target question and the matching degree between the first target question and the answer type, and improving the validity of the question type data set.

[0187] The above embodiment describes the steps for obtaining the question type dataset. Next, the steps for obtaining the question reference information dataset are described.

[0188] In one embodiment, the process of obtaining the question reference information dataset includes:

[0189] From multiple first target questions in the question type data set, determine second candidate questions whose answer type is a reference data query type; splice each second candidate question and element decomposition prompt information, and input the splicing results into multiple large language models in parallel to obtain multiple candidate reference data corresponding to each second candidate question; based on the multiple candidate reference data corresponding to each second candidate question, screen out second target questions that meet second preset conditions from each second candidate question, and determine reference data that matches each second target question from the multiple candidate reference data corresponding to each second target question; summarize each second target question and the reference data that matches each second target question to obtain a question reference information data set.

[0190] The question type data set includes multiple first target questions and answer types that match the first target questions. Based on this, according to the type differences of the answer types, the multiple first target questions in the question type data set can be divided into three categories, namely, the first target questions whose answer types are reference data query types, the first target questions whose answer types are semantic similarity matching types, and the first target questions whose answer types are rejection reply types. In the embodiment of the present application, the first target question whose answer type is reference data query type is determined as the second candidate question in the question reference information data set.

[0191] For any second candidate question, the second candidate question and the element decomposition prompt information are spliced, and the spliced ​​result is copied k times and input into k large language models in parallel to instruct each large language model to output the candidate reference data of the second candidate question, and k candidate reference data corresponding to the second candidate question are obtained. Then, it is determined whether the k candidate reference data meet the second preset condition, and if the second preset condition is met, the second candidate question is determined as the second target question, and the reference data matching the second target question is determined from the k candidate reference data of the second candidate question.

[0192] For example, the second preset condition may be that the candidate reference data with the highest frequency among the k candidate reference data is not less than m times. Based on this, the frequency of occurrence of different types of candidate reference data among the k candidate reference data may be counted, and the maximum frequency q kmax Compared with m, if q kmax >m, the second candidate problem is determined as the second target problem, from the maximum frequency q kmax The corresponding candidate reference data is determined as the reference data matching the second target question. kmax <n, the second candidate question is determined to be invalid, and the second candidate question is no longer aggregated into the question reference information data set.

[0193] The above steps are repeated to select at least one second target question from the plurality of second candidate questions, and reference data matching each second target question is determined from the plurality of candidate reference data corresponding to each second target question.

[0194] Each second target question and the reference data matching each second target question are combined to obtain multiple sets of second target question reference information data pairs, and the multiple sets of second target question reference information data pairs are summarized to construct a question reference information data set.

[0195] In the embodiment of the present application, the questions whose answer types in the question type data set are reference data query types are determined as the second candidate questions in the question reference information data set, and the answer types of the second candidate questions are unified to preliminarily ensure the validity of the questions in the question reference information data set. Furthermore, candidate reference data for each second candidate question is automatically generated through multiple large language models, and verification and judgment are performed, which improves the compatibility of the reference data with the question and greatly reduces the cost of manual annotation.

[0196] In one embodiment, according to the plurality of candidate reference data corresponding to each second candidate question, second target questions that meet the second preset condition are screened out from each second candidate question, and reference data matching each second target question is determined from the plurality of candidate reference data corresponding to each second target question, including:

[0197] For any second candidate question, determine the quantitative proportion of different types of candidate reference data in the multiple candidate reference data corresponding to the second candidate question; if the maximum value among the quantitative proportion values ​​is not less than the second proportion threshold, the second candidate question is determined to be the second target question, and the candidate reference data corresponding to the maximum value among the quantitative proportion values ​​is determined as the reference data matching the second target question.

[0198] For any second candidate question, the corresponding candidate reference data can be an event table, a summary table, and a character setting table. Based on this, the number of candidate reference data that are event tables in the multiple candidate reference data corresponding to the second candidate question is calculated as the proportion of the number of candidate reference data that are summary tables in the multiple candidate reference data corresponding to the second candidate question, and the number of candidate reference data that are character setting tables in the multiple candidate reference data corresponding to the second candidate question are calculated, respectively, to obtain the quantity ratio values ​​of the three candidate reference data types corresponding to the second candidate question, and compare the three quantity ratio values ​​to obtain the maximum quantity ratio value.

[0199] Furthermore, the maximum value and the second ratio threshold are compared. If the maximum value is less than the first ratio threshold, the second candidate question is determined to be invalid and the second candidate question is discarded. If the maximum value is not less than the second ratio threshold, the second candidate question is determined to be the second target question, and from the multiple candidate reference data corresponding to the second target question, the candidate reference data corresponding to the largest quantity proportion value is determined as the reference data matching the second target question. Finally, the second target question and the reference data matching the second target question are combined into a group of question reference information data pairs, which are summarized into a question reference information data set.

[0200] In an embodiment of the present application, for multiple candidate reference data of the second candidate question, by comparing the quantity proportion values ​​of different types of candidate reference data and the second proportion threshold, when the second proportion threshold is met, the second candidate question is determined to be the second target question, and the candidate reference data corresponding to the largest quantity proportion value is determined as the reference data that matches the second target question, thereby ensuring the validity of the second target question and the matching degree between the second target question and the reference data, and improving the validity of the question reference information data set.

[0201] In one embodiment, the initial data question answering model is trained according to the data question answering dataset to obtain the corresponding target data question answering model, including:

[0202] Obtain a question analysis data set; randomly mix the data in the question type data set, the question reference information data set, and the question analysis data set to obtain a mixed processing data set; input the mixed processing data set into the initial data question answering model for training until the number of training rounds meets the preset number of rounds, the training is completed, and the target data question answering model is obtained.

[0203] The question analysis data set includes multiple sets of question analysis data, each set of question analysis data includes: questions, answer data matching the questions, and question answer analysis content. It should be noted that the questions in the question analysis data set cover the questions in the question type data set.

[0204] The question type dataset and question reference information dataset in the data question answering dataset, as well as the order of each group of data in the question analysis dataset are randomly shuffled to obtain a mixed processing dataset. Multiple groups of data pairs in the mixed processing dataset are input into the initial data question answering model in batches according to the preset order and batch training size to obtain the prediction results output by the initial data question answering model.

[0205] It should be noted that in the mixed processing data set, the question type data set includes multiple groups of "question-answer type", and the answer type is the real result; the question reference information data set includes multiple groups of "question-reference data", and the reference data is the real result; the question analysis data set includes multiple groups of "question answer data-analysis content", and the analysis content is the real result.

[0206] Next, the loss error between the predicted result and the actual result of the input question is calculated, and the parameters in the initial data question-answering model are updated until the number of training rounds reaches the preset number of rounds (all data in the data question-answering dataset is input into the initial data question-answering model once for one round of training), and the target data question-answering model is obtained.

[0207] In the embodiment of the present application, according to the question type data set in the mixed processing data set, the logical reasoning ability of the initial data question and answer model to determine the answer type according to the question text is trained, according to the question reference information data set in the mixed processing data set, the logical reasoning ability of the initial data question and answer model to determine the reference data according to the question text is trained, and according to the question analysis data set in the mixed processing data set, the logical reasoning ability of the initial data question and answer model to analyze the question is trained. Adding question analysis data to the mixed processing data set improves the initial data question and answer model to understand the principles of question answer type analysis and factor analysis during the training process, so that the target data question and answer model completed by training has diverse logical reasoning capabilities, which greatly enriches the application scenarios of the target data question and answer model.

[0208] In one embodiment, obtaining a problem analysis data set includes:

[0209] The first target questions in the question type data set and the answer data matching each first target question are combined to obtain multiple candidate question result combinations; each candidate question result combination and the question analysis prompt word are spliced, and the spliced ​​results are input into multiple large language models in parallel to obtain multiple candidate analysis texts corresponding to each candidate question result combination; according to the multiple candidate analysis texts corresponding to each candidate question result combination, a target question result combination that meets a third preset condition is screened out from the multiple candidate question result combinations, and an analysis text matching each target question result combination is determined from the multiple candidate analysis texts corresponding to each target question result combination; each target question result combination and the analysis text matching each target question result combination are summarized to obtain a question analysis data set.

[0210] By manually marking or querying historical question answer data, the answer data of the first target question in the question type data set is determined, and then each first target question and the answer data matching each first target question are combined to obtain multiple candidate question result combinations.

[0211] For any candidate question result combination, the candidate question result combination and the question analysis prompt word are spliced, and the splicing results are input into k large language models in parallel to instruct the large language models to output the candidate analysis texts of the candidate question result combination, and k candidate analysis texts corresponding to the candidate question result combination are obtained. Then, it is determined whether the k candidate analysis texts meet the third preset condition. If the third preset condition is met, the candidate question result combination is determined as the target question result combination, and the analysis text matching the candidate question result combination is determined from the k candidate analysis texts of the candidate question result combination.

[0212] Exemplarily, the third preset condition may be that the semantic similarities between the k candidate analysis texts are all greater than a preset semantic similarity threshold. The semantic similarities between the k candidate analysis texts corresponding to the candidate question result combination are calculated. If each semantic similarity is greater than the semantic similarity threshold, the candidate question result combination is determined to be the target question result combination, and any one of the two candidate analysis texts with the greatest semantic similarity is determined as the analysis text matching the candidate question result combination; if any semantic similarity is less than the semantic similarity threshold, the candidate question result combination is determined to be invalid, and the candidate question result combination will not be aggregated into the question analysis data set.

[0213] The above steps are repeated to select at least one target question result combination from the multiple candidate question result combinations, and an analysis text matching each target question result combination is determined from the multiple candidate analysis texts corresponding to each target question result combination.

[0214] Each target problem result combination and the analysis text matching the target problem result combination are combined to obtain multiple groups of problem analysis data, and the multiple groups of problem analysis data are summarized to construct a problem analysis data set.

[0215] In the embodiment of the present application, for any candidate question result combination of multiple candidate analysis texts, when the multiple candidate analysis texts meet the third preset condition, the candidate question result combination is determined to be the target question result combination to ensure the validity of the target question result combination. Then, from the multiple candidate analysis texts corresponding to the target question result combination, the analysis text that matches the target question result combination is determined to improve the matching degree between the second target question and the reference data, thereby improving the validity of the question analysis data set.

[0216] In one embodiment, according to the multiple candidate analysis texts corresponding to each candidate question result combination, a target question result combination that meets the third preset condition is screened out from the multiple candidate question result combinations, and an analysis text matching each target question result combination is determined from the multiple candidate analysis texts corresponding to each target question result combination, including:

[0217] For any candidate question result combination, multiple candidate analysis texts corresponding to the candidate question result combination are encoded to obtain candidate vectors of each candidate analysis text, and the vector similarity between every two candidate vectors is calculated; if the minimum vector similarity among the vector similarities is not less than the vector similarity threshold, the candidate question result combination is determined to be the target question result combination, and the analysis text that matches the target question result combination is determined from the multiple candidate analysis texts corresponding to the target question result combination.

[0218] For any candidate question result combination, multiple candidate analysis texts corresponding to the candidate question result combination are respectively encoded through a pre-trained language encoding model to obtain candidate vectors of each candidate analysis text; two different candidate vectors are input into a similarity calculation model to determine the vector similarity between every two candidate vectors; the multiple determined vector similarities are sorted to obtain the minimum vector similarity; the minimum vector similarity is compared with the vector similarity threshold, if the minimum vector similarity is less than the vector similarity threshold, the candidate question result combination is determined to be an invalid question and the question is discarded; if the minimum vector similarity is not less than the vector similarity threshold, the candidate question result combination is determined to be a target question result combination, and the analysis text matching the target question result combination is determined from the multiple candidate analysis texts corresponding to the target question result combination.

[0219] Exemplarily, the candidate question result combination Q The corresponding candidate analysis text is expressed as o i , analyze the candidate text oi The candidate vector obtained by encoding is expressed as e i , calculate the vector similarity between two candidate vectors s ij , the vector similarity threshold is ,like , then discard the candidate question result combination Q ,like , then determine the candidate question result combination Q Combine the target question results and multiple candidate analysis texts corresponding to the target question results o i , determine the analysis text that matches the target question result combination.

[0220] In an embodiment of the present application, for each candidate question result combination of multiple candidate analysis texts, each candidate analysis text is encoded, and the characteristics of the candidate analysis text are objectively represented in the form of a candidate vector to facilitate the subsequent evaluation of the degree of proximity between the candidate analysis texts. The minimum vector similarity between different candidate vectors is then determined. When the minimum vector similarity is not less than a vector similarity threshold, the candidate question combination result is determined to be the target question combination result, and from the multiple candidate analysis texts corresponding to the target question result combination, an analysis text that matches the target question result combination is determined to ensure the validity of the target question result combination and the matching degree between the target question result combination and the analysis text.

[0221] In one embodiment, determining an analysis text matching the target question result combination from a plurality of candidate analysis texts corresponding to the target question result combination includes:

[0222] Calculate multiple vector similarities between the candidate vector of each candidate analysis text and the candidate vectors of other candidate analysis texts, and determine the minimum vector similarity corresponding to each candidate analysis text; from the minimum vector similarities corresponding to each candidate analysis text, screen out the target vector similarity that meets the fourth preset condition, and determine the candidate analysis text corresponding to the target vector similarity as the analysis text that matches the target question result combination.

[0223] After determining the target question result combination, the minimum vector similarity between each candidate analysis text corresponding to the target question result combination and each other candidate analysis text is calculated, and the candidate analysis text corresponding to the minimum vector similarity that meets the fourth preset condition is determined as the analysis text matching the target question result combination.

[0224] Among them, the larger the minimum vector similarity of the candidate analysis text, the closer the candidate analysis text is to other candidate analysis texts, and the closer the candidate analysis text is to the real analysis text; the smaller the minimum vector similarity of the candidate analysis text, the more different the candidate analysis text is from other candidate analysis texts, and the greater the gap between the candidate analysis text and the real analysis text.

[0225] In one embodiment, the fourth preset condition may be a function for determining the maximum value of vector similarity. The candidate analysis text corresponding to the target question result combination is represented as o i , analyze the candidate text o i The candidate vector obtained by encoding is expressed as e i , calculate the vector similarity between two candidate vectors s ij , as an example, the vector similarity between each candidate vector and other candidate vectors is calculated respectively, and the minimum vector similarity of each candidate necklace is obtained. Then, from multiple candidate vectors, determine the candidate vector with the largest minimum vector similarity t , the expression of the fourth preset condition is as follows:

[0226]

[0227] In the above formula, Represents the candidate vector i The minimum vector similarity among the vector similarities between the candidate vectors. Indicates from i Take the maximum value among the minimum vector similarities.

[0228] Then the candidate vector t The corresponding candidate analysis text is determined as the analysis text that matches the target question result combination. The target question result combination and the analysis text are concatenated to obtain a set of data pairs of "question, analysis text and answer data", which are summarized into the question analysis data set.

[0229] In an embodiment of the present application, the minimum vector similarity corresponding to each candidate analysis text is calculated to characterize the gap between each candidate analysis text and other candidate texts, quantify the gap between each candidate analysis text and the actual analysis text, and provide a reliable and accurate basis for determining the analysis text that matches the target question result combination. Then, in combination with the fourth preset condition, the candidate analysis text with the largest similarity among the minimum vector similarities is determined as the analysis text that matches the target question result combination, thereby improving the matching degree between the target question result combination and the analysis text, thereby improving the validity of the problem analysis data set.

[0230] In one embodiment, the data question-and-answer method provided by the present application can be applied to the evaluation scenario of candidate scripts. In this scenario, the target multimedia data is the candidate script, the data question-and-answer model is the target script question-and-answer model, the data question-and-answer dataset is the script dataset, and the script evaluator efficiently understands the content of the candidate script through question-and-answer format.

[0231] In the actual processing process, the script question answering solution can be divided into the training process of the target script question answering model and the prediction process of the target script question answering model.

[0232] Please refer to Figure 5, which is a schematic diagram of the training of the target script question-answering model. According to the original script, a question type dataset, a question reference information dataset and a question analysis dataset are constructed; the data in the question type dataset, the question reference information dataset and the question analysis dataset are randomly shuffled to obtain a script dataset; the initial script question-answering model is fine-tuned according to the script dataset to obtain the target script question-answering model. Specifically, the following steps are included:

[0233] S501, segmenting the original script.

[0234] S502, based on the segmented original script text, construct a question type dataset, a question reference information dataset and a question analysis dataset.

[0235] It should be noted that there is overlap between the questions in the question type data set and the questions in the question analysis data set. In this embodiment, the questions in the question analysis data set may be questions in the question type data set whose answer type is a reference data query type.

[0236] S503, obtaining a script dataset according to the question type dataset, the question reference information dataset and the question analysis dataset.

[0237] S504, based on the script data set, train the initial script question-answering model to obtain the target script question-answering model.

[0238] Below, the construction methods of the question type dataset, question reference information dataset and question analysis dataset in the script dataset are explained respectively.

[0239] (1) Question type dataset

[0240] According to any segmentation method of single episode, single chapter, single scene, continuous multiple episodes, continuous multiple chapters, and continuous multiple scenes, the original script is segmented to obtain multiple script fragments. For each script fragment, please refer to Figure 6, each script fragment is spliced ​​with different question construction prompt information, and input into the question generation model to generate script questions of different granularity, different categories, and comprehensive; Figure 6 takes any generated script question as an example, splices the script question with the answer type prompt information of k different general large language models, and inputs into different general large language models to obtain the answer type prediction results output by each language model; according to the first preset condition, predict whether the answer type prediction result output by the large language model is accurate: determine the number proportion of different types of answer type prediction results in all answer type prediction results, if the maximum value in the number proportion value is not less than the first proportion threshold, then the answer type prediction result of this type is the answer type of the script question, and the "script question-answer type" group of data pairs is added to the question type data set; if the maximum value in the number proportion value is less than the first proportion threshold, it is considered that the confidence of the obtained answer type prediction result is low, and the script question is discarded. For each script segment, repeat the above steps until the question type dataset is constructed.

[0241] (2) Question reference information dataset.

[0242] The questions whose answer type in the question type data set is the reference data query type are used as candidate questions of the question reference information data set. For each candidate question, please refer to Figure 7, each candidate question is spliced ​​with the element disassembly prompt information of different general large language models, and input into different general large language models to obtain the reference data prediction results output by each language model; according to the second preset condition, predict whether the reference data prediction results output by the large language model are accurate: determine the quantitative proportion of different types of reference data prediction results in all reference data prediction results, if the maximum value in the quantitative proportion value is not less than the second proportion threshold, then the reference data prediction result of this type is used as the reference data of the candidate question, and the "candidate question-reference data" data pair is added to the question reference information data set; if the maximum value in the quantitative proportion value is less than the second proportion threshold, it is considered that the confidence of the obtained reference data prediction result is low, and the candidate question is discarded. Repeat the above steps for each candidate question until the construction of the question reference information data set is completed.

[0243] (3) Problem Analysis Dataset

[0244] Combine a part of the questions in the question type data set with the answer content of this part of the questions to obtain question-answer pairs. For each question-answer pair, please refer to Figure 8, splice each question-answer pair with the question analysis prompt information of different general large language models, input them into different general large language models, obtain the reference analysis text output by each large language model, encode each prediction analysis text, obtain the text vector corresponding to each prediction analysis text, obtain the minimum vector similarity between each text vector and other text vectors, and predict whether the analysis text prediction result output by the large language model is accurate according to the vector similarity threshold: if multiple minimum vector similarities are not less than the vector similarity threshold, then the prediction analysis text corresponding to the maximum value of the minimum vector similarity is determined as the analysis text of the question-answer pair, and the "question-answer-analysis text" data pair is added to the question analysis data set; if there is a minimum vector similarity among multiple minimum vector similarities that is less than the vector similarity threshold, then the question-answer pair is discarded. Repeat the above steps for each question-answer pair until the construction of the question analysis data set is completed.

[0245] Please refer to FIG. 9 , which is a schematic diagram of the prediction process of the target script question-answering model, including:

[0246] S901, splicing the script evaluation questions and answer type prompt information input by the script evaluator, inputting the splicing result into the target script question and answer model, and instructing the target script question and answer model to output the target answer type.

[0247] S902: If the target answer type is a rejection reply type, instruct the terminal to display a rejection reply prompt message.

[0248] S903: If the target answer type is a semantic similarity matching type, instruct the terminal to display semantic similarity matching prompt information.

[0249] S904, if the target answer type is a reference data query type, the script evaluation question and the element decomposition prompt information are spliced, and the splicing result is input into the target script question and answer model, instructing the target script question and answer model to output the target reference data, and determining the answer data corresponding to the target question from multiple sets of key information of the target multimedia data based on the target reference data.

[0250] In the embodiments of the present application, by automatically constructing a script question and answer dataset, the manual annotation cost is significantly reduced, and the script question and answer model is fine-tuned according to the script question and answer dataset, so that the target script question and answer model can efficiently and accurately judge the type of answers to script questions and decompose database queries, enhance the target script question and answer model's understanding of the distribution logic and answer search strategy of the target question, improve the prediction accuracy and generalization ability of the target script question and answer model, and provide a smarter and more efficient solution for information retrieval in script question and answer scenarios.

[0251] In one embodiment, a data question and answer method is provided, and the method is applied to a server as an example for explanation. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. As shown in FIG10 , the method includes the following steps:

[0252] S1001, dividing the target multimedia data into multiple sub-target multimedia data.

[0253] S1002, input each sub-target multimedia data into the question generation model respectively, obtain multiple first target questions of each sub-target multimedia data and the answer type matching the first target question, and construct a question type data set.

[0254] S1003, determining a second candidate question whose answer type is a reference data query type from a plurality of first target questions in the question type data set.

[0255] S1004, concatenate each second candidate question and element decomposition prompt information, input the concatenation results into multiple large language models in parallel, obtain multiple second target questions and reference data matching each second target question, and construct a question reference information data set.

[0256] S1005, combining the first target question and the answer data matching the first target question, and splicing them with the question analysis prompt words, and inputting the splicing results into multiple large language models in parallel, to obtain multiple target question result combinations and analysis texts matching each target question result combination, and construct a question analysis data set.

[0257] S1006, randomly mixing the data in the question type data set, the question reference information data set and the question analysis data set to obtain a mixed processing data set.

[0258] S1007, input the mixed processed data set into the initial data question-answering model for training, until the number of training rounds meets the preset number of rounds, the training is completed, and the target data question-answering model is obtained.

[0259] S1008: Obtain a target question for the target multimedia data.

[0260] S1009, determining a supplementary area in the answer type prompt information according to the question supplementary prompt information in the answer type prompt information, filling the target question into the supplementary area, and obtaining a first question text.

[0261] S1010, input the first question text into the target data question-answering model, and determine the target answer type that matches the target question.

[0262] S1011, when the target answer type is a reference data query type, supplement prompt information according to the question in the element decomposition prompt information, and determine the supplement area in the element decomposition prompt information.

[0263] S1012, fill the target question into the supplementary area to obtain a second question text, input the second question text into the target data question-answering model, and determine the target reference data that matches the target question text features.

[0264] S1013, determining answer data corresponding to the target question based on the target reference data.

[0265] In an embodiment of the present application, when processing the target question, first determine the target answer type based on the logical relationship between the question text feature and the answer type, preliminarily clarify the method for obtaining the answer content corresponding to the target question, and improve the speed of obtaining the answer content. Then, in the case where the target answer type is a reference data query type, determine the target reference data based on the logical relationship between the question text feature and the reference data, further clarify the search scope of the answer content corresponding to the target question, and improve the search speed of the answer content. Finally, quickly search for the key information corresponding to the target question based on the target reference data, and use the key information as the answer data corresponding to the target question. This is equivalent to, when facing the processing task of the target question, using the task decomposition strategy to decompose the processing task into the answer type acquisition task and the reference data search task, simplifying the complex problem processing task, so that the server can select the appropriate answer data acquisition method according to the question text feature of the target question, and then accurately find the answer data of the target question, and improve the accuracy of question and answer.

[0266] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0267] Based on the same inventive concept, the embodiment of the present application also provides a data question and answer device for implementing the data question and answer method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data question and answer device embodiments provided below can refer to the limitations of the data question and answer method above, and will not be repeated here.

[0268] In one embodiment, as shown in FIG. 11 , a data question-answering device is provided, including: a question acquisition module 1102 , a type determination module 1104 , and a content determination module 1106 , wherein:

[0269] A question acquisition module 1102 is used to acquire a target question for target multimedia data;

[0270] A type determination module 1104 is used to determine a target answer type corresponding to a target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type;

[0271] The content determination module 1106 is used to determine the target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data when the target answer type is a reference data query type, and determine the answer data corresponding to the target question based on the target reference data; the answer data includes the content obtained by extracting key information from the target multimedia data.

[0272] In one embodiment, the type determination module 1104 includes:

[0273] A first concatenation unit is used to concatenate the answer type prompt information and the target question to obtain a first question text;

[0274] The type prediction unit is used to input the first question text into the first data question-answering model, extract the target question text features of the target question in the first question text through the first data question-answering model, and determine the target answer type that matches the target question based on the logical relationship between the question text features and the answer type.

[0275] In one embodiment, the first splicing unit is further used to determine the supplementary area in the answer type prompt information according to the question supplementary prompt information in the answer type prompt information; fill the target question into the supplementary area to obtain the first question text.

[0276] In one embodiment, the type prediction unit is further used to determine that the target question text features match the reference data query type and determine the target answer type to be the reference data query type when the target question text features include at least one of chapter text features, character attribute text features, and character event text features; determine that the target question text features match the semantic similarity matching type and determine the target answer type to be the semantic similarity matching type when the target question text features include at least one of event name text features, scene attribute text features, and non-character attribute word text features; and determine that the target answer type is a rejection reply type when the degree of matching between the target question text features and the multimedia data is less than a preset matching degree threshold.

[0277] In one embodiment, the content determination module 1106 includes:

[0278] A second splicing unit is used to splice the target question and the element decomposition prompt information to obtain a second question text;

[0279] The reference data prediction unit is used to input the second question text into the second data question-answering model, extract the target question text features of the target question in the second question text through the second data question-answering model, and determine the target reference data that matches the target question text features from the reference data based on the logical relationship between the question text features and the reference data.

[0280] In one embodiment, the data question and answer device also includes: a logic training unit, which is used to train the initial data question and answer model according to the data question and answer data set to obtain a target data question and answer model; the data question and answer data set includes a first data set for training the initial data question and answer model to determine the logical relationship of the answer type according to the question text and a second data set for training the initial data question and answer model to determine the logical relationship of the reference data according to the question text.

[0281] In one embodiment, the second splicing unit is further used to supplement the prompt information according to the question in the element disassembly prompt information, determine the supplementary area in the element disassembly prompt information; fill the target question into the supplementary area to obtain the second question text.

[0282] In one embodiment, the reference data prediction unit is further used to, when the target question text feature includes a character name text feature, determine that the text feature of the target question matches the event table, and determine that the target reference data is the event table; when the target question text feature includes a numeric range text feature, determine that the text feature of the target question matches the summary table, and determine that the target reference data is the summary table; when the target question text feature includes a character attribute text feature, determine that the text feature of the target question matches the character setting table, and determine that the target reference data is the character setting table.

[0283] In one embodiment, the data question answering device further includes:

[0284] An interface display module, used to display a first interface corresponding to the target multimedia data, the first interface being used to indicate input of a target question;

[0285] A question sending module, used for sending a target question for target multimedia data to a server, so as to instruct the server to obtain answer data corresponding to the target question according to the target question;

[0286] The data receiving module is used to receive the answer data returned by the server and display the answer data on the first interface.

[0287] Based on the same inventive concept, the embodiment of the present application also provides a data question-answering device for implementing the data question-answering model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data question-answering model training device embodiments provided below can refer to the limitations of the data question-answering model training method above, and will not be repeated here.

[0288] In one embodiment, as shown in FIG. 12 , a data question-answering model training device is provided, including: a data set acquisition module 1202 and a model training module 1204, wherein:

[0289] The data set acquisition module 1202 is used to acquire a data question and answer data set; the data question and answer data set includes a question type data set and a question reference information data set; the question type data set includes questions and answer types matching the questions; the question reference information data set includes questions and answer data matching the questions;

[0290] A model training module 1204 is used to train the initial data question answering model according to the data question answering data set to obtain a corresponding target data question answering model;

[0291] The target data question-answering model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type. When the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data.

[0292] In one embodiment, the data set acquisition module 1202 further includes:

[0293] A data segmentation unit, used for segmenting the target multimedia data to obtain a plurality of sub-target multimedia data;

[0294] A question generation unit, used to input each sub-target multimedia data into a question generation model respectively, and construct questions for each sub-target multimedia data based on a plurality of question construction prompt words through the question generation model to obtain a plurality of first candidate questions for each sub-target multimedia data;

[0295] A question type concatenation unit, used to concatenate each first candidate question and answer type prompt information, and input the concatenation results into multiple large language models in parallel to obtain multiple candidate answer types corresponding to each first candidate question;

[0296] A first question screening unit is used to screen out first target questions that meet a first preset condition from a plurality of first candidate questions according to a plurality of candidate answer types corresponding to each first candidate question, and determine an answer type that matches each first target question from a plurality of candidate answer types corresponding to each first target question;

[0297] The first question summarizing unit is used to summarize each first target question and the answer type matching each first target question to obtain a question type data set.

[0298] In one embodiment, the first question screening unit is further configured to:

[0299] For any first candidate question, determining the quantitative proportion of different types of candidate answer types in the multiple candidate answer types corresponding to the first candidate question;

[0300] When the maximum value among the quantity proportion values ​​is not less than the first ratio threshold, the first candidate question is determined to be the first target question, and the candidate answer type corresponding to the maximum value among the quantity proportion values ​​is determined as the answer type that matches the first target question.

[0301] In one embodiment, the data set acquisition module 1202 further includes:

[0302] A question determination unit, configured to determine, from a plurality of first target questions in the question type data set, a second candidate question whose answer type is a reference data query type;

[0303] A question prompt concatenation unit is used to concatenate each second candidate question and the element decomposition prompt information, and input the concatenation results into multiple large language models in parallel to obtain multiple candidate reference data corresponding to each second candidate question;

[0304] A second question screening unit is used to screen out second target questions that meet a second preset condition from each second candidate question according to a plurality of candidate reference data corresponding to each second candidate question, and to determine reference data that matches each second target question from the plurality of candidate reference data corresponding to each second target question;

[0305] The second question summarizing unit is used to summarize each second target question and the reference data matching each second target question to obtain a question reference information data set.

[0306] In one embodiment, the second question screening unit is further used to:

[0307] For any second candidate question, determining a quantity ratio of different types of candidate reference data in the multiple candidate reference data corresponding to the second candidate question;

[0308] When the maximum value among the quantity proportion values ​​is not less than the second ratio threshold, the second candidate question is determined to be the second target question, and the candidate reference data corresponding to the maximum value among the quantity proportion values ​​is determined as the reference data matching the second target question.

[0309] In one embodiment, the model training module 1204 further includes:

[0310] An analysis data set acquisition unit, used for acquiring a problem analysis data set;

[0311] A data set concatenation unit is used to randomly mix the data in the question type data set, the question reference information data set and the question analysis data set to obtain a mixed processing data set;

[0312] The target model acquisition unit is used to input the mixed processed data set into the initial data question and answer model for training until the number of training rounds meets the preset number of rounds, the training is completed, and the target data question and answer model is obtained.

[0313] In one embodiment, the analysis data set acquisition unit is further configured to:

[0314] Combining the first target questions in the question type data set and the answer data matching each first target question to obtain a plurality of candidate question result combinations;

[0315] Each candidate question result combination and the question analysis prompt word are concatenated, and the concatenated results are input into multiple large language models in parallel to obtain multiple candidate analysis texts corresponding to each candidate question result combination;

[0316] According to the multiple candidate analysis texts corresponding to each candidate question result combination, a target question result combination that meets the third preset condition is screened out from the multiple candidate question result combinations, and an analysis text matching each target question result combination is determined from the multiple candidate analysis texts corresponding to each target question result combination;

[0317] Each target question result combination and the analysis text matching each target question result combination are summarized to obtain a question analysis data set.

[0318] In one embodiment, the analysis data set acquisition unit is further configured to:

[0319] For any candidate question result combination, multiple candidate analysis texts corresponding to the candidate question result combination are encoded to obtain candidate vectors of each candidate analysis text, and the vector similarity between every two candidate vectors is calculated;

[0320] If the minimum vector similarity among the vector similarities is not less than the vector similarity threshold, the candidate question result combination is determined to be the target question result combination, and an analysis text matching the target question result combination is determined from multiple candidate analysis texts corresponding to the target question result combination.

[0321] In one embodiment, the analysis data set acquisition unit is further configured to:

[0322] Calculating multiple vector similarities between the candidate vector of each candidate analysis text and the candidate vectors of other candidate analysis texts, and determining the minimum vector similarity corresponding to each candidate analysis text;

[0323] From the minimum vector similarities corresponding to the candidate analysis texts, a target vector similarity that meets the fourth preset condition is screened out, and the candidate analysis text corresponding to the target vector similarity is determined as the analysis text that matches the target question result combination.

[0324] Each module in the above-mentioned data question-answering device and data question-answering model training device can be implemented in whole or in part by software, hardware, and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0325] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG13. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as target multimedia videos and question libraries. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data question answering method or a data question answering model training method is implemented.

[0326] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG14. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data question-answering method or a data question-answering model training method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0327] Those skilled in the art will understand that the structures shown in FIGS. 13 and 14 are merely block diagrams of partial structures related to the scheme of the present application, and do not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than those shown in the figures, or combine certain components, or have a different arrangement of components.

[0328] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0329] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0330] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0331] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0332] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0333] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0334] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data question-answering method, characterized in that: The method comprises: Obtaining target questions for target multimedia data; Determining a target answer type corresponding to the target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type; In the case where the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data; the target reference data is an identifier of the answer content corresponding to the target question in the multimedia database, and is used to determine a method for searching the multimedia database for the target question.

2. The method according to claim 1, characterized in that The determining of the target answer type corresponding to the target question based on the logical relationship between the question text feature represented by the answer type prompt information corresponding to the target multimedia data and the answer type comprises: Concatenate the answer type prompt information and the target question to obtain a first question text; The first question text is input into a first data question-answering model, target question text features of a target question in the first question text are extracted through the first data question-answering model, and the target answer type matching the target question is determined based on a logical relationship between the question text features and the answer type.

3. The method according to claim 2, characterized in that The step of concatenating the answer type prompt information and the target question to obtain a first question text includes: Determine a supplementary area in the answer type prompt information according to the question supplement prompt information in the answer type prompt information; Fill the target question into the supplementary area to obtain the first question text.

4. The method according to claim 2, characterized in that: The extracting the target question text features of the target question in the first question text by the first data question-answering model, and determining the target answer type matching the target question according to the logical relationship between the question text features and the answer type, includes at least one of the following methods: If the target question text feature includes at least one of a chapter text feature, a character attribute text feature, and a character role event text feature, determining that the target question text feature matches the reference data query type, and determining that the target answer type is the reference data query type; If the target question text feature includes at least one of an event name text feature, a scene attribute text feature, and a non-character attribute word text feature, determining that the target question text feature matches a semantic similarity matching type, and determining that the target answer type is the semantic similarity matching type; If the matching degree between the target question text feature and the multimedia data is less than a preset matching degree threshold, the target answer type is determined to be a rejection reply type.

5. The method according to claim 2, characterized in that: The determining of the target reference data of the target question based on the logical relationship between the question text feature represented by the element disassembly prompt information corresponding to the target multimedia data and the reference data includes: The target question and the element decomposition prompt information are concatenated to obtain a second question text; The second question text is input into a second data question-answering model, target question text features of a target question in the second question text are extracted through the second data question-answering model, and according to a logical relationship between the question text features and reference data, the target reference data matching the target question text features is determined from the reference data.

6. The method according to claim 5, characterized in that The first data question-answering model and the second data question-answering model are the same target data question-answering model, and the method further includes: An initial data question and answer model is trained according to a data question and answer dataset to obtain a target data question and answer model; the data question and answer dataset includes a first dataset for training the initial data question and answer model to determine a logical relationship of an answer type according to a question text and a second dataset for training the initial data question and answer model to determine a logical relationship of reference data according to the question text.

7. The method according to claim 5, characterized in that The step of combining the target question and the element decomposition prompt information to obtain a second question text includes: Determine a supplementary area in the element disassembly prompt information according to the question supplement prompt information in the element disassembly prompt information; Fill the target question into the supplementary area to obtain the second question text.

8. The method according to claim 5, characterized in that The element disassembly prompt information includes: at least one of an event table, a summary table, and a character setting table; the determining, from the reference data, the target reference data matching the target question text feature according to the logical relationship between the question text feature and the reference data includes at least one of the following methods: If the target question text feature includes a role name text feature, determining that the text feature of the target question matches the event table, and determining that the target reference data is the event table; If the target question text feature includes a numeric range text feature, determining that the text feature of the target question matches the summary table, and determining that the target reference data is the summary table; If the target question text feature includes a character attribute text feature, it is determined that the text feature of the target question matches the character setting table, and the target reference data is determined to be the character setting table.

9. A data question answering model training method, characterized in that: The method comprises: Acquire a data question and answer dataset; the data question and answer dataset includes a question type dataset and a question reference information dataset; the question type dataset includes questions and answer types matching the questions; the question reference information dataset includes questions and reference data matching the questions; According to the data question and answer data set, an initial data question and answer model is trained to obtain a corresponding target data question and answer model; The target data question-answering model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type. When the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data; the target reference data is an identifier of the answer content corresponding to the target question in the multimedia database, and is used to determine a method for searching the multimedia database for the target question.

10. The method according to claim 9, characterized in that The process of obtaining the problem type data set includes: Segmenting the target multimedia data to obtain a plurality of sub-target multimedia data; Inputting each of the sub-target multimedia data into a question generation model respectively, constructing a question for each of the sub-target multimedia data based on a plurality of question construction prompt words through the question generation model, and obtaining a plurality of first candidate questions for each of the sub-target multimedia data; splicing each of the first candidate questions and the answer type prompt information, and inputting the splicing results into multiple large language models in parallel to obtain multiple candidate answer types corresponding to each of the first candidate questions; According to the multiple candidate answer types corresponding to the first candidate questions, the first target questions that meet the first preset condition are screened out from the multiple first candidate questions, and the answer types that match the first target questions are determined from the multiple candidate answer types corresponding to the first target questions; The first target questions and the answer types matching the first target questions are aggregated to obtain the question type data set.

11. The method according to claim 10, characterized in that The step of selecting a first target question that meets a first preset condition from the plurality of first candidate questions according to the plurality of candidate answer types corresponding to the first candidate questions, and determining an answer type that matches the first target question from the plurality of candidate answer types corresponding to the first target questions includes: For any first candidate question, determining, from among the multiple candidate answer types corresponding to the first candidate question, a quantitative proportion of different types of candidate answer types in the multiple candidate answer types; If the maximum value among the quantity proportion values ​​is not less than the first ratio threshold, the first candidate question is determined to be the first target question, and the candidate answer type corresponding to the maximum value among the quantity proportion values ​​is determined as the answer type that matches the first target question.

12. The method according to claim 9, characterized in that The process of obtaining the question reference information dataset includes: Determine, from the plurality of first target questions in the question type data set, a second candidate question whose answer type is a reference data query type; Splicing each of the second candidate questions and the element decomposition prompt information, and inputting the splicing results into multiple large language models in parallel to obtain multiple candidate reference data corresponding to each of the second candidate questions; According to the plurality of candidate reference data corresponding to each of the second candidate questions, a second target question satisfying a second preset condition is screened out from each of the second candidate questions, and reference data matching each of the second target questions is determined from the plurality of candidate reference data corresponding to each of the second target questions; Each of the second target questions and the reference data matching each of the second target questions are aggregated to obtain the question reference information data set.

13. The method according to claim 12, characterized in that The step of screening out second target questions that meet a second preset condition from each of the second candidate questions according to the plurality of candidate reference data corresponding to each of the second candidate questions, and determining reference data that matches each of the second target questions from the plurality of candidate reference data corresponding to each of the second target questions includes: For any second candidate question, determining, from the plurality of candidate reference data corresponding to the second candidate question, a quantity ratio of different types of candidate reference data in the plurality of candidate reference data; If the maximum value among the quantity proportion values ​​is not less than the second ratio threshold, the second candidate question is determined to be the second target question, and the candidate reference data corresponding to the maximum value among the quantity proportion values ​​is determined as the reference data matching the second target question.

14. The method according to claim 9, characterized in that The step of training the initial data question answering model according to the data question answering dataset to obtain a corresponding target data question answering model includes: Obtain problem analysis dataset; Randomly mixing the data in the question type dataset, the question reference information dataset and the question analysis dataset to obtain a mixed processing dataset; The mixed processed data set is input into the initial data question-answering model for training until the number of training rounds meets the preset number of rounds, and the training is completed to obtain the target data question-answering model.

15. The method according to claim 14, characterized in that The step of obtaining a problem analysis data set includes: Combining the first target questions in the question type data set and the answer data matching each of the first target questions to obtain a plurality of candidate question result combinations; Each of the candidate question result combinations and the question analysis prompt words are spliced, and the splicing results are input into multiple large language models in parallel to obtain multiple candidate analysis texts corresponding to each of the candidate question result combinations; According to the multiple candidate analysis texts corresponding to each of the candidate question result combinations, a target question result combination that meets the third preset condition is screened out from the multiple candidate question result combinations, and an analysis text matching each of the target question result combinations is determined from the multiple candidate analysis texts corresponding to each of the target question result combinations; Each target question result combination and the analysis text matching each target question result combination are summarized to obtain the question analysis data set.

16. The method according to claim 15, characterized in that The method further comprises: screening a target question result combination that meets a third preset condition from the multiple candidate analysis texts corresponding to each of the candidate question result combinations, and determining an analysis text that matches each of the target question result combinations from the multiple candidate analysis texts corresponding to each of the target question result combinations, including: For any candidate question result combination, multiple candidate analysis texts corresponding to the candidate question result combination are encoded to obtain candidate vectors of each candidate analysis text, and the vector similarity between every two candidate vectors is calculated; If the minimum vector similarity among the vector similarities is not less than the vector similarity threshold, the candidate question result combination is determined to be the target question result combination, and an analysis text matching the target question result combination is determined from multiple candidate analysis texts corresponding to the target question result combination.

17. The method according to claim 16, characterized in that The step of determining an analysis text matching the target question result combination from a plurality of candidate analysis texts corresponding to the target question result combination includes: Calculating multiple vector similarities between the candidate vector of each candidate analysis text and the candidate vectors of other candidate analysis texts, and determining the minimum vector similarity corresponding to each candidate analysis text; From the minimum vector similarities corresponding to the candidate analysis texts, a target vector similarity that meets the fourth preset condition is screened out, and the candidate analysis text corresponding to the target vector similarity is determined as the analysis text that matches the target question result combination.

18. A data question-answering device, characterized in that: The device comprises: A question acquisition module, used for acquiring target questions for target multimedia data; A type determination module, configured to determine a target answer type corresponding to the target question based on a logical relationship between question text features represented by answer type prompt information corresponding to the target multimedia data and the answer type; A content determination module is used to determine the target reference data of the target question based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and to determine the answer data corresponding to the target question based on the target reference data when the target answer type is a reference data query type; the answer data includes content obtained by extracting key information from the target multimedia data; the target reference data is an identifier of the answer content corresponding to the target question in the multimedia database, and is used to determine a method for searching the multimedia database for the target question.

19. The device according to claim 18, characterized in that The type determination module comprises: A first concatenation unit is used to concatenate the answer type prompt information and the target question to obtain a first question text; A type prediction unit is used to input the first question text into a first data question-answering model, extract target question text features of a target question in the first question text through the first data question-answering model, and determine the target answer type that matches the target question based on a logical relationship between the question text features and the answer type.

20. The device according to claim 19, characterized in that The first splicing unit is further used to determine a supplementary area in the answer type prompt information according to the question supplementary prompt information in the answer type prompt information; fill the target question into the supplementary area to obtain the first question text.

21. The device according to claim 19, characterized in that The type prediction unit is further configured to determine that the target question text feature matches the reference data query type, and determine that the target answer type is the reference data query type when the target question text feature includes at least one of a chapter text feature, a character attribute text feature, and a character role event text feature; In a case where the target question text feature includes at least one of an event name text feature, a scene attribute text feature, and a non-character attribute word text feature, determining that the target question text feature matches a semantic similarity matching type, and determining that the target answer type is the semantic similarity matching type; When the matching degree between the target question text feature and the multimedia data is less than a preset matching degree threshold, the target answer type is determined to be a rejection reply type.

22. The device according to claim 19, characterized in that The content determination module comprises: A second concatenation unit is used to concatenate the target question and the element decomposition prompt information to obtain a second question text; A reference data prediction unit is used to input the second question text into a second data question-answering model, extract target question text features of a target question in the second question text through the second data question-answering model, and determine the target reference data that matches the target question text features from the reference data based on a logical relationship between the question text features and the reference data.

23. The device according to claim 22, characterized in that The first data question and answer model and the second data question and answer model are the same target data question and answer model. The data question and answer device also includes: a logic training unit, which is used to train the initial data question and answer model according to the data question and answer data set to obtain the target data question and answer model; the data question and answer data set includes a first data set for training the initial data question and answer model to determine the logical relationship of the answer type according to the question text and a second data set for training the initial data question and answer model to determine the logical relationship of the reference data according to the question text.

24. The device according to claim 22, characterized in that The second splicing unit is further used to supplement the prompt information according to the question in the element disassembly prompt information, determine the supplementary area in the element disassembly prompt information; fill the target question into the supplementary area to obtain the second question text.

25. The device according to claim 22, characterized in that The element disassembly prompt information includes: at least one of an event table, a summary table and a character setting table; the reference data prediction unit is also used to determine that the text feature of the target question matches the event table when the target question text feature includes a character name text feature, and determine that the target reference data is the event table; when the target question text feature includes a numeric range text feature, determine that the text feature of the target question matches the summary table, and determine that the target reference data is the summary table; when the target question text feature includes a character attribute text feature, determine that the text feature of the target question matches the character setting table, and determine that the target reference data is the character setting table.

26. A data question-answering model training device, characterized in that: The device comprises: A data set acquisition module is used to acquire a data question and answer data set; the data question and answer data set includes a question type data set and a question reference information data set; the question type data set includes questions and answer types matching the questions; the question reference information data set includes questions and answer data matching the questions; A model training module, used to train the initial data question answering model according to the data question answering dataset to obtain a corresponding target data question answering model; The target data question-answering model is used to determine the target answer type corresponding to the target question of the target multimedia data based on the logical relationship between the question text features represented by the answer type prompt information corresponding to the target multimedia data and the answer type. When the target answer type is a reference data query type, the target reference data of the target question is determined based on the logical relationship between the question text features represented by the element decomposition prompt information corresponding to the target multimedia data and the reference data, and the answer data corresponding to the target question is determined based on the target reference data; the answer data includes content obtained by extracting key information from the target multimedia data; the target reference data is an identifier of the answer content corresponding to the target question in the multimedia database, and is used to determine a method for searching the multimedia database for the target question.

27. The device according to claim 26, characterized in that The data set acquisition module further includes: A data segmentation unit, used for segmenting the target multimedia data to obtain a plurality of sub-target multimedia data; A question generation unit, used for inputting each of the sub-target multimedia data into a question generation model respectively, constructing a question for each of the sub-target multimedia data based on a plurality of question construction prompt words through the question generation model, and obtaining a plurality of first candidate questions for each of the sub-target multimedia data; a question type concatenation unit, configured to concatenate each of the first candidate questions and the answer type prompt information, and input the concatenation results into a plurality of large language models in parallel to obtain a plurality of candidate answer types corresponding to each of the first candidate questions; A first question screening unit is used to screen out first target questions that meet a first preset condition from the multiple first candidate questions according to the multiple candidate answer types corresponding to the first candidate questions, and determine an answer type that matches each of the first target questions from the multiple candidate answer types corresponding to the first target questions; The first question summarizing unit is used to summarize each of the first target questions and the answer type matching each of the first target questions to obtain the question type data set.

28. The device according to claim 27, characterized in that The first question screening unit is further used to: for any first candidate question, determine the quantitative proportions of different types of candidate answer types in the multiple candidate answer types corresponding to the first candidate question; when the maximum value among the quantitative proportions is not less than a first proportion threshold, determine that the first candidate question is the first target question, and determine the candidate answer type corresponding to the maximum value among the quantitative proportions as the answer type that matches the first target question.

29. The device according to claim 26, characterized in that The data set acquisition module further includes: A question determination unit, configured to determine, from a plurality of first target questions in the question type data set, a second candidate question whose answer type is a reference data query type; A question prompt concatenation unit, configured to concatenate each of the second candidate questions and the element decomposition prompt information, and input the concatenation results into a plurality of large language models in parallel to obtain a plurality of candidate reference data corresponding to each of the second candidate questions; A second question screening unit is used to screen out second target questions that meet a second preset condition from each of the second candidate questions according to the plurality of candidate reference data corresponding to each of the second candidate questions, and to determine reference data that matches each of the second target questions from the plurality of candidate reference data corresponding to each of the second target questions; The second question summarizing unit is used to summarize each of the second target questions and the reference data matching each of the second target questions to obtain the question reference information data set.

30. The device according to claim 29, characterized in that The second question screening unit is further used for: For any second candidate question, determining, from the plurality of candidate reference data corresponding to the second candidate question, a quantity ratio of different types of candidate reference data in the plurality of candidate reference data; When the maximum value among the quantity proportion values ​​is not less than the second ratio threshold, the second candidate question is determined to be the second target question, and the candidate reference data corresponding to the maximum value among the quantity proportion values ​​is determined as the reference data matching the second target question.

31. The device according to claim 26, characterized in that The model training module further includes: An analysis data set acquisition unit, used for acquiring a problem analysis data set; A data set concatenation unit, used for randomly mixing the data in the question type data set, the question reference information data set and the question analysis data set to obtain a mixed processing data set; The target model acquisition unit is used to input the mixed processing data set into the initial data question and answer model for training until the number of training rounds meets the preset number of rounds, the training is completed, and the target data question and answer model is obtained.

32. The device according to claim 31, characterized in that The analysis data set acquisition unit is further used to: combine the first target questions in the question type data set and the answer data matching each of the first target questions to obtain a plurality of candidate question result combinations; splice each of the candidate question result combinations and the question analysis prompt words, and input the splicing results into a plurality of large language models in parallel to obtain a plurality of candidate analysis texts corresponding to each of the candidate question result combinations; According to the multiple candidate analysis texts corresponding to each of the candidate question result combinations, a target question result combination that meets the third preset condition is screened out from the multiple candidate question result combinations, and an analysis text matching each of the target question result combinations is determined from the multiple candidate analysis texts corresponding to each of the target question result combinations; Each target question result combination and the analysis text matching each target question result combination are summarized to obtain the question analysis data set.

33. The device according to claim 32, characterized in that The analysis data set acquisition unit is also used to: for any candidate question result combination, encode multiple candidate analysis texts corresponding to the candidate question result combination, obtain candidate vectors of each candidate analysis text, and calculate the vector similarity between every two candidate vectors; when the minimum vector similarity among the vector similarities is not less than a vector similarity threshold, determine that the candidate question result combination is a target question result combination, and determine the analysis text that matches the target question result combination from the multiple candidate analysis texts corresponding to the target question result combination.

34. The device according to claim 33, characterized in that The analysis data set acquisition unit is also used to: calculate multiple vector similarities between the candidate vector of each candidate analysis text and the candidate vectors of other candidate analysis texts, and determine the minimum vector similarity corresponding to each candidate analysis text; from the minimum vector similarities corresponding to each candidate analysis text, screen out the target vector similarity that meets the fourth preset condition, and determine the candidate analysis text corresponding to the target vector similarity as the analysis text that matches the target question result combination.

35. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 17 are implemented.

36. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

37. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 17 are implemented.

Images