Corpus management method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and provides a corpus management method and device, equipment and a storage medium, the method comprises the following steps: obtaining at least one corpus management demand, the corpus management demand type is a newly added target corpus, and the corpus management demand comprises a corpus code, an answer type and a question of the target corpus; then, based on the question of each target corpus and the answer type of the target corpus in each corpus management demand, determining a target answer of each target corpus, the answer type of the target corpus comprising any one of the following items: an SQL type, an API type and a text type; based on the question of each target corpus and the target answer of each target corpus, writing each target corpus into a to-be-tested directory in batches, and testing each target corpus in the to-be-tested directory to obtain a test result of each target corpus; and under the condition that the test result of each target corpus is that the test is passed, each target corpus is written into the preset corpus, so that effective corpus management is realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a corpus management method, device, equipment and storage medium. Background Art

[0002] In user-oriented human-computer dialogue open platforms such as intelligent question-answering models, it is necessary to enter corresponding training corpora for different scenarios and intentions. The system then uses these training corpora to train the intelligent question-answering model, and uses the trained model to recognize the utterances input by the end user in the actual dialogue scenario.

[0003] Therefore, how to effectively manage the corpus has become a technical problem that needs to be urgently solved in the industry. Summary of the invention

[0004] The present invention provides a corpus management method, device, equipment and storage medium to achieve effective management of corpus.

[0005] In a first aspect, the present invention provides a corpus management method, the method comprising the following steps: Acquire at least one corpus management requirement; the corpus management requirement type is adding a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked of the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; Determine the target answer of each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using the target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0006] According to a corpus management method provided by the present invention, determining a target answer for each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus includes: In the case where it is determined that the answer type of the target corpus corresponding to each of the corpus management requirements is an SQL type, based on the corpus classification of each of the target corpora and the questions of each of the target corpora, a target sentence generation model is used to generate a first answer for each of the target corpora; wherein the target sentence generation model is a deep learning model obtained by training an initial sentence generation model using a large amount of sample data and labels corresponding to the sample data, the sample data is a corpus classification of multiple first corpora and multiple questions of the first corpora, the labels corresponding to the sample data are the answers to each of the first corpora, and the first corpora are of SQL type; Determining the accuracy corresponding to the first answer of each of the target corpora; When it is determined that the accuracy corresponding to the first answer of each target corpus meets the expected threshold, the first answer of each target corpus is determined as the answer of each target corpus.

[0007] According to a corpus management method provided by the present invention, determining a target answer for each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus includes: When it is determined that the answer type of the target corpus is an API type, matching is performed in the preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result; A target answer for the target corpus is determined based on the matching result.

[0008] According to a corpus management method provided by the present invention, determining a target answer for each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus includes: In the case where the answer type of the target corpus is determined to be a text type, the question of the text type target corpus is determined to be the answer of the text type target corpus.

[0009] According to a corpus management method provided by the present invention, the target corpora are batch-written into a directory to be tested based on the questions of the target corpora and the target answers of the target corpora, and the target corpora in the directory to be tested are tested to obtain the test results of the target corpora, including: Generate each of the target corpora based on the questions of each of the target corpora and the target answers of each of the target corpora; Batch updating each of the target corpora to the directory to be tested; Each of the target corpora in the to-be-tested directory is tested one by one to obtain a test result of each of the target corpora.

[0010] According to a corpus management method provided by the present invention, the method further includes: When the test result of each target corpus is that the test is passed, determining the corpus status of each target corpus as that the test is passed; The validity status of each of the target corpora is determined to be valid.

[0011] According to a corpus management method provided by the present invention, the method further includes: When the test result of each of the target corpora is a test failure, determining the corpus status of the target corpus as a test failure, and determining the validity status of the target corpus as invalid; generating a corpus repair task based on the corpus encoding and the type of test failure of the target corpus; The corpus repair task is assigned.

[0012] In a second aspect, the present invention further provides a corpus management device, the device comprising the following modules: An acquisition module is used to acquire at least one corpus management requirement; the corpus management requirement type is to add a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked by the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; A management module, for determining a target answer for each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using a target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0013] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the corpus management method described above is implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described corpus management methods.

[0015] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned corpus management methods.

[0016] The corpus management method, device, equipment and storage medium provided by the present invention first obtain at least one corpus management requirement; the corpus management requirement type is to add a new target corpus, wherein the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus and at least one of the following: data items of the target corpus, actual questions asked by the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the requirement of the target corpus and the date of submission of the requirement of the target corpus; The answer type includes any one of the following: SQL type, API type and text type; then, based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, the target answer of each target corpus is determined, wherein the answer type of the target corpus includes any one of the following: SQL type, API type and text type; further, based on the questions of each target corpus and the target answer of each target corpus, each target corpus is batch written into a directory to be tested, and each target corpus in the directory to be tested is tested to obtain a test result of each target corpus; then, when the test result of each target corpus is that the test passes, each target corpus is written into a preset corpus.

[0017] In the present invention, based on the corpus management requirements of the newly added target corpus, the target answer of each target corpus is determined by the question of each target corpus in each corpus management requirement and the answer type of the target corpus. The answer type of the target corpus includes any one of the following: SQL type, API type and text type. Then, each target corpus is written into a directory to be tested in batches and each target corpus in the directory to be tested is tested to obtain the test result of each target corpus. Then, when each target corpus passes the test, each target corpus is written into a preset corpus, thereby realizing effective management of the corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the corpus management method provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the corpus management requirements provided by the present invention.

[0021] Figure 3 This is the second flow chart of the corpus management method provided by the present invention.

[0022] Figure 4 This is the third flow chart of the corpus management method provided by the present invention.

[0023] Figure 5 This is the fourth flow chart of the corpus management method provided by the present invention.

[0024] Figure 6 It is a schematic diagram of the interface of task assignment provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the corpus management device provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Combine the following Figure 1-Figure 8 The present invention describes the corpus management method, device, equipment and storage medium.

[0029] Figure 1 It is one of the flow charts of the corpus management method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101, obtaining at least one corpus management requirement; the corpus management requirement type is adding a target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked in the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; Specifically, first of all, it should be noted that the execution subject of the present invention is an electronic device, which is used to achieve effective management of corpus.

[0030] First, obtain the corpus management requirements. The corpus management requirements refer to what kind of management operations are performed on the corpus, such as adding a new target corpus or modifying a target corpus. Usually, the corpus management requirements are submitted by the requirements personnel. The corpus management requirements include the corpus code of the target corpus (for example, 10001, 10002, 10003), the answer type of the target corpus (the present invention mainly introduces three types: SQL type, API type and text type), the question of the target corpus (that is, the actual problem corresponding to the target corpus), the corpus classification of the target corpus, the corpus status of the target corpus (for example, the test status), the effective status of the target corpus (valid or invalid), and the following At least one item: data item of target corpus, actual question of target corpus, question keywords of target corpus, additional corpus of target corpus, corpus parameter table of target corpus, constant maintenance table of target corpus, person in charge of target corpus requirements and date of target corpus requirements submission. The corresponding list fields need to be submitted: question code, data item, classification, answer type, question description, question, actual question, required keywords, additional corpus, corpus status, person in charge of requirements, date of submission of requirements, person in charge of middle office, date of submission of middle office, person in charge of data, whether it has been entered, date of entry, person in charge of test, latest test time, whether it is effective. The dwd_mt_ai_raw_corpus_v7 corpus list is shown in Table 1 below. Figure 2 is a schematic diagram of the corpus management requirements provided by the present invention, and the corpus management requirements obtained are as follows: Figure 2 As shown: Table 1:

[0031] Among them, dwd_mt_ai_raw_corpus_args_v7 (corpus parameter table) is shown in Table 2: Table 2:

[0032] Among them, sy_cd_mt_ct_sys_const (constant mapping table) is shown in Table 3: Table 3:

[0033] Step 102: based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, determine the target answer of each target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using the target sentence generation model, and the target sentence generation model is a deep learning model; Specifically, after obtaining the corpus management requirements of the newly added target corpus, the target answer of each target corpus needs to be determined according to the question of each target corpus in each corpus management requirement and the answer type of the target corpus.

[0034] The answer type of the target corpus includes any of the following: SQL (Structured Query Language) type, API type and text type. The SQL type target corpus means that the target expected answer is expressed in SQL statements, the Application Programming Interface (API) type target corpus means that the target expected answer is expressed in API, and the text type target corpus means that the target expected answer is expressed in text.

[0035] Step 103: Write each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and test each target corpus in the directory to be tested to obtain a test result of each target corpus; Specifically, it can be understood that the final target corpus can be obtained by storing the corresponding questions of the target corpus and the answers to the target corpus. After obtaining the answers to each target corpus, the target corpus is batch-written into the directory to be tested based on the questions of each target corpus and the target answers of each target corpus, that is, the batch running of each target corpus is realized. For example, through a pop-up window confirmation, the pop-up window information is "Confirm to enter the following 100 corpora to be entered?" Click to confirm to enter, the corpus status changes to [Entering], and the corpus status changes to [Entered] after completion.

[0036] Furthermore, each target corpus in the test directory can be tested to obtain the test results of each target corpus. The purpose of the test usually includes: whether the requirements meet the actual scenario, whether the generalization ability of the model meets the standards, and whether the generated answers are correct. The test results usually include two types: test failure and test pass.

[0037] Step 104: When the test result of each target corpus is that the test passes, each target corpus is written into a preset corpus.

[0038] Specifically, when the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus, that is, the target corpus is officially launched.

[0039] Correspondingly, when the test result of each target corpus is a test failure, the corpus is repaired and the above steps are repeated until the corpus is officially launched.

[0040] The method provided in this embodiment first obtains at least one corpus management requirement; the corpus management requirement type is to add a new target corpus, wherein the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked in the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the requirement of the target corpus, and the requirement of the target corpus. Request a date; then, determine the target answer of each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus, wherein the answer type of the target corpus includes any one of the following: SQL type, API type and text type; further, write each target corpus in batches into a directory to be tested based on the question of each target corpus and the target answer of each target corpus, test each target corpus in the directory to be tested to obtain a test result of each target corpus; and then, when the test result of each target corpus is that the test passes, write each target corpus into a preset corpus.

[0041] In the present invention, based on the corpus management requirements of the newly added target corpus, the target answer of each target corpus is determined by the question of each target corpus in each corpus management requirement and the answer type of the target corpus. The answer type of the target corpus includes any one of the following: SQL type, API type and text type. Then, each target corpus is written into a directory to be tested in batches and each target corpus in the directory to be tested is tested to obtain the test result of each target corpus. Then, when each target corpus passes the test, each target corpus is written into a preset corpus, thereby realizing effective management of the corpus.

[0042] According to a corpus management method provided by the present invention, based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, a target answer of each target corpus is determined, including: When it is determined that the answer type of the target corpus corresponding to each corpus management requirement is an SQL type, based on the corpus classification of each target corpus and the questions of each target corpus, a target sentence generation model is used to generate a first answer for each target corpus; wherein the target sentence generation model is a deep learning model obtained by training an initial sentence generation model using a large amount of sample data and labels corresponding to the sample data, the sample data is a corpus classification of multiple first corpora and multiple questions of the first corpora, the labels corresponding to the sample data are answers to each first corpus, and the first corpus is of SQL type; Determine the accuracy of the first answer of each target corpus; When it is determined that the accuracy corresponding to the first answer of each target corpus meets the expected threshold, the first answer of each target corpus is determined as the answer of each target corpus.

[0043] Specifically, SQL (Structured Query Language) is a programming language specifically used to manage and operate relational databases. It is widely used to store, retrieve, and modify data in databases. The main functions of SQL include: Querying: Retrieving data from a database.

[0044] Inserting: Adding new data to a database table.

[0045] Updating: Modifying data in the database.

[0046] Deleting: Removing data from a database table.

[0047] Creating: Create a new database and table.

[0048] Altering: Modifying the structure of an existing database, such as adding or removing columns from a table.

[0049] Dropping: Deleting a database or table.

[0050] Here are some basic SQL statement examples: Statement to query data: SELECT column1, column2 FROM table_name WHERE condition; Statement to insert data: INSERT INTO table_name (column1, column2) VALUES (value1, value2); Statement to update data: UPDATE table_name SET column1 = value1 WHERE condition; SQL statements can contain multiple clauses, such as WHERE, GROUP BY, ORDER BY, etc., to implement complex queries and data operations. In addition, SQL also supports advanced functions such as transaction processing, permission control and backup.

[0051] In some embodiments, in step 102, based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, the target answer of each target corpus is determined, including: When it is determined that the answer type of the target corpus corresponding to each corpus management requirement is the SQL type, an example of the process of generating the answer of the target corpus is as follows: Firstly, based on the corpus classification of each target corpus and the questions of each target corpus, a target sentence generation model is used to generate the first answer of each target corpus.

[0052] Among them, the target sentence generation model is a deep learning model. Deep learning models are a type of machine learning algorithms inspired by the structure of the human brain. They use multi-layer neural networks to learn complex patterns and representations of data. Deep learning models have achieved remarkable success in image recognition, speech recognition, natural language processing, game intelligence and other fields. Common types of deep learning models include: Convolutional Neural Network (CNN): Suitable for image processing, capable of extracting spatial hierarchical features in images. Recurrent Neural Network (RNN): Suitable for sequence data, such as time series analysis and natural language processing. Long Short-Term Memory Network (LSTM): A special RNN that can learn long-distance dependencies. Gated Recurrent Unit (GRU): A variant of LSTM with fewer parameters and faster training. Transformer: A model based on the self-attention mechanism, widely used in natural language processing tasks such as machine translation. Generative Adversarial Network (GAN): Consists of a generator and a discriminator, used to generate new data samples.

[0053] The target sentence generation model is obtained by training the initial sentence generation model using a large amount of sample data and labels corresponding to the sample data. The sample data is a corpus classification of multiple first corpora and multiple questions of the first corpora. The labels corresponding to the sample data are the answers to each first corpus, and the first corpus is an SQL type. The training process of the initial sentence generation model requires a large amount of sample data and computing resources, and is usually accelerated using a GPU or TPU. The trained target sentence generation model has a high accuracy in determining the first answer of the target expected SQL type.

[0054] For example, the model is a text-to-SQL model trained based on preset database data and domain knowledge. When the text-to-SQL interface is called, the interface will use the target statement generation model to generate the corresponding SQL statement according to the user's question, that is, generate the first answer of each corresponding target corpus.

[0055] Furthermore, after obtaining the first answer of the SQL type expected by the model generation target, the accuracy of the first answer corresponding to each target corpus can be further determined. The model accuracy is obtained by automatically scoring the generated SQL based on the user's question, recalling relevant table column information through the domain knowledge base, and combining it with the current needs.

[0056] Furthermore, when it is determined that the accuracy corresponding to the first answer of each target corpus meets the expected threshold (the accuracy threshold corresponding to the actual demand), the first answer of each target corpus is determined as the answer of each target corpus. On the contrary, when it is determined that the accuracy corresponding to the first answer of each target corpus does not meet the expected threshold, the target sentence generation model can be reused to generate the first answer of the target corpus.

[0057] Figure 3 This is the second flow chart of the corpus management method provided by the present invention. Figure 3 As shown, the method includes: Requirements: Submit the requirements for adding / modifying corpus to the middle office; Middle platform: The middle platform determines the accuracy of the answers generated by the model; marks them for entry; runs batches (updates to the intranet); and automatically transfers them to the directory to be tested; Testing: Test each corpus; if the test passes and the status is valid, go online directly; if the test fails and the status is invalid, assign the task of repairing the corpus to the corresponding process.

[0058] The method provided in this embodiment, when determining that the answer type of the target corpus corresponding to each corpus management requirement is the SQL type, generates the first answer of each target corpus based on the corpus classification of each target corpus and the question of each target corpus using the target sentence generation model; wherein the target sentence generation model is a deep learning model; then, the accuracy corresponding to the first answer of each target corpus is determined; further, when determining that the accuracy corresponding to the first answer of each target corpus meets the expected threshold, the first answer of each target corpus is determined as the answer of each target corpus. The present invention realizes the generation of target corpus based on the model and realizes the effective management of corpus.

[0059] According to a corpus management method provided by the present invention, based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, a target answer of each target corpus is determined, including: When it is determined that the answer type of the target corpus is the API type, matching is performed in a preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result; The target answer of the target corpus is determined based on the matching results.

[0060] Specifically, an API is a software component that allows different software applications to interact with each other. The API defines the request format, transmission method, data structure, and operation rules, allowing developers to access the functions or data of a software or service without having to understand its internal workings. The types of APIs include the following: Local API: API used within a single application, such as the API provided by the operating system.

[0061] Network API: APIs that communicate over the network, such as RESTful API, SOAP, GraphQL, etc., which allow data to be exchanged between different systems.

[0062] Remote API: An API that allows remote access, typically used in a client-server architecture.

[0063] Web API: API based on HTTP protocol, allowing access through the Internet.

[0064] In some embodiments, the specific implementation process of step 102 of determining the target answer of each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus includes the following steps: First, when the answer type of the target corpus is determined to be an API type, a match is performed in a preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result. Among them, the matching rules can be preset according to the actual situation, and the matching process can be implemented by a program or manually confirmed.

[0065] For example, when the target corpus is an API corpus, the process of generating the first answer is as follows: 1) Locate the platform name of the problem source based on the "data item" field of the target corpus; 2) Confirm the specific module of the source platform based on the problem description; 3) Match the corresponding API service and determine the corresponding API service as the first answer of the API corpus.

[0066] Furthermore, after obtaining the matching result of the target corpus, the target answer of the target corpus can be determined based on the matching result. For example, the matching result is determined as the target answer of the target corpus, and then the question of the target corpus and the matching result of the target corpus can be stored in correspondence.

[0067] Figure 4 This is the third flow chart of the corpus management method provided by the present invention, such as Figure 4 As shown, the method includes: Requirements: Submit the requirements for adding / modifying corpus to the middle office; Middle platform: The middle platform supplements the answers; marks to be entered; runs the batch (updates to the intranet); automatically transfers to the directory to be tested; Testing: Test each corpus; if the test passes and the status is valid, go online directly; if the test fails and the status is invalid, assign the task of repairing the corpus to the corresponding process.

[0068] The method provided in this embodiment, when determining that the answer type of the target corpus is the API type, performs matching in a preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result; then, the target answer of the target corpus is determined based on the matching result. The present invention realizes effective management of the corpus.

[0069] According to a corpus management method provided by the present invention, based on the questions of each target corpus in each corpus management requirement and the answer type of the target corpus, a target answer of each target corpus is determined, including: In the case where the answer type of the target corpus is determined to be a text type, the question of the text type target corpus is determined to be the answer of the text type target corpus.

[0070] Specifically, in some embodiments, the specific implementation process of step 102 of determining the target answer of each target corpus includes the following steps: When the answer type of the target corpus is determined to be a text type, the question of the text type target corpus is determined to be the answer of the text type target corpus. The text type answer can also be understood as a knowledge type.

[0071] Figure 5 FIG. 4 is a flow chart of the corpus management method provided by the present invention. Figure 5 As shown, the method includes: Requirements: Mark the newly added / modified corpus for entry; run the batch (update to the intranet); automatically transfer to the directory to be tested; Testing: Test each corpus; if the test passes and the status is valid, go online directly; if the test fails and the status is invalid, assign the task of repairing the corpus to the corresponding process (requirement).

[0072] The method provided in this embodiment, when determining that the answer type of the target corpus is a text type, determines the questions of the target corpus of text type as the answers of the target corpus of text type, thereby effectively managing the corpus.

[0073] According to a corpus management method provided by the present invention, each target corpus is batch-written into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and each target corpus in the directory to be tested is tested to obtain the test results of each target corpus, including: Generate each target corpus based on the questions of each target corpus and the target answers of each target corpus; Update each target corpus in batches to the directory to be tested; Test each target corpus in the test directory one by one to obtain the test results of each target corpus.

[0074] Specifically, in some embodiments, in step 103, each target corpus is batch-written into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and each target corpus in the directory to be tested is tested to obtain a test result of each target corpus, including: First, based on the questions of each target corpus and the target answers of each target corpus, each target corpus is generated. It can be understood that the core content of the target corpus includes questions and answers, which correspond one to one, which facilitates the subsequent training of the intelligent question-answering model and makes the intelligent question-answering model more accurate.

[0075] After obtaining the answers for each target corpus, each target corpus is updated in batches to the directory to be tested, that is, batch running is realized (for example, updating to a specified storage path).

[0076] Optionally, the generation of exponential generalization sets can also be achieved based on the corpus parameters and corresponding answers of each target corpus, so as to facilitate the rapid generation of large-scale corpora.

[0077] Further, each target corpus in the test directory is tested one by one to obtain the test results of each target corpus. Testing a corpus usually refers to using natural language processing (NLP) technology to evaluate and verify the quality, consistency or performance of a specific model of the corpus. The following are the general steps for testing a corpus: 1. Define the test objectives: clarify the purpose of the test, such as the coverage, consistency, ambiguity, error rate, etc. of the test corpus.

[0078] 2. Data preprocessing: Cleaning data, including removing useless information, standardizing text, word segmentation, removing stop words, etc.

[0079] 3. Label the data. If necessary, manually or automatically label the corpus.

[0080] 4. Select or develop test tools: Select appropriate NLP tools or develop customized test scripts based on the test objectives.

[0081] 5. Sample extraction: Randomly select a certain number of samples from the entire corpus for testing to ensure the representativeness of the results.

[0082] 6. Execute test: Run the test tool or script to test the extracted samples.

[0083] 7. Iterative improvement: Iteratively improve the model or the corpus based on the test results.

[0084] 8. Documentation: Record the test process and results to provide reference for future testing and improvements.

[0085] 9. Continuous Monitoring: The corpus is continuously monitored to ensure that the newly added data meets the quality standards.

[0086] The specific implementation details of each step will vary depending on the characteristics of the corpus and the test objectives. For example, if you are testing the output of a machine translation system, you may need to focus on the accuracy, fluency, and consistency of the translation.

[0087] Test results usually include two types: test passed or test failed.

[0088] The method provided in this embodiment first generates each target corpus based on the questions of each target corpus and the target answers of each target corpus; then, updates each target corpus in batches to a directory to be tested; and tests each target corpus in the directory to be tested one by one to obtain the test results of each target corpus.

[0089] According to a corpus management method provided by the present invention, the method further includes: When the test result of each target corpus is that the test has passed, the corpus status of each target corpus is determined as that the test has passed; The validity status of each target corpus is determined to be valid.

[0090] Specifically, in some embodiments, the corpus state generally includes any of the following:

Add new corpus

[0091] [Answer to be supplemented]: Click to submit the middle platform.

[0092] [To be entered]: Click the button to mark product / middle platform / data as to be entered.

[0093]

Entering

[0094] [Already entered]: After clicking the enter button and completing the entry.

[0095] [Entry failed]: After clicking the entry button, the entry fails. [If there is no success response within 10 hours, it means timeout, and the entry fails automatically after the timeout]

Test passed

[0096] [Test Failure]: Test click test failure and assign.

[0097] [Deleted]: After clicking Delete, the Deleted option is not checked by default.

[0098] Correspondingly, the method further includes: When the test result of each target corpus is that the test is passed, the corpus status of each target corpus is determined as the test is passed, and the validity status of each target corpus is determined as valid.

[0099] After that, the target corpus that has passed the test can be officially launched online, for example, written into a preset corpus.

[0100] The method provided in this embodiment determines the corpus status of each target corpus as passed and the validity status of each target corpus as valid when the test result of each target corpus is passed, thereby achieving effective management of the corpus.

[0101] According to a corpus management method provided by the present invention, the method further includes: When the test result of each target corpus is a test failure, the corpus state of the target corpus is determined as a test failure, and the validity state of the target corpus is determined as invalid; Generate corpus repair tasks based on the corpus encoding and test failure types of the target corpus; Assign the corpus repair task.

[0102] Specifically, in some embodiments, the method further includes: First, when the test result of each target corpus is a test failure, the corpus status of the target corpus is determined to be a test failure, and the validity status of the target corpus is determined to be invalid.

[0103] Furthermore, based on the corpus encoding of the target corpus and the type of test failure, a corpus repair task is generated, and the corpus repair task is assigned.

[0104] Among them, the assignment to the corresponding process needs to be determined according to the type of failure of the target corpus test. For example, if the question of the target corpus is not clearly stated or incorrectly stated, it will be assigned to the product responsible department. If the answer to the target corpus does not match or is incorrect, it will be assigned to the middle station to regenerate the answer. Figure 6 This is a schematic diagram of the interface for task assignment provided by the present invention. Click the button to pop up a pop-up window, select Assign to (required), and fill in the assignment description. The corpus status changes to: Test failed; Validity changes to: Invalid. Record the number of click failures. Generate a new history record.

[0105] The method provided in this embodiment, when the test result of each target corpus is a test failure, determines the corpus status of the target corpus as a test failure, and determines the effective status of the target corpus as invalid; then, based on the corpus encoding of the target corpus and the type of test failure, generates a corpus repair task, and assigns the corpus repair task. The present invention realizes effective management of corpus.

[0106] Optionally, the corpus state flow of the target corpus in this embodiment is as follows: Case 1: Newly added SQL / API corpus: Click to add new corpus [Add new corpus] -> Click to submit to the middle platform [Answers to be added] -> Click to submit for entry [To be entered] -> Click to run the batch [Entering] -> [Entered] -> [Test passed / Test failed].

[0107] Click to add new corpus [Add new corpus] -> Click to submit to the middle platform [Answers to be supplemented] -> Click to submit for entry [Waiting for entry] -> Click to run the batch [Entering] -> [Entry failed].

[0108] Case 2: New knowledge corpus: Click to add a new corpus [Add a new corpus] -> click Submit for entry [To be entered] -> click Run Batch [Entering] -> [Entered] -> [Test passed / Test failed].

[0109] Click to add new corpus [Add new corpus] -> click Submit for entry [Waiting for entry] -> click Run batch [Entering] -> [Entry failed].

[0110] Case 3: Test failure assigned to requirement: Click on the test failure and assign [Test failed] -> Click on the submit center [Answers to be supplemented] > Click on submit to be entered [To be entered] -> Click on run batch [Entering] -> [Entered] -> [Test passed / Test failed].

[0111] Case 4: Test failure is assigned to the middle platform: Click on the test failure and assign [Test failed] -> click Save in the middle platform [Answers to be supplemented] -> click Submit to be entered [To be entered] -> click Run batch [Entering] -> [Already entered] -> [Test passed / Test failed].

[0112] Case 5: Test passed requirement modification: Click to submit the middle platform [Answers to be supplemented] -> Click to submit for entry [To be entered] -> Click to run the batch [Entering] -> [Already entered] -> [Test passed / Test failed].

[0113] Case 6: The test passed the middle platform modification: Click Submit to wait for entry [Waiting for entry] -> Click Run Batch [Entering] -> [Entered] -> [Test passed / Test failed].

[0114] The corpus management device provided by the present invention is described below. The corpus management device described below and the corpus management method described above can be referenced to each other.

[0115] Figure 7 is a schematic diagram of the structure of the corpus management device provided by the present invention, such as Figure 7 As shown, the corpus management device 700 includes the following modules: The acquisition module 710 is used to acquire at least one corpus management requirement; the corpus management requirement type is to add a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked by the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; The management module 720 is used to determine the target answer of each target corpus based on the question of each target corpus in each corpus management requirement; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using the target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0116] The device provided by this embodiment, first, the acquisition module 710 is used to obtain at least one corpus management requirement; the corpus management requirement type is to add a new target corpus, wherein the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus and at least one of the following: the data item of the target corpus, the actual question of the target corpus, the question keywords of the target corpus, the additional corpus of the target corpus, the corpus parameter table of the target corpus, the constant maintenance table of the target corpus, the person in charge of the target corpus and the date of the request of the target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; then, the management module 720 is used to determine the target answer of each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus, wherein the answer type of the target corpus includes any one of the following: SQL type, API type and text type, and further, based on the question of each target corpus and the target answer of each target corpus, each target corpus is batch written into a to-be-tested directory, and each target corpus in the to-be-tested directory is tested to obtain a test result of each target corpus; and then, when the test result of each target corpus is a pass, each target corpus is written into a preset corpus.

[0117] In the present invention, based on the corpus management requirements of the newly added target corpus, the target answer of each target corpus is determined by the question of each target corpus in each corpus management requirement and the answer type of the target corpus. The answer type of the target corpus includes any one of the following: SQL type, API type and text type. Then, each target corpus is written into a directory to be tested in batches and each target corpus in the directory to be tested is tested to obtain the test result of each target corpus. Then, when each target corpus passes the test, each target corpus is written into a preset corpus, thereby realizing effective management of the corpus.

[0118] According to a corpus management device 700 provided by the present invention, the management module 720 is specifically used for: In the case where it is determined that the answer type of the target corpus corresponding to each of the corpus management requirements is an SQL type, based on the corpus classification of each of the target corpora and the questions of each of the target corpora, a target sentence generation model is used to generate a first answer for each of the target corpora; wherein the target sentence generation model is a deep learning model obtained by training an initial sentence generation model using a large amount of sample data and labels corresponding to the sample data, the sample data is a corpus classification of multiple first corpora and multiple questions of the first corpora, the labels corresponding to the sample data are the answers to each of the first corpora, and the first corpora are of SQL type; Determining the accuracy corresponding to the first answer of each of the target corpora; When it is determined that the accuracy corresponding to the first answer of each target corpus meets the expected threshold, the first answer of each target corpus is determined as the answer of each target corpus.

[0119] According to a corpus management device 700 provided by the present invention, the management module 720 is further used for: When it is determined that the answer type of the target corpus is an API type, matching is performed in the preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result; A target answer for the target corpus is determined based on the matching result.

[0120] According to a corpus management device 700 provided by the present invention, the management module 720 is further used for: In the case where the answer type of the target corpus is determined to be a text type, the question of the text type target corpus is determined to be the answer of the text type target corpus.

[0121] According to a corpus management device 700 provided by the present invention, the management module 720 is further used for: Generate each of the target corpora based on the questions of each of the target corpora and the target answers of each of the target corpora; Batch updating each of the target corpora to the directory to be tested; Each of the target corpora in the to-be-tested directory is tested one by one to obtain a test result of each of the target corpora.

[0122] According to a corpus management device 700 provided by the present invention, the management module 720 is further used for: When the test result of each target corpus is that the test is passed, determining the corpus status of each target corpus as that the test is passed; The validity status of each of the target corpora is determined to be valid.

[0123] According to a corpus management device 700 provided by the present invention, the management module 720 is further used for: When the test result of each of the target corpora is a test failure, determining the corpus status of the target corpus as a test failure, and determining the validity status of the target corpus as invalid; generating a corpus repair task based on the corpus encoding and the type of test failure of the target corpus; The corpus repair task is assigned.

[0124] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communications interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the corpus management method, which includes: Acquire at least one corpus management requirement; the corpus management requirement type is adding a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked of the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; Based on the questions of each target corpus in each corpus management requirement, a target answer for each target corpus is determined; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using a target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0125] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0126] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the corpus management method provided by the above methods, the method includes: Acquire at least one corpus management requirement; the corpus management requirement type is adding a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked of the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; Based on the questions of each target corpus in each corpus management requirement, a target answer for each target corpus is determined; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using a target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0127] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the corpus management method provided by the above methods is implemented, and the method includes: Acquire at least one corpus management requirement; the corpus management requirement type is adding a new target corpus; the corpus management requirement includes the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked of the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the target corpus requirement, and date of request of the target corpus; Based on the questions of each target corpus in each corpus management requirement, a target answer for each target corpus is determined; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using a target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

[0128] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A corpus management method, characterized in that: include: Obtain at least one corpus management requirement; The requirement type of the corpus management is to add a new target corpus; The corpus management requirements include the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked by the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the requirements of the target corpus, and the date of the requirement submission of the target corpus; Determine the target answer of each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using the target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

2. The corpus management method according to claim 1, characterized in that: The questions of each target corpus in each corpus management requirement and the answer type of the target corpus include: In the case where it is determined that the answer type of the target corpus corresponding to each of the corpus management requirements is of SQL type, based on the corpus classification of each of the target corpora and the questions of each of the target corpora, the first answer of each of the target corpora is generated by using the target sentence generation model; wherein the target sentence generation model is obtained by training an initial sentence generation model using a large amount of sample data and labels corresponding to the sample data, the sample data is corpus classification of multiple first corpora and multiple questions of the first corpora, the labels corresponding to the sample data are the answers of each of the first corpora, and the first corpora are of SQL type; Determining the accuracy corresponding to the first answer of each of the target corpora; When it is determined that the accuracy corresponding to the first answer of each target corpus meets the expected threshold, the first answer of each target corpus is determined as the answer of each target corpus.

3. The corpus management method according to claim 1, characterized in that: The questions of each target corpus in each corpus management requirement and the answer type of the target corpus include: When it is determined that the answer type of the target corpus is an API type, matching is performed in the preset corpus based on the corpus classification of the target corpus and the keywords of the target corpus to obtain a matching result; A target answer for the target corpus is determined based on the matching result.

4. The corpus management method according to claim 1, characterized in that: The questions of each target corpus in each corpus management requirement and the answer type of the target corpus include: In the case where the answer type of the target corpus is determined to be a text type, the question of the text type target corpus is determined to be the answer of the text type target corpus.

5. The corpus management method according to claim 1, characterized in that: The step of writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus includes: Generate each of the target corpora based on the questions of each of the target corpora and the target answers of each of the target corpora; Batch updating each of the target corpora to the directory to be tested; Each of the target corpora in the to-be-tested directory is tested one by one to obtain a test result of each of the target corpora.

6. The corpus management method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the test result of each target corpus is that the test is passed, determining the corpus status of each target corpus as that the test is passed; The validity status of each of the target corpora is determined to be valid.

7. The corpus management method according to any one of claims 1 to 5, characterized in that: The method further comprises: When the test result of each of the target corpora is a test failure, determining the corpus status of the target corpus as a test failure, and determining the validity status of the target corpus as invalid; generating a corpus repair task based on the corpus encoding and the type of test failure of the target corpus; The corpus repair task is assigned.

8. A corpus management device, characterized in that: include: An acquisition module, used to acquire at least one corpus management requirement; The requirement type of the corpus management is to add a new target corpus; The corpus management requirements include the corpus code of the target corpus, the answer type of the target corpus, the question of the target corpus, the corpus classification of the target corpus, the corpus status of the target corpus, the effective status of the target corpus, and at least one of the following: data items of the target corpus, actual questions asked by the target corpus, question keywords of the target corpus, additional corpus of the target corpus, corpus parameter table of the target corpus, constant maintenance table of the target corpus, person in charge of the requirements of the target corpus, and the date of the requirement submission of the target corpus; A management module, configured to determine a target answer for each target corpus based on the question of each target corpus in each corpus management requirement and the answer type of the target corpus; the answer type of the target corpus includes any one of the following: SQL type, API type and text type; the first answer of the SQL type target corpus is generated by using a target sentence generation model, and the target sentence generation model is a deep learning model; Writing each target corpus in batches into a directory to be tested based on the questions of each target corpus and the target answers of each target corpus, and testing each target corpus in the directory to be tested to obtain a test result of each target corpus; When the test result of each target corpus is that the test is passed, each target corpus is written into a preset corpus.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the corpus management method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the corpus management method according to any one of claims 1 to 7 is implemented.

Images