Corpus dynamic acquisition and quality evaluation method based on MCP

Abstract

The invention relates to the technical field of natural language processing, and discloses an MCP-based corpus dynamic acquisition and quality evaluation method, which comprises the following steps of: 1, acquiring an acquisition demand of a user; 2, calling an external collection tool through an MCP protocol based on a user collection demand; 3, the collection tool directionally analyzes the target website according to the collection requirement of the user, and webpage content highly related to the user requirement is obtained; 4, preliminarily preprocessing the collected webpage content to generate preliminary corpus data; and step 5, performing quality evaluation on the generated preliminary corpus data, and dynamically adjusting an acquisition strategy according to a quality evaluation result, so that the method has multiple advantages of efficient acquisition, dynamic adjustment, quality evaluation, reduction of labor cost and high adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for dynamic corpus collection and quality assessment based on MCP. Background Art

[0002] In today's era of rapid development of artificial intelligence (AI), natural language processing (NLP), as one of its core branches, is gradually transforming how humans interact with machines. From intelligent voice assistants to automated text translation, to complex sentiment analysis and semantic understanding, NLP's applications are becoming increasingly widespread and in-depth. Corpora play a cornerstone role in this process. With advances in computer science and the advent of the big data era, massive amounts of text data can be collected, stored, and analyzed, providing unprecedented opportunities for corpus construction. Furthermore, the interdisciplinary integration of linguistics, computer science, statistics, and other disciplines has driven continuous innovation in corpus processing methods and technologies, enabling them to more accurately serve the training and optimization of NLP models. However, faced with the complex and ever-changing linguistic phenomena and diverse application scenarios, ensuring the quality and quantity of corpora to meet the development needs of NLP technology remains a major challenge in current research, making the background research on corpora increasingly critical and urgent.

[0003] Traditional corpus collection methods mainly rely on manual annotation and fixed crawler programs. These methods have many shortcomings:

[0004] High cost of manual annotation: Manual annotation of corpus requires a lot of manpower and time, and is prone to inconsistent annotation.

[0005] Crawler programs have poor flexibility: Traditional crawler programs can usually only collect predefined websites and data formats, and are difficult to adapt to dynamically changing web content.

[0006] The quality of the corpus varies greatly: the collected corpus often needs to go through multiple rounds of screening and cleaning before it can be used for model training, which increases the complexity of data processing.

[0007] Lack of dynamic evaluation mechanism: Traditional corpus collection methods lack a real-time evaluation mechanism for the quality of collected corpora and are unable to promptly eliminate low-quality corpora. Summary of the Invention

[0008] (1) Technical problems solved

[0009] In response to the shortcomings of the existing technology, the present invention provides a dynamic corpus collection and quality assessment method based on MCP. It has the advantages of achieving dynamic optimization of the corpus through automated demand analysis, flexible scheduling of multi-source collection tools, and combining real-time semantic relevance, information integrity and data accuracy assessment, reducing manual annotation costs and improving collection efficiency; eliminating low-quality data through dynamic adjustment strategies to ensure corpus quality; supporting multi-dimensional evaluation to adapt to complex scenario requirements, and can quickly build a structured, high-quality corpus to provide a reliable data foundation while reducing redundant collection and data processing complexity.

[0010] (2) Technical solution

[0011] To achieve the above object, the present invention provides the following technical solution: a method for dynamic corpus collection and quality assessment based on MCP, comprising the following steps:

[0012] Step 1: Obtain user collection requirements;

[0013] Step 2: Based on user collection requirements, call external collection tools through the MCP protocol;

[0014] Step 3: The collection tool analyzes the target website according to the user's collection needs and obtains web page content that is highly relevant to the user's needs;

[0015] Step 4: Pre-process the collected web page content to generate preliminary corpus data;

[0016] Step 5: Perform a quality assessment on the generated preliminary corpus data and dynamically adjust the collection strategy based on the quality assessment results.

[0017] Preferably, in step 1, user requirements include target domain, corpus type, and collection frequency.

[0018] Preferably, in step 2, calling external collection tools includes but is not limited to web crawlers, API interface calls, and file downloaders.

[0019] Preferably, in step 4, the preliminary preprocessing includes removing HTML tags by regular expressions; extracting text content by using tags; and performing word segmentation by using Jieba.

[0020] Preferably, in step 5, the specific method of quality assessment includes:

[0021] A1. Calculate the semantic similarity between the collected preliminary corpus data and user requirements to evaluate text relevance;

[0022] A2. Evaluate the completeness of preliminary corpus data;

[0023] A3. Evaluate the accuracy of the preliminary corpus data based on known accurate data;

[0024] A4. Calculate an overall quality score based on text relevance, information completeness, and data accuracy.

[0025] Preferably, the calculation formula of the text relevance is:

[0026]

[0027] In the calculation formula, Similaty represents text relevance, A represents the keyword set of user needs, B represents the keyword set of the collected corpus, |A∩B| represents the intersection of the keywords A and B, that is, the number of keywords that appear together in the two sets, and |A∪B| represents the union of the keywords A and B, that is, the number of all non-repeated keywords in the two sets.

[0028] Preferably, the calculation formula for the information integrity is:

[0029]

[0030] In the calculation formula, Completeness represents information completeness, Number of Key Information represents the actual number of key information collected, that is, the number of key information items in the corpus that meet the requirements, and Total Key Information represents the total number of key information required by users, that is, the number of all key information items that need to be collected.

[0031] Preferably, the calculation formula for the data accuracy is:

[0032]

[0033] In the calculation formula, Accuracy represents data accuracy, Number of Correct Information represents the amount of correct information in the corpus, that is, the number of information items that meet user needs or actual situations, and Total Information represents the total amount of all information in the corpus, including correct information and incorrect information.

[0034] Preferably, the calculation formula for the comprehensive quality score is:

[0035] Score=ω1*Similaty+ω2*Completeness+ω3*Accuracy

[0036] In the calculation formula, Score represents the comprehensive quality score, ω1 represents the weight of text relevance, ω2 represents the weight of information completeness, and ω3 represents the weight of data accuracy, ω1+ω2+ω3=1.

[0037] Preferably, the dynamic adjustment of the acquisition strategy is implemented as follows:

[0038] B1. Set a comprehensive quality score threshold and compare the calculated comprehensive quality score with the quality score threshold;

[0039] B2. Extract and save the corpus with a comprehensive quality score higher than or equal to the comprehensive quality score threshold;

[0040] B3. Extract and remove the corpus whose comprehensive quality score is lower than the comprehensive quality score threshold.

[0041] Compared with the existing technology, the present invention provides a dynamic corpus collection and quality assessment method based on MCP, which has the following beneficial effects:

[0042] 1. This invention achieves dynamic optimization of the corpus, reduces manual annotation costs, and improves collection efficiency through automated demand analysis, flexible scheduling of multi-source collection tools, and real-time semantic relevance, information integrity, and data accuracy assessment. It also eliminates low-quality data through dynamic adjustment strategies to ensure corpus quality. It supports multi-dimensional evaluation to adapt to complex scenario requirements, can quickly build a structured, high-quality corpus, provide a reliable data foundation, and reduce redundant collection and data processing complexity.

[0043] 2. The present invention has multiple advantages, including efficient collection, dynamic adjustment, quality assessment, reduced labor costs, and strong adaptability, which are specifically manifested in:

[0044] Efficient collection: By calling multiple external collection tools through the MCP protocol, a large amount of corpus related to user needs can be collected quickly and efficiently.

[0045] Dynamic adjustment: Based on the quality feedback of the collected corpus, the collection strategy is dynamically adjusted to improve the collection efficiency and corpus quality.

[0046] Quality assessment: A comprehensive quality assessment system has been established to evaluate the quality of collected corpora in real time, ensuring that all corpora stored in the corpus are of high quality.

[0047] Reduce labor costs: It reduces the workload of manual annotation and data cleaning, and reduces the labor cost of corpus construction.

[0048] Strong adaptability: It can adapt to the corpus collection needs of different fields and has strong versatility and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a step diagram of the method of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] See also Figure 1 ,The corpus dynamic collection and quality assessment method based on MCP,includes the following steps:

[0052] Step 1: Obtain user collection requirements, including target domain, corpus type, and collection frequency;

[0053] By clarifying user needs, we ensure that the collected corpus is highly relevant to the target domain, avoid redundant data, and improve collection efficiency. At the same time, we provide a benchmark for subsequent quality assessment, ensuring the accuracy and practicality of the corpus. In addition, by dynamically responding to changes in user needs, we can flexibly adjust the collection strategy, reduce resource waste, enhance the system's adaptability and user experience, and ultimately build a high-quality dynamic corpus.

[0054] Step 2: Based on user collection needs, call external collection tools through the MCP protocol, including but not limited to web crawlers, API interface calls, and file downloaders;

[0055] The MCP protocol integrates tools such as web crawlers, API interfaces, and file downloaders to efficiently collect multi-source data, ensuring the timeliness and freshness of the corpus. It also standardizes the data collection process and format, providing a structured and reliable data foundation for subsequent quality assessment. Furthermore, the protocol dynamically adapts to collection frequency and tool selection, lowering technical barriers and resource consumption, improving the system's flexibility and scalability, and ultimately ensuring the high-quality construction and continuous updating of the corpus.

[0056] Step 3: The collection tool analyzes the target website according to the user's collection needs and obtains web page content that is highly relevant to the user's needs;

[0057] By combining user-defined target domains and corpus types, the collection tool can focus on parsing relevant web content, avoiding interference from irrelevant data and significantly improving the relevance and purity of collected data. Precise collection based on user needs reduces redundant data, lowers storage and computing costs, and ensures that corpus content is highly consistent with evaluation indicators, thereby supporting more efficient dynamic updates and quality optimization.

[0058] Step 4: Pre-process the collected web page content to generate preliminary corpus data;

[0059] The initial preprocessing includes removing HTML tags through regular expressions; extracting text content using tags; and using Jieba for word segmentation.

[0060] By removing HTML tags and irrelevant code through regular expressions, pure text content can be extracted, preventing noise data from interfering with subsequent analysis. Using tags to extract text content can accurately locate the target text and improve data relevance. Combined with Jieba word segmentation, the text is segmented into word units, providing a structured foundation for subsequent quality assessment, ensuring the standardization and consistency of the corpus data, while reducing the burden of redundant information on storage and computing, laying a high-quality data foundation for dynamic updates and efficient evaluation.

[0061] Step 5: Conduct a quality assessment on the generated preliminary corpus data and dynamically adjust the collection strategy based on the quality assessment results;

[0062] Specific methods for quality assessment include:

[0063] A1. Calculate the semantic similarity between the collected preliminary corpus data and user requirements to evaluate text relevance. The calculation formula is:

[0064]

[0065] In the calculation formula, Similaty represents text relevance, A represents the keyword set of user needs, B represents the keyword set of the collected corpus, |A∩B| represents the intersection of keywords A and B, that is, the number of keywords that appear together in the two sets, and |A∪B| represents the union of keywords A and B, that is, the number of all non-repeated keywords in the two sets.

[0066] For example, if the user demand is "traffic congestion" and the collected corpus contains two keywords "traffic" and "congestion", the text relevance is 1;

[0067] A2. Evaluate the completeness of the preliminary corpus data. The calculation formula is:

[0068]

[0069] In the calculation formula, Completeness represents information completeness, Number of Key Information represents the actual number of key information collected, that is, the number of key information items in the corpus that meet the requirements, and Total Key Information represents the total number of key information required by the user, that is, the number of all key information items that need to be collected;

[0070] For example, in the field of transportation, key information includes the type, time, and location of traffic incidents. User requirements contain three key pieces of information: incident type, time, and location. The collected corpus contains two key pieces of information: time and location. The information completeness is 0.67.

[0071] A3. Based on known accurate data, evaluate the accuracy of the preliminary corpus data. The calculation formula is:

[0072]

[0073] In the calculation formula, Accuracy represents data accuracy, Number of Correct Information represents the number of correct information in the corpus, that is, the number of information items that meet user needs or actual situations, and Total Information represents the total amount of all information in the corpus, including correct information and incorrect information;

[0074] For example, if there are 5 information points in the collected traffic incident data, 4 of which are consistent with the official data, the data accuracy is 0.8;

[0075] A4. Calculate the overall quality score based on text relevance, information completeness, and data accuracy. The calculation formula is:

[0076] Score=ω1*Similaty+ω2*Completeness+ω3*Accuracy

[0077] In the calculation formula, Score represents the comprehensive quality score, ω1 represents the weight of text relevance, ω2 represents the weight of information completeness, and ω3 represents the weight of data accuracy, ω1+ω2+ω3=1;

[0078] The weights are ω1=0.4, ω2=0.3, and ω3=0.3, respectively. The text relevance is 1, the information completeness is 0.67, the data accuracy is 0.8, and the overall quality score is 0.841.

[0079] The implementation method for dynamically adjusting the collection strategy is as follows:

[0080] B1. Set a comprehensive quality score threshold and compare the calculated comprehensive quality score with the quality score threshold;

[0081] B2. Extract and save the corpus with a comprehensive quality score higher than or equal to the comprehensive quality score threshold;

[0082] B3. Extract and remove corpora with comprehensive quality scores below the comprehensive quality score threshold;

[0083] For example, if the comprehensive quality score threshold is set to 0.7 and the comprehensive quality score is 0.841, the comprehensive quality score of the corpus data is higher than the comprehensive quality score threshold, and the corpus data is saved;

[0084] Through multi-dimensional evaluation of semantic similarity, information completeness and data accuracy, it can accurately identify the degree of match between corpus data and user needs, and quantify text relevance, content coverage and correctness; the comprehensive quality score further integrates multiple indicators to provide a global evaluation of the data. Combined with dynamic adjustment strategies, it can screen high-quality corpus based on thresholds and eliminate low-quality data, thereby ensuring the accuracy, completeness and reliability of the corpus. This closed-loop mechanism not only improves data collection efficiency and reduces redundant resource consumption, but also continuously optimizes collection strategies through feedback loops to ensure the quality stability and practicality of dynamic updates of the corpus.

[0085] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The MCP-based corpus dynamic collection and quality assessment method is characterized by: The following steps are involved: Step 1: Obtain user collection requirements; Step 2: Based on user collection requirements, call external collection tools through the MCP protocol; Step 3: The collection tool analyzes the target website according to the user's collection needs and obtains web page content that is highly relevant to the user's needs; Step 4: Pre-process the collected web page content to generate preliminary corpus data; Step 5: Perform a quality assessment on the generated preliminary corpus data and dynamically adjust the collection strategy based on the quality assessment results.

2. The MCP-based corpus dynamic collection and quality assessment method according to claim 1, characterized in that: In step 1, user requirements include target domain, corpus type, and collection frequency.

3. The MCP-based corpus dynamic collection and quality assessment method according to claim 2, characterized in that: In step 2, calling external collection tools includes but is not limited to web crawlers, API interface calls, and file downloaders.

4. The MCP-based corpus dynamic collection and quality assessment method according to claim 3, characterized in that: In the step 4, the preliminary preprocessing includes removing HTML tags by regular expressions; extracting the text content by using the tags; Use jieba for word segmentation.

5. The MCP-based corpus dynamic collection and quality assessment method according to claim 4, characterized in that: In step 5, the specific methods of quality assessment include: A1. Calculate the semantic similarity between the collected preliminary corpus data and user requirements to evaluate text relevance; A2. Evaluate the completeness of the preliminary corpus data; A3. Evaluate the accuracy of the preliminary corpus data based on known accurate data; A4. Calculate an overall quality score based on text relevance, information completeness, and data accuracy.

6. The MCP-based corpus dynamic collection and quality assessment method according to claim 5, characterized in that: The calculation formula of the text relevance is: In the calculation formula, Similaty represents text relevance, A represents the keyword set of user needs, B represents the keyword set of the collected corpus, |A∩B| represents the intersection of the keywords A and B, that is, the number of keywords that appear together in the two sets, and |A∪B| represents the union of the keywords A and B, that is, the number of all non-repeated keywords in the two sets.

7. The MCP-based corpus dynamic collection and quality assessment method according to claim 6, characterized in that: The calculation formula for the information integrity is: In the calculation formula, Completeness represents information completeness, Number of Key Information represents the actual number of key information collected, that is, the number of key information items in the corpus that meet the requirements, and Total Key Information represents the total number of key information required by users, that is, the number of all key information items that need to be collected.

8. The MCP-based corpus dynamic collection and quality assessment method according to claim 7, characterized in that: The calculation formula for the data accuracy is: In the calculation formula, Accuracy represents data accuracy, Number of Correct Information represents the amount of correct information in the corpus, that is, the number of information items that meet user needs or actual situations, and Total Information represents the total amount of all information in the corpus, including correct information and incorrect information.

9. The MCP-based corpus dynamic collection and quality assessment method according to claim 8, characterized in that: The calculation formula for the comprehensive quality score is: Score=ω1*Similaty+ω2*Completeness+ω3*Accuracy In the calculation formula, Score represents the comprehensive quality score, ω1 represents the weight of text relevance, ω2 represents the weight of information completeness, and ω3 represents the weight of data accuracy, ω1+ω2+ω3=1.

10. The MCP-based corpus dynamic collection and quality assessment method according to claim 9, characterized in that: The implementation method of dynamically adjusting the acquisition strategy is as follows: B1. Set a comprehensive quality score threshold and compare the calculated comprehensive quality score with the quality score threshold; B2. Extract and save the corpus with a comprehensive quality score higher than or equal to the comprehensive quality score threshold; B3. Extract and remove the corpus whose comprehensive quality score is lower than the comprehensive quality score threshold.

Images