Domain data collection method and device, electronic equipment and storage medium

Abstract

The invention discloses a field data collection method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: obtaining an open source data set in an open source corpus, and carrying out the field correlation labeling of all data in the open source data set, and obtaining a labeled data set; screening the annotated data set according to the annotated domain correlation to obtain an annotated data subset related to the target domain; performing compliance annotation on each piece of data in the annotation data subset to obtain an annotation data subset; and screening the annotation data subset according to the compliance of the annotation to obtain target domain data. According to the method, professional corpus data which not only has financial field correlation but also meets compliance requirements is efficiently extracted from a large amount of open source corpora, the quantity and quality of data in the target field are ensured, and a reliable data basis is provided for pre-training of a large model in the financial field; therefore, the understanding ability and expression accuracy of the model in the financial field are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a field data collection method, device, electronic device and storage medium. Background Art

[0002] At present, the pre-training of large models in specific fields mainly relies on huge corpora in specific fields. Specific corpora in specific fields can help general large language models understand complex text content related to specific fields. For example, financial data in financial corpora can help general large language models understand complex financial concepts and accurately grasp the language expressions unique to the financial industry. Therefore, the pre-training process of large models in the financial field requires the collection of a large number of diverse, compliant and secure professional corpora in the financial field, including company announcements, financial analysis reports, investment research reports, etc. However, since these data usually involve company privacy, commercial confidential information, etc., there are few open source corpora in the financial field, including only FinCorpus and FinGLM data sets.

[0003] Therefore, how to obtain a large amount of diverse, compliant and secure professional corpus data in a specific field is a problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of this application is to provide a domain data collection method, device, electronic device and storage medium, aiming to solve the technical problem of how to obtain a large amount of diverse, compliant and secure professional corpus data in a specific domain.

[0005] To achieve the above objectives, the present application proposes a method for collecting domain data, which includes:

[0006] Obtain an open source data set from an open source corpus, and annotate each data in the open source data set with domain relevance to obtain an annotated data set;

[0007] The annotated data set is screened according to the domain relevance of the annotations to obtain an annotated data subset related to the target domain;

[0008] Annotate each data in the labeled data subset for compliance, and obtain the annotated data subset;

[0009] The annotation data subset is filtered according to the compliance of the annotations to obtain the target domain data.

[0010] In one embodiment, the domain relevance of each data in the open source data set is annotated by a first data classification model, and the first data classification model is trained based on the first sample data. The domain data collection method further includes:

[0011] Extract part of the data in the open source data set as the first sample data;

[0012] Annotating the domain relevance of each data in the first sample data by using a preset large language model to obtain first annotated sample data;

[0013] The first labeled sample data is used as a first training set of a first pre-trained language model, and the first pre-trained language model is trained based on the first training set to obtain a first data classification model.

[0014] In one embodiment, the step of training a first pre-trained language model based on a first training set to obtain a first data classification model includes:

[0015] For any one of the plurality of preset training parameter configurations, training a first pre-trained language model under the training parameter configuration based on a first training set to obtain a first candidate data classification model;

[0016] After traversing each training parameter, each first candidate data classification model under each training parameter configuration is obtained;

[0017] Obtaining a first validation set of the first pre-trained language model from the first training set, and performing classification performance score evaluation on each first candidate data classification model based on the first validation set to obtain a first evaluation result, wherein the first evaluation result includes the classification performance score of each first candidate data classification model;

[0018] The target first candidate data classification model corresponding to the highest classification performance score in the first evaluation result is used as the first data classification model.

[0019] In one embodiment, the compliance of each data in the labeled data subset is annotated by a second data classification model, and the second data classification model is trained based on the second sample data. The field data collection method also includes:

[0020] Extract part of the data in the open source dataset as the second sample data;

[0021] Annotating the compliance of each data in the second sample data by using a preset large language model to obtain second annotated sample data;

[0022] The second labeled sample data is used as a second training set of a second pre-trained language model, and the second pre-trained language model is trained based on the second training set to obtain a second data classification model.

[0023] In one embodiment, the step of training the second pre-trained language model based on the second training set to obtain the second data classification model includes:

[0024] For any one of the plurality of preset training parameter configurations, training a second pre-trained language model under the training parameter configuration based on a second training set to obtain a second candidate data classification model;

[0025] After traversing each training parameter, each second candidate data classification model under each training parameter configuration is obtained;

[0026] Obtaining a second validation set of the second pre-trained language model from the second training set, and evaluating the precision and recall of each second candidate data classification model based on the second validation set to obtain a second evaluation result;

[0027] A second data classification model among the second candidate data classification models is determined according to the second evaluation result.

[0028] In one embodiment, the step of determining the second data classification model in each second candidate data classification model according to the second evaluation result includes:

[0029] Calculating the classification performance score of each second candidate data classification model according to the second evaluation result, and determining the remaining classification performance scores within a preset score range of the highest classification performance score among the classification performance scores;

[0030] If there are other classification performance scores, then the target candidate data classification model with the highest recall rate among the target candidate data classification models corresponding to the highest classification performance score and the other classification performance scores is selected as the second data classification model;

[0031] If there are no other classification performance scores, the second candidate data classification model corresponding to the highest classification performance score is used as the second data classification model.

[0032] In one embodiment, the step of annotating each data in the open source data set with respect to domain relevance to obtain the annotated data set includes:

[0033] Perform data cleaning on the open source data set to obtain the cleaned open source data set;

[0034] The cleaned open source dataset is annotated with domain relevance by a preset large language model guided by preset prompt words to obtain an annotated dataset.

[0035] In addition, to achieve the above purpose, the present application also proposes a field data collection device, which includes:

[0036] The domain annotation module is used to obtain the open source data set in the open source corpus and annotate the domain relevance of each data in the open source data set to obtain an annotated data set;

[0037] The domain screening module is used to screen the annotated data set according to the domain relevance of the annotations to obtain a subset of the annotated data related to the target domain;

[0038] A compliance annotation module is used to perform compliance annotation on each data in the labeled data subset to obtain an annotated data subset;

[0039] The compliance screening module is used to screen the annotation data subset according to the compliance of the annotations to obtain the target domain data.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the field data collection method as described above.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the field data collection method described above are implemented.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the field data collection method described above are implemented.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] This application first obtains an open source data set from an open source corpus, and annotates the domain relevance of each data in the open source data set to obtain an annotated data set, which provides a reliable screening basis for screening out specific domain data from the open source corpus; the annotated data set is screened according to the annotated domain relevance to obtain an annotated data subset related to the target domain, and by narrowing the scope of the data set, the data subset is more focused on the target domain, thereby improving the relevance of the data set to the target domain; compliance annotation is performed on each data in the annotated data subset to obtain an annotated data subset, which provides a reliable judgment basis for ensuring that the collected data complies with relevant laws and regulations and security risk requirements of the target domain; the annotated data subset is screened according to the compliance of the annotations to obtain the target domain data, thereby reducing the risk of data use and ensuring the compliance of the data set, and obtaining the target domain professional corpus data that meets both domain relevance and compliance requirements, providing a high-quality and reliable data source for pre-training of large models in the target domain.

[0045] This application avoids the problems of data in specific fields being difficult to obtain due to privacy and confidential information involved, as well as insufficient data or lack of focus on the field due to the scarcity of open source corpora in specific fields, by labeling and annotating the data in the dataset and continuously screening the dataset based on the relevant labels and annotations. It achieves efficient extraction of professional corpus data that is both relevant to the specific field and meets compliance requirements from a large number of open source corpora, ensures the quantity and quality of data in the target field, and provides a reliable data foundation for the pre-training of large models in specific fields, thereby improving the model's understanding ability and expression accuracy in specific fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 A flow chart of the first embodiment of the data collection method in the field of this application;

[0049] Figure 2 A flow chart of the second embodiment of the data collection method in the field of this application is provided;

[0050] Figure 3 A brief flowchart of the field data collection method provided in Example 2 of the present application;

[0051] Figure 4 A schematic flow chart of a scenario of a method for collecting domain data provided in Embodiment 2 of the present application;

[0052] Figure 5 This is a schematic diagram of the module structure of a data collection device in the field of an embodiment of the present application;

[0053] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the field data collection method in the embodiment of the present application.

[0054] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0056] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0057] The main solution of the embodiment of the present application is: obtain an open source data set in an open source corpus, and annotate each data in the open source data set with domain relevance to obtain an annotated data set; filter the annotated data set according to the annotated domain relevance to obtain an annotated data subset related to the target domain; annotate each data in the annotated data subset with compliance to obtain an annotated data subset; filter the annotated data subset according to the compliance of the annotation to obtain target domain data.

[0058] Since the current pre-training of large models in specific fields mainly relies on huge corpora in specific fields, which can help general large language models understand complex text content related to specific fields. For example, the financial field data in the financial field corpus can help general large language models understand complex financial concepts and accurately grasp the language expressions unique to the financial industry. Therefore, the pre-training process of large models in the financial field requires the collection of a large number of diverse, compliant and secure professional corpora in the financial field, including company announcements, financial analysis reports, investment research reports, etc. However, since these data usually involve company privacy, commercial confidential information, etc., there are few open source financial field corpora, which only include FinCorpus and FinGLM data sets. Therefore, how to obtain a large number of diverse, compliant and secure professional corpus data in specific fields is a problem that needs to be solved urgently.

[0059] The present application provides a solution, which labels and annotates the data in the dataset and continuously screens the dataset based on the relevant labels and annotations, thereby avoiding the problems of data in specific fields being difficult to obtain due to privacy and confidential information involved, as well as insufficient data or lack of focus on the field due to the scarcity of open source corpora in specific fields. It achieves efficient extraction of professional corpus data that is both relevant to specific fields and meets compliance requirements from a large number of open source corpora, ensures the quantity and quality of data in the target field, and provides a reliable data foundation for the pre-training of large models in specific fields, thereby improving the model's understanding ability and expression accuracy in specific fields.

[0060] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions. The following takes an electronic device as an example to illustrate this embodiment and the following embodiments.

[0061] Based on this, the present application embodiment provides a method for collecting domain data, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data collection method in the present application field.

[0062] In this embodiment, the field data collection method includes steps S10 to S40:

[0063] Step S10, obtaining an open source data set in an open source corpus, and annotating each data in the open source data set with respect to domain relevance to obtain an annotated data set;

[0064] It should be noted that an open source dataset refers to a publicly accessible collection of multiple text data in multiple fields. These data can come from public resources on the Internet, such as academic papers, news reports, public reports, etc.; domain relevance refers to the degree of correlation between data and a specific field, such as the degree of correlation between data and the financial field, that is, whether the data content involves financial knowledge, concepts, terms or industry trends, etc.; annotated dataset refers to a data set that has been processed by automated tools and has marked the domain relevance of each data in the original open source dataset.

[0065] It is understandable that, since the data in the open source dataset is mixed, performing step S10 can avoid the problem of being unable to identify data related to a specific field, which in turn causes poor model training results due to the use of irrelevant data, thereby improving the specialization and relevance of the dataset.

[0066] Exemplarily, open source data sets consisting of a Chinese data set and an English data set are respectively obtained from an open source corpus, and the Chinese data set is scored and labeled with a score of 0 to 5 based on its financial relevance through the bge_small_chinese model in the BERT (Bidirectional Encoder Representation from Transformers) model, and the English data is scored and labeled with a score of 0 to 5 based on its financial relevance through the snowflake-arctic-embed-xs model in the BERT model, thereby obtaining a labeled data set consisting of a labeled Chinese data set and a labeled English data set.

[0067] Step S20, screening the annotated data set according to the domain relevance of the annotations to obtain an annotated data subset related to the target domain;

[0068] It should be noted that the target domain refers to a specific domain for pre-training of large models, and this embodiment is used to collect data for pre-training of large models in this specific domain; the labeled data subset refers to a data set that is screened from the labeled data set and is highly relevant to the target domain.

[0069] It is understandable that, since the annotated data set contains some data that is not strongly related to the target field or is not the target field, performing step S20 can avoid the generalization and inaccuracy problems of the data set and achieve accurate selection of data in the data set.

[0070] Exemplarily, the annotated data set is screened according to a set financial relevance score threshold. When the score threshold is set to 3, the data in the annotated data set with a financial relevance score lower than 3 is filtered out, and the data in the annotated data set with a financial relevance score not lower than 3 is obtained as the annotated data subset.

[0071] Step S30, annotating each data in the labeled data subset for compliance to obtain an annotated data subset;

[0072] It should be noted that compliance refers to whether the data complies with relevant laws and regulations, industry standards and company policies, or regulations involving privacy protection, trade secrets and intellectual property rights; the annotated data subset refers to a data set that has been evaluated and annotated for compliance of each data in the labeled data subset, indicating whether the data is suitable for large model training.

[0073] It is understandable that since the data used for pre-training of large models in a specific field needs to comply with the relevant regulations and requirements of the field, performing step S30 can avoid the legal risks, business risks and security risks brought about by using illegal data for pre-training of large models, and provide an effective and reliable screening basis to ensure the compliance of the data set.

[0074] Exemplarily, a labeled data subset including a labeled Chinese data subset and a labeled English data subset is obtained, and the bge_base_chinese model in the BERT model is used to annotate the labeled Chinese data subset from three dimensions: whether it contains discriminatory content, whether it contains commercial illegal and irregular content, and whether it infringes on the legitimate rights and interests of others. The snowflake-arctic-embed-xs model in the BERT model is used to annotate the English data subset from three dimensions: whether it contains discriminatory content, whether it contains commercial illegal and irregular content, and whether it infringes on the legitimate rights and interests of others, to obtain an annotated data subset consisting of the annotated Chinese data subset and the annotated English data subset.

[0075] Step S40, screening the annotation data subset according to the compliance of the annotations to obtain target domain data.

[0076] It should be noted that the target domain data refers to the data set that is finally screened out and has both domain relevance and meets compliance requirements. This data set can be used for pre-training of large models in specific domains.

[0077] It is understandable that since the annotation data subset may contain non-compliant data that does not comply with the relevant rules of the target domain, performing step S40 can avoid the risk of using non-compliant data in the pre-training of large models in the target domain, thereby obtaining a specific domain data set that is both professional and compliant, providing high-quality data support for large model pre-training.

[0078] Exemplarily, a subset of annotation data containing relevant annotations in three dimensions, namely, whether it contains discriminatory content, whether it contains commercial illegal and irregular content, and whether it infringes upon the legitimate rights and interests of others, is screened, and relevant data with non-compliant annotations in any dimension of the annotation data subset is removed to obtain target field data that is compliant in all three dimensions.

[0079] This embodiment provides a method for collecting domain data. By labeling and annotating the data in the data set and continuously screening the data set based on the relevant labels and annotations, the problem of data in a specific domain being difficult to obtain due to privacy and confidential information involved, as well as insufficient data or lack of domain focus due to the scarcity of open source corpora in specific domains is avoided. Professional corpus data that is both relevant to a specific domain and meets compliance requirements can be efficiently extracted from a large number of open source corpora, ensuring the quantity and quality of data in the target domain, and providing a reliable data foundation for pre-training of large models in specific domains, thereby improving the model's understanding ability and expression accuracy in specific domains.

[0080] In a feasible implementation manner, the domain relevance of each data in the open source data set is annotated by a first data classification model, and the first data classification model is trained based on the first sample data. The domain data collection method may further include steps S11 to S13:

[0081] Step S11, extracting part of the data in the open source data set as first sample data;

[0082] It should be noted that the first sample data refers to a part of the data extracted from the open source dataset and used in the subsequent pre-training model training process. This part of the data is randomly selected through undersampling technology, aiming to construct a feature that is balanced in categories and represents the entire dataset.

[0083] It is understandable that since the data classification model that labels domain relevance needs to be pre-trained to ensure labeling accuracy, step S11 is performed. By extracting representative data from the entire data set, the high cost and low efficiency of using the entire large data set for pre-training of the data classification model can be avoided, thereby providing real and effective training data for the pre-training of the data classification model.

[0084] Exemplarily, undersampling technology is performed to extract 50,000 data items from the open source data set as the first sample data.

[0085] Step S12, annotating the domain relevance of each data in the first sample data by using a preset large language model to obtain first annotated sample data;

[0086] It should be noted that the preset large language model refers to a deep learning model trained using a large amount of text data, which may be a Qwen-72B model, etc.; the first labeled sample data refers to a data set obtained by annotating each data item in the first sample data with domain relevance, and these data items have been labeled with scores with specific domain relevance.

[0087] Additionally, it should be noted that the first labeled sample data can update the domain relevance of the annotation in response to the modification instruction to further ensure the annotation quality of the sample data for training the pre-trained language model, thereby improving the training effect of the first data classification model.

[0088] It is understandable that since there may be a variety of different data from different fields in the sample data, performing step S12 can avoid the problem of failure to identify and mark specific field-related content in the data set, thereby causing the use of irrelevant data in the large model pre-training process to cause the model performance to deteriorate, thereby achieving the specialization and precision of the data set.

[0089] Exemplarily, the Qwen-72B model guided by preset prompt words is used to label and score each data in the extracted first sample data with a score of 0 to 5 according to the relevance in the financial field, so as to obtain the first labeled sample data, wherein the financial field relevance of the data refers to whether the data contains information directly or indirectly related to the financial field, which may involve all aspects of the financial market, including but not limited to policies and regulations, economic indicators, financial products, trading activities, risk management, and market analysis.

[0090] Step S13: using the first labeled sample data as a first training set of a first pre-trained language model, and training the first pre-trained language model based on the first training set to obtain a first data classification model.

[0091] It should be noted that the first pre-trained language model refers to a large language model that has the ability to understand and generate natural language but has not yet been optimized for classification in a specific financial field. It may include BERT models such as the bge_small_chinese model, the bge_base_chinese model, and the snowflake-arctic-embed-xs model. By adding a regression layer as a classification head to the traditional BERT model mentioned above, and fixing the embedding layer and the encoder layer, the BERT model classification performance optimization process is implemented based on the newly added classification head; the first training set refers to a data set used to train the first pre-trained language model, which contains annotation information related to a specific field; the first data classification model refers to a model that can classify input data according to field relevance after training, and the model is further optimized based on the first pre-trained language model.

[0092] It is understandable that the usual pre-trained language model lacks specificity when processing data in a specific field, so step S13 is performed to optimize the pre-trained language model for a specific field, thereby realizing a professional model that can effectively identify and classify data in a specific field, thereby improving the classification performance of the model in a specific field.

[0093] In this implementation, by performing data sampling, large language model annotation, and pre-trained language model fine-tuning, the problems of low model training efficiency and insufficient generalization ability caused by directly using a large amount of unlabeled and unscreened data are avoided, and representative sample data related to a specific domain is efficiently extracted and labeled from open source data sets. Based on these data, the pre-trained language model is optimized for a specific domain, and a first data classification model that can accurately classify data in a specific domain is obtained, thereby improving the professionalism and performance of the model in a specific domain.

[0094] In a feasible implementation manner, the step of using the first labeled sample data as a first training set of a first pre-trained language model, and training the first pre-trained language model based on the first training set to obtain the first data classification model in step S13 may include steps S131 to S134:

[0095] Step S131, for any one of the plurality of preset training parameter configurations, training a first pre-trained language model under the training parameter configuration based on a first training set to obtain a first candidate data classification model;

[0096] It should be noted that training parameters refer to adjustable parameters in the model training process, such as the initial learning rate, number of iterations, etc., which affect the effect and efficiency of model training; the first candidate data classification model refers to the candidate version of the data classification model obtained after training the first pre-trained language model under specific training parameter configuration.

[0097] It is understandable that, since the classification performance of the pre-trained language model under different training parameter configurations is different, performing step S131 can avoid the problem of poor model performance that may be caused by using a single training parameter configuration, thereby achieving the selection of training parameters under the optimal configuration.

[0098] Exemplarily, AdamW is used as the optimizer and get_linear_schedule_with_warmup scheduler in the first pre-trained language model, the initial learning rate is selected between 1e-4 and 5e-4, the number of iterations is selected from 15, 20, and 25, and the first pre-trained language model under each initial learning rate and number of iterations configuration is trained based on the first training set.

[0099] Step S132, after traversing each training parameter, obtaining each first candidate data classification model under each training parameter configuration;

[0100] Step S133, obtaining a first validation set of the first pre-trained language model from the first training set, and performing classification performance score evaluation on each first candidate data classification model based on the first validation set to obtain a first evaluation result, wherein the first evaluation result includes the classification performance score of each first candidate data classification model;

[0101] It should be noted that the first validation set refers to a part of data separated from the first training set, which is used to evaluate model performance and ensure the generalization ability of the model on unknown data; the classification performance score can be the F1 score, which is an evaluation indicator in statistics that comprehensively considers the model's precision and recall rate. It is used to measure the classification performance of the model. The maximum value of the score is 1 and the minimum value is 0.

[0102] It is understandable that, since the performance of each candidate model needs to be objectively evaluated, performing step S133 can avoid the inability to accurately judge the actual performance of the model, thereby ensuring the accuracy and reliability of the model evaluation results.

[0103] Exemplarily, during the training process, the classification precision and recall of the candidate data classification models trained under each training parameter configuration are evaluated through the validation set, and the F1 score of each candidate data classification model is calculated based on the precision and recall.

[0104] Step S134: taking the target first candidate data classification model corresponding to the highest classification performance score in the first evaluation result as the first data classification model.

[0105] Exemplarily, among the candidate data classification models trained under each training parameter configuration (the initial learning rate is selected between 1e-4 and 5e-4, and the number of iterations is selected from 15, 20, and 25), the target first candidate data classification model with the highest F1 score is used as the first data classification model. The initial learning rate of the model is 3e-4, and the number of iterations is 20. When the F1 score threshold is set to 3, the bge_small_chinese model achieves an F1 score of 82.7% on the Chinese validation set, and the snowflake-arctic-embed-xs model achieves an F1 score of 86% on the English validation set.

[0106] In this implementation, by performing training parameter tuning, model training, validation set evaluation, and model selection, the problems of insufficient model performance, overfitting, or poor generalization ability that may be caused by a single training parameter configuration are avoided, and the pre-trained language model is carefully optimized. By comparing the model performance under different training parameters, the first candidate data classification model with the best performance in classification performance score is selected as the final model, thereby improving the accuracy and efficiency of the model in data classification tasks in specific fields.

[0107] In a feasible implementation manner, the compliance of each data in the annotated data subset is annotated by a second data classification model, and the second data classification model is trained based on the second sample data. The domain data collection method may further include steps S31 to S33:

[0108] Step S31, extracting part of the data in the open source data set as second sample data;

[0109] It should be noted that the second sample data refers to a part of the data extracted again from the open source dataset, which is used for labeling and model training specifically for compliance. This part of the data is randomly selected through undersampling technology, aiming to construct a feature that is balanced in categories and represents the entire dataset.

[0110] It is understandable that since the data classification model for annotation compliance needs to be pre-trained to ensure the accuracy of the annotations, step S31 is performed. By extracting representative data from the entire data set, the high cost and low efficiency of using the entire large data set for pre-training the data classification model can be avoided, thereby providing real and effective training data for the pre-training of the data classification model.

[0111] Exemplarily, undersampling technology is performed to extract 30,000 data items from the open source data set as the second sample data.

[0112] Step S32, annotating the compliance of each data in the second sample data by using a preset large language model to obtain second annotated sample data;

[0113] It should be noted that the preset large language model refers to a deep learning model trained using a large amount of text data, which may be a Qwen-72B model, etc.; the second labeled sample data refers to a data set obtained by annotating each data item in the second sample data for compliance, and these data items have been annotated with content with relevant dimensional compliance.

[0114] In addition, it should be noted that the second labeled sample data can update the compliance of the labeling in response to the modification instruction to further ensure the labeling quality of the sample data for training the pre-trained language model, thereby improving the training effect of the second data classification model.

[0115] It is understandable that since there may be a variety of different data that may be safe or bad in the sample data, performing step S32 can avoid the inability to identify and annotate bad data-related content in the data set, thereby avoiding the risk of using bad data in the large model pre-training process, thereby improving the security and compliance of the data set.

[0116] Exemplarily, the Qwen-72B model guided by preset prompt words annotates the data in the extracted second sample data from three dimensions: whether it contains discriminatory content, whether it contains commercial illegal and irregular content, and whether it infringes on the legitimate rights and interests of others, to obtain the second labeled sample data.

[0117] Step S33: Use the second labeled sample data as a second training set for a second pre-trained language model, and train the second pre-trained language model based on the second training set to obtain a second data classification model.

[0118] It should be noted that the second pre-trained language model refers to a large language model that has the ability to understand and generate natural language but has not yet been optimized for compliance classification in a specific financial field. It may include BERT models such as the bge_small_chinese model, the bge_base_chinese model, and the snowflake-arctic-embed-xs model. By adding a regression layer as a classification head to the traditional BERT model mentioned above, and fixing the embedding layer and the encoder layer, the BERT model classification performance optimization process is implemented based on the newly added classification head; the second training set refers to a data set used to train the second pre-trained language model, which contains annotation information on data compliance in a specific field; the second data classification model refers to a model that can classify input data for compliance in a specific field after training, and the model is further optimized based on the second pre-trained language model.

[0119] It is understandable that when processing data in a specific field, a conventional pre-trained language model cannot determine whether the data complies with the relevant rules and regulations of the specific field. Therefore, step S33 is performed to optimize the pre-trained language model for compliance judgment in a specific field, thereby realizing a professional model that can effectively identify and classify the compliance of data in a specific field, thereby improving the classification performance of the model in terms of data compliance in a specific field.

[0120] In this implementation, by performing data sampling, large language model annotation and model training, compliance risks that may arise in data processing in specific fields are avoided, and representative compliance sample data are efficiently extracted and annotated from open source data sets. Based on these data, a second data classification model is trained to accurately classify whether data items are compliant, thereby improving the model's ability to identify and manage data compliance in specific fields and ensuring the security and compliance of data processing.

[0121] In a feasible implementation manner, the step of training the second pre-trained language model based on the second training set to obtain the second data classification model in step S33 may include steps S331 to S334:

[0122] Step S331, for any one of the plurality of preset training parameter configurations, training a second pre-trained language model under the training parameter configuration based on a second training set to obtain a second candidate data classification model;

[0123] It should be noted that the second training set refers to a data set used to train the second pre-trained language model, which contains second sample data that has been labeled with compliance; the second candidate data classification model refers to multiple potential data classification models obtained after training the second pre-trained language model under different training parameter configurations.

[0124] It is understandable that, since the classification performance of the pre-trained language model under different training parameter configurations is different, performing step S331 can avoid the problem of poor model performance that may be caused by using a single training parameter configuration, thereby achieving the selection of training parameters under the optimal configuration.

[0125] Exemplarily, AdamW is used as the optimizer and get_linear_schedule_with_warmup scheduler in the second pre-trained language model, the initial learning rate is selected between 1e-4 and 5e-4, the number of iterations is selected from 15, 20, and 25, and the second pre-trained language model under each initial learning rate and number of iterations configuration is trained based on the second training set.

[0126] Step S332, after traversing each training parameter, obtaining each second candidate data classification model under each training parameter configuration;

[0127] Step S333, obtaining a second validation set of the second pre-trained language model from the second training set, and evaluating the precision and recall of each second candidate data classification model based on the second validation set to obtain a second evaluation result;

[0128] It should be noted that the second validation set refers to a part of the data separated from the second training set, which is used to evaluate the performance of the second candidate data classification model; the precision rate refers to the proportion of samples correctly predicted by the model as positive (compliant or non-compliant) to all samples predicted as positive; the recall rate refers to the proportion of samples correctly predicted by the model as positive to all samples that are actually positive.

[0129] It is understandable that, since the performance of each candidate model needs to be objectively evaluated, performing step S333 can avoid the inability to accurately judge the actual performance of the model, thereby ensuring the accuracy and reliability of the model evaluation results.

[0130] Step S334: determine the second data classification model among the second candidate data classification models according to the second evaluation result.

[0131] Exemplarily, among the candidate data classification models trained under each training parameter configuration (the initial learning rate is selected between 1e-4 and 5e-4, and the number of iterations is selected from 15, 20, and 25), the candidate data classification model with the highest F1 score and recall rate is taken as the second data classification model, and the initial learning rate of the model is 3e-4, and the number of iterations is 20 times.

[0132] In this implementation, by performing training parameter tuning, model training, validation set evaluation, and model selection, the problems of insufficient model performance, overfitting, or poor generalization ability that may be caused by a single training parameter configuration are avoided, and the second pre-trained language model is carefully optimized. By comparing the model performance under different training parameters and selecting the second data classification model with the best performance based on the precision and recall evaluation results, the prediction accuracy and completeness of the model in data compliance classification tasks in specific fields are improved, thereby ensuring the effectiveness and reliability of the model in practical applications.

[0133] In a feasible implementation, step S334 may include steps S301 to S303:

[0134] Step S301, calculating the classification performance score of each second candidate data classification model according to the second evaluation result, and determining the remaining classification performance scores within a preset score range of the highest classification performance score among the classification performance scores;

[0135] It should be noted that the preset score interval refers to a pre-set classification performance score range, which is used to screen out the classification performance scores of other candidate models that are close to the highest classification performance score.

[0136] It is understandable that since the ultimate goal of training the second data classification model is to remove data with security risks in specific field data and reduce the proportion of illegal and bad data, more attention should be paid to the recall rate of bad data during the training process. Therefore, performing step S301 can avoid judging the model performance based on a single indicator classification performance score, thereby achieving performance evaluation that tends to actual needs.

[0137] Exemplarily, the F1 score of each second candidate data classification model is calculated by the following formula:

[0138]

[0139] Among them, precision refers to the accuracy rate, and recall refers to the recall rate. And determine the remaining F1 scores within the preset score range of the highest F1 score among each F1 score.

[0140] Step S302, if there are other classification performance scores, then select the target candidate data classification model with the highest recall rate among the target candidate data classification models corresponding to the highest classification performance score and the other classification performance scores as the second data classification model;

[0141] It should be noted that the target candidate data classification model refers to a candidate data classification model whose classification performance score falls within a preset score range and is likely to be selected as the final model.

[0142] It is understandable that, since the recall rate indicator needs to be further considered among models with similar performance under the classification performance score consideration, performing step S302 can avoid selecting a model with poor recall rate, thereby ensuring that the final model can accurately remove bad data when identifying compliance issues.

[0143] Step S303: If there are no other classification performance scores, the second candidate data classification model corresponding to the highest classification performance score is used as the second data classification model.

[0144] It is understandable that, since the model with the best performance can be directly selected when there are no other models with similar performance under classification performance score consideration, performing step S303 can avoid the problem of poor model performance caused by excessive inclination towards recall rate, thereby achieving efficient model selection.

[0145] In this implementation, by calculating the classification performance score, screening the preset score range and comparing the recall rate, the problem of model selection being difficult to match the actual application scenario due to single indicator evaluation is avoided, and a comprehensive evaluation of the second candidate data classification model on multiple performance indicators is achieved, ensuring that the selected second data classification model not only performs well in the classification performance score, but also has the highest possible recall rate, thereby achieving the technical effect of balancing accuracy and completeness in data compliance classification tasks in specific fields, and improving the predictive ability and reliability of the model in practical applications.

[0146] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , Figure 2 This is a flow chart of the second embodiment of the data collection method in the present application field.

[0147] The step of annotating the domain-relatedness of each data in the open source dataset in step S10 to obtain the annotated dataset may include steps S101 to S102:

[0148] Step S101, performing data cleaning on the open source data set to obtain a cleaned open source data set;

[0149] It should be noted that data cleaning refers to filtering sensitive words, filtering personal basic information, unifying traditional and simplified Chinese, etc. on each data in the open source data set, so as to improve the overall quality of the data set and provide reliable basic data for subsequent model training.

[0150] It is understandable that since the original data set may contain sensitive information and other data that are not suitable for the development of a specific domain model, performing step S101 can avoid the effect of model training being affected by data quality issues, thereby improving the security of the data during use and the efficiency of the model training process.

[0151] Step S102 , using a preset large language model guided by preset prompt words to annotate the cleaned open source dataset with domain relevance, to obtain an annotated dataset.

[0152] It is understandable that, since data related to a specific field needs to be identified and labeled, step S102 is performed, which can effectively avoid judging field-related data through a large language model guided by prompt words, thereby ensuring that the model can focus on specific knowledge and language expressions in a specific field.

[0153] In this embodiment, by performing data cleaning on the open source dataset, the impact of sensitive information or inconsistent data that may be contained in the original dataset on the model training process is avoided, which not only improves the purity and availability of the dataset, but also ensures that the trained model can better understand and handle specific problems in specific fields, thereby improving the performance and accuracy of the model in specific field applications.

[0154] In a feasible implementation, the field data collection method may further include steps S01-S02:

[0155] Step S01, obtaining data features of an open source data set, and matching each prompt word in a preset prompt word database according to the data features to obtain a target prompt word collection in the preset prompt word database;

[0156] It should be noted that data features refer to text information such as keywords, topics, contexts, etc. extracted from open source datasets. This information can reflect the domain characteristics and content features of the open source datasets. The preset prompt word database is a collection of various domain-related prompt words, which are used to guide the large language model to perform more accurate labeling. The target prompt word collection refers to a collection of prompt words that are screened from a large number of prompt words and have a high correlation with the domain of the open source dataset.

[0157] It can be understood that in order to screen out prompt words with high relevance to the open source dataset from a large number of prompt words, step S01 is performed, which can avoid using prompt words that are irrelevant or poorly relevant to the dataset for inefficient guidance, thereby reducing labeling errors, improving the accuracy and efficiency of labeling, achieving accurate labeling of datasets in specific fields, and improving the quality of subsequent data screening.

[0158] Step S02: generating preset prompt words based on the target prompt word collection.

[0159] It is understandable that in order to ensure that the prompt words used in the annotation process can effectively guide the large language model and make the annotations generated by the large language model more accurate and consistent, step S02 is performed to avoid annotation deviation or confusion caused by vague or improper combination of prompt words, ensure the consistency and reliability of the annotation process, thereby achieving more efficient and reliable domain-related annotation, and thus improving the overall quality of the data set.

[0160] In this embodiment, by extracting the features of the data set to generate more suitable prompt words to guide the large language model for annotation, the problem of inaccurate and inefficient data annotation caused by the mismatch between the prompt words and the data set domain is avoided, and the customized generation of highly relevant prompt words based on the specific domain characteristics of the data set is achieved, thereby improving the accuracy of data annotation and the degree of automation of the annotation process, thereby achieving the effect of improving the quality of the domain data set and the performance of the pre-trained model.

[0161] For example, to help understand the implementation process of the field data collection method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 A brief flowchart of a domain data collection method is provided, specifically:

[0162] First, open source datasets including Chinese datasets, English datasets and other categories are obtained from open source datasets, and data cleaning is performed on the open source datasets by filtering sensitive words, filtering personal information, unifying traditional and simplified Chinese, etc., and then the cleaned open source datasets are sent to the financial classifier, i.e. the first data classification model, for domain relevance annotation to determine the financial relevance score of each data in the open source dataset, and obtain the labeled dataset. Then the filtered labeled data subset is sent to the security risk identification classifier, i.e. the second data classification model, for compliance annotation to determine the compliance of each data in the labeled data subset, and obtain the annotated data subset. Finally, the annotated data subset is screened to obtain high-quality financial text that is safe and compliant, i.e. the target domain data. Both the financial classifier and the security risk identification classifier need to be pre-trained, i.e. the BERT model is trained with data labeled or annotated by a large language model.

[0163] In addition, in actual application, the above two classifiers are regularly updated and verified to adapt to the dynamic changes in the financial field. At the same time, a special BERT classifier is built for open source corpora from different Internet sources to enhance the customization and versatility of the model. Relatively speaking, the prompt words that guide the large language model to generate annotations or addresses are also changed according to the data characteristics of different Internet sources.

[0164] For further information, please refer to Figure 4 , Figure 4A scenario schematic flow chart of a domain data collection method is provided, specifically:

[0165] High-quality financial texts are annotated through a sampling large language model, and the annotated second training set is used to train the security risk identification classifier to perform security risk identification through the security risk identification classifier. The identification results can be manually sampled and verified. When the verification result is that the illegal bad rate is less than 1%, the data identified and screened by the security risk identification classifier is determined to be a large-scale financial field data set that is safe and compliant, that is, the target field data; when the verification result is that the illegal bad rate is greater than or equal to 1%, the security risk identification classification model is retrained, and the classification task is re-performed through the security risk identification classification model to ensure that the security risk identification classification model adapts to the data characteristics, thereby achieving the effect that the illegal bad rate of the final financial data set is less than 1%.

[0166] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data collection method in the field of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0167] This application also provides a field data collection device, please refer to Figure 5 , the field data collection device comprises:

[0168] The domain annotation module 10 is used to obtain an open source data set in the open source corpus and annotate each data in the open source data set with domain relevance to obtain an annotated data set;

[0169] A domain screening module 20 is used to screen the annotated data set according to the domain relevance of the annotations to obtain a subset of the annotated data related to the target domain;

[0170] A compliance annotation module 30, used to perform compliance annotation on each data in the labeled data subset to obtain an annotated data subset;

[0171] The compliance screening module 40 is used to screen the annotation data subset according to the compliance of the annotations to obtain target domain data.

[0172] Optionally, the domain relevance of each data in the open source data set is annotated by a first data classification model, the first data classification model is trained based on the first sample data, and the first training module 50 in the domain data collection device is used to:

[0173] Extract part of the data in the open source data set as the first sample data;

[0174] Annotating the domain relevance of each data in the first sample data by using a preset large language model to obtain first annotated sample data;

[0175] The first labeled sample data is used as a first training set of a first pre-trained language model, and the first pre-trained language model is trained based on the first training set to obtain a first data classification model.

[0176] Optionally, the first training module 50 is further used for:

[0177] For any one of the plurality of preset training parameter configurations, training a first pre-trained language model under the training parameter configuration based on a first training set to obtain a first candidate data classification model;

[0178] After traversing each training parameter, each first candidate data classification model under each training parameter configuration is obtained;

[0179] Obtaining a first validation set of the first pre-trained language model from the first training set, and performing classification performance score evaluation on each first candidate data classification model based on the first validation set to obtain a first evaluation result, wherein the first evaluation result includes the classification performance score of each first candidate data classification model;

[0180] The target first candidate data classification model corresponding to the highest classification performance score in the first evaluation result is used as the first data classification model.

[0181] Optionally, the compliance of each data in the labeled data subset is annotated by a second data classification model, the second data classification model is trained based on the second sample data, and the second training module 60 in the domain data collection device is used to:

[0182] Extract part of the data in the open source dataset as the second sample data;

[0183] Annotating the compliance of each data in the second sample data by using a preset large language model to obtain second annotated sample data;

[0184] The second labeled sample data is used as a second training set of a second pre-trained language model, and the second pre-trained language model is trained based on the second training set to obtain a second data classification model.

[0185] Optionally, the second training module 60 is further used for:

[0186] For any one of the plurality of preset training parameter configurations, training a second pre-trained language model under the training parameter configuration based on a second training set to obtain a second candidate data classification model;

[0187] After traversing each training parameter, each second candidate data classification model under each training parameter configuration is obtained;

[0188] Obtaining a second validation set of the second pre-trained language model from the second training set, and evaluating the precision and recall of each second candidate data classification model based on the second validation set to obtain a second evaluation result;

[0189] A second data classification model among the second candidate data classification models is determined according to the second evaluation result.

[0190] Optionally, the second training module 60 is further used for:

[0191] Calculating the classification performance score of each second candidate data classification model according to the second evaluation result, and determining the remaining classification performance scores within a preset score range of the highest classification performance score among the classification performance scores;

[0192] If there are other classification performance scores, then the target candidate data classification model with the highest recall rate among the target candidate data classification models corresponding to the highest classification performance score and the other classification performance scores is selected as the second data classification model;

[0193] If there are no other classification performance scores, the second candidate data classification model corresponding to the highest classification performance score is used as the second data classification model.

[0194] Optionally, the domain labeling module 10 is further used for:

[0195] Perform data cleaning on the open source data set to obtain the cleaned open source data set;

[0196] The cleaned open source dataset is annotated with domain relevance by a preset large language model guided by preset prompt words to obtain an annotated dataset.

[0197] Optionally, the domain labeling module 10 is further used for:

[0198] Acquire data features of the open source data set, and match each prompt word in the preset prompt word database according to the data features to obtain a target prompt word collection in the preset prompt word database;

[0199] Generate preset prompt words based on the target prompt word collection.

[0200] The domain data collection device provided by this application adopts the domain data collection method in the above embodiment, which can solve the technical problem of how to obtain a large amount of diverse, compliant and safe professional corpus data in a specific field. Compared with the prior art, the beneficial effects of the domain data collection device provided by this application are the same as the beneficial effects of the domain data collection method provided by the above embodiment, and the other technical features in the domain data collection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0201] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the field data collection method in the above-mentioned embodiment one.

[0202] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0203] like Figure 6 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0204] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0205] The electronic device provided by this application adopts the domain data collection method in the above embodiment, which can solve the technical problem of how to obtain a large amount of diverse, compliant and safe professional corpus data in a specific field. Compared with the prior art, the beneficial effects of the electronic device provided by this application are the same as the beneficial effects of the domain data collection method provided in the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0206] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0207] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0208] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the domain data collection method in the above-mentioned embodiment.

[0209] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0210] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0211] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains an open source data set in an open source corpus, and annotates each data in the open source data set with domain relevance to obtain an annotated data set; filters the annotated data set according to the annotated domain relevance to obtain an annotated data subset related to the target domain; annotates each data in the annotated data subset with compliance to obtain an annotated data subset; filters the annotated data subset according to the compliance of the annotation to obtain target domain data.

[0212] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0213] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0214] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0215] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned domain data collection method, and can solve the technical problem of how to obtain a large amount of diverse, compliant and safe professional corpus data in a specific field. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the domain data collection method provided in the above-mentioned embodiment, and will not be repeated here.

[0216] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned field data collection method when executed by a processor.

[0217] The computer program product provided by this application can solve the technical problem of how to obtain a large amount of diverse, compliant and secure professional corpus data in a specific field. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the field data collection method provided by the above embodiment, which will not be repeated here.

[0218] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for collecting domain data, characterized in that: The data collection methods in the field include: Obtain an open source data set in an open source corpus, and annotate each data in the open source data set with domain relevance to obtain an annotated data set; The annotated data set is screened according to the domain relevance of the annotations to obtain an annotated data subset related to the target domain; Annotating each data in the labeled data subset for compliance to obtain an annotated data subset; The annotation data subset is screened according to the compliance of the annotations to obtain target domain data.

2. The field data collection method according to claim 1, characterized in that: The domain relevance of each data in the open source data set is annotated by a first data classification model, and the first data classification model is trained based on the first sample data. The domain data collection method further includes: Extracting part of the data in the open source data set as the first sample data; Annotating the domain relevance of each data in the first sample data by using a preset large language model to obtain first annotated sample data; The first labeled sample data is used as a first training set of a first pre-trained language model, and the first pre-trained language model is trained based on the first training set to obtain the first data classification model.

3. The field data collection method according to claim 2, characterized in that: The step of training the first pre-trained language model based on the first training set to obtain the first data classification model includes: For any one of a plurality of preset training parameter configurations, training the first pre-trained language model under the training parameter configuration based on the first training set to obtain a first candidate data classification model; After traversing each training parameter, each first candidate data classification model under each training parameter configuration is obtained; Acquire a first validation set of the first pre-trained language model from the first training set, and perform classification performance score evaluation on each of the first candidate data classification models based on the first validation set to obtain a first evaluation result, wherein the first evaluation result includes the classification performance score of each of the first candidate data classification models; The target first candidate data classification model corresponding to the highest classification performance score in the first evaluation result is used as the first data classification model.

4. The field data collection method according to claim 1, characterized in that: The compliance of each data in the labeled data subset is annotated by a second data classification model, and the second data classification model is trained based on the second sample data. The domain data collection method also includes: Extracting part of the data in the open source data set as the second sample data; Annotating the compliance of each data in the second sample data by using a preset large language model to obtain second annotated sample data; The second labeled sample data is used as a second training set of a second pre-trained language model, and the second pre-trained language model is trained based on the second training set to obtain the second data classification model.

5. The field data collection method according to claim 4, characterized in that: The step of training the second pre-trained language model based on the second training set to obtain the second data classification model comprises: For any one of the plurality of preset training parameter configurations, training the second pre-trained language model under the training parameter configuration based on the second training set to obtain a second candidate data classification model; After traversing each training parameter, each second candidate data classification model under each training parameter configuration is obtained; Acquire a second validation set of the second pre-trained language model from the second training set, and evaluate the precision and recall of each of the second candidate data classification models based on the second validation set to obtain a second evaluation result; The second data classification model among the second candidate data classification models is determined according to the second evaluation result.

6. The field data collection method according to claim 5, characterized in that: The step of determining the second data classification model among the second candidate data classification models according to the second evaluation result comprises: Calculating the classification performance scores of the second candidate data classification models according to the second evaluation result, and determining the remaining classification performance scores within a preset score range of the highest classification performance score among the classification performance scores; If there are other classification performance scores, then selecting the target candidate data classification model with the highest recall rate among the target candidate data classification models corresponding to the highest classification performance score and the other classification performance scores as the second data classification model; If there are no other classification performance scores, the second candidate data classification model corresponding to the highest classification performance score is used as the second data classification model.

7. The field data collection method according to claim 1, characterized in that: The step of annotating the domain relevance of each data in the open source data set to obtain the annotated data set comprises: Performing data cleaning on the open source data set to obtain a cleaned open source data set; The cleaned open source data set is annotated with domain relevance by a preset large language model guided by preset prompt words to obtain the annotated data set.

8. A field data collection device, characterized in that: The field data collection device comprises: A domain annotation module is used to obtain an open source data set in an open source corpus and annotate each data in the open source data set with domain relevance to obtain an annotated data set; A domain screening module, used to screen the annotated data set according to the domain relevance of the annotations to obtain an annotated data subset related to the target domain; A compliance annotation module, used for performing compliance annotation on each data in the labeled data subset to obtain an annotated data subset; The compliance screening module is used to screen the annotation data subset according to the compliance of the annotations to obtain target domain data.

9. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the field data collection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the field data collection method according to any one of claims 1 to 7 are implemented.

Images