Fund name concept extraction method, device and related media based on deep learning

Through the fund name concept extraction model based on deep learning, using the RoBERTa-wwm model to train and predict, the problems of low accuracy and high maintenance cost in the existing technology are solved, and the fund concept extraction and reduction with high accuracy are achieved maintenance costs.

CN117972105BActive Publication Date: 2024-11-29CHINA MERCHANTS FUND MANAGEMENT CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311765710.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-21
Publication Date
2024-11-29
Estimated Expiration
2043-12-21

AI Technical Summary

Technical Problem

The existing fund concept extraction method has low accuracy and high maintenance costs, so it is impossible to effectively identify emerging fund concepts or their different expressions.

Method used

A deep learning-based method is used to construct a fund name concept extraction model, and the concept prediction and extraction of new input fund data are achieved by collecting and labeling fund data, preprocessing it, and training with the RoBERTa-wwm model.

Benefits of technology

It improves the accuracy of fund concept extraction, reduces maintenance costs, and can identify emerging fund concepts and their different expressions without manual update of dictionary.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117972105B_ABST
    Figure CN117972105B_ABST
Patent Text Reader

Abstract

The present invention discloses a fund name concept extraction method, device and related media based on deep learning. The method includes constructing an original fund data set, marking concept words in the original fund data set, and obtaining a training fund data set; performing data preprocessing on the training fund data set to obtain fund preprocessing data; constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model; using the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain a model prediction result; performing concept word conversion on the model prediction result to obtain a fund name concept extraction result. The present invention improves the accuracy of fund concept extraction and reduces maintenance costs by inputting newly input fund data into the trained fund concept extraction model for prediction, and extracting the fund name concept extraction result using the model prediction result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a fund name concept extraction method, device and related media based on deep learning. Background Art

[0002] In recent years, with the rapid development of the fund industry, the number of fund products and the investment concepts involved have continued to increase. By analyzing the investment field information in the fund application materials, we can extract and count the concepts of the applied funds, so as to grasp the hot concepts and development trends of the current newly issued funds and provide decision support for the issuance layout of fund products.

[0003] At present, the extraction of fund concepts mainly adopts the dictionary matching method, which involves collecting a large number of historical fund concept words, needs to build a Trie dictionary tree, and then matches the fund name and concept words through the Trie dictionary tree. However, the dictionary matching method has two disadvantages: the first is that the accuracy of extracting fund concepts by the dictionary matching method is not high; the second is that the dictionary matching method can only extract concepts that already exist in the dictionary, and cannot effectively identify newly emerging fund concepts or different expressions of the same concept, and requires manual continuous updating of the dictionary to cover the newly discovered fund concept words, which has a high maintenance cost. Summary of the invention

[0004] The embodiments of the present invention provide a fund name concept extraction method, device and related media based on deep learning, aiming to solve the problems of low accuracy and high maintenance cost of the fund concept extraction method in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a fund name concept extraction method based on deep learning, comprising:

[0006] Collect the data of the names of the declared funds and the names of the issued funds respectively to construct the original fund data set;

[0007] Respectively label each fund data in the original fund data set with a concept word to obtain a training fund data set;

[0008] Performing data preprocessing on the training fund data set to obtain fund preprocessing data;

[0009] Constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model;

[0010] Using the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain model prediction results;

[0011] The prediction results of the model are converted into concept words to obtain the fund name concept extraction results.

[0012] In a second aspect, an embodiment of the present invention provides a fund name concept extraction device based on deep learning, comprising:

[0013] A data construction unit, used to collect the declared fund name data and the issued fund name data respectively to construct the original fund data set;

[0014] A data labeling unit, used to label the concept words of each fund data in the original fund data set to obtain a training fund data set;

[0015] A data processing unit, used for performing data preprocessing on the training fund data set to obtain fund preprocessing data;

[0016] A model training unit, used for constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model;

[0017] A model prediction unit, used to use the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain a model prediction result;

[0018] The result output unit is used to convert the model prediction result into concept words to obtain the fund name concept extraction result.

[0019] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the fund name concept extraction method based on deep learning of the first aspect is implemented.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the fund name concept extraction method based on deep learning of the first aspect is implemented.

[0021] The embodiment of the present invention provides a fund name concept extraction method based on deep learning, including constructing an original fund data set, marking the concept words in the original fund data set, and obtaining a training fund data set; performing data preprocessing on the training fund data set to obtain fund preprocessing data; constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model; using the trained fund concept extraction model to perform concept prediction on the newly input fund data to obtain a model prediction result; performing concept word conversion on the model prediction result to obtain a fund name concept extraction result. The present invention improves the accuracy of fund concept extraction and reduces maintenance costs by inputting the newly input fund data into the trained fund concept extraction model for prediction, and extracting the fund name concept extraction result using the model prediction result.

[0022] An embodiment of the present invention also provides a fund name concept extraction device, computer equipment and storage medium based on deep learning, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0024] Figure 1 A flowchart of a fund name concept extraction method based on deep learning provided by an embodiment of the present invention;

[0025] Figure 2 A structural diagram of a fund concept extraction model provided by an embodiment of the present invention;

[0026] Figure 3 A prediction flow chart of a fund concept extraction model provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of label conversion provided by an embodiment of the present invention;

[0028] Figure 5 A schematic block diagram of a fund name concept extraction device based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0031] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0032] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] See below Figure 1 , Figure 1 A flowchart of a fund name concept extraction method based on deep learning provided in an embodiment of the present invention specifically includes: steps S101 to S106.

[0034] S101, respectively collecting the data of the name of the declared fund and the name of the issued fund to construct an original fund data set;

[0035] S102, respectively labeling the concept words of each fund data in the original fund data set to obtain a training fund data set;

[0036] S103, performing data preprocessing on the training fund data set to obtain fund preprocessing data;

[0037] S104, constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model;

[0038] S105, using the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain a model prediction result;

[0039] S106, converting the model prediction result into concept words to obtain a fund name concept extraction result.

[0040] In step S101, fund name information is collected from official websites of fund application and issuance or other reliable data sources. Application fund data refers to funds that are being applied for or approved, while issuance fund data refers to funds that have been issued to the public. The collected application fund and issuance fund names are integrated to form an original fund data set. This data set usually contains information on various types of funds on the market.

[0041] In one embodiment, the step S101 includes:

[0042] Obtain the fund name of the historically declared fund and obtain the declared fund name data;

[0043] Obtain the fund name of the historically issued fund and obtain the issuing fund name data;

[0044] Perform multi-dimensional stratified sampling on the declared fund name data and the issued fund name data respectively to obtain multiple fund sampling data; wherein the multi-dimensional stratified sampling includes fund type data sampling and declaration year and month data sampling;

[0045] The plurality of fund sampling data are merged to obtain the original fund data set.

[0046] In this embodiment, first, the names of funds declared in history are obtained and sorted into declared fund name data; the names of funds issued in history are obtained and sorted into issued fund name data; these fund data can be collected manually or exported in batches after screening the existing fund data. Then, multi-dimensional stratified sampling is performed on the declared and issued fund name data, including sampling by fund type and declaration year and month, to enhance the representativeness and diversity of the data set. Finally, the sampled fund data are merged to construct a comprehensive original fund data set. By including declared and issued funds, the data set provides a more comprehensive perspective for market analysis, and multi-dimensional sampling ensures that the data covers various fund types and time periods, enhancing the representativeness of the data set. These data can also be further used for trend analysis and forecasting to support the formulation of fund product strategies.

[0047] Specifically, considering that investment hotspots and financial innovations each year may lead to differences in the distribution of fund data, the multi-dimensional stratified sampling method can ensure that the data set covers various types of funds. This sampling strategy increases the coverage and diversity of the data set, thereby improving the accuracy and generalization ability of subsequent model training. In general, multi-dimensional stratified sampling is an effective data processing method that can significantly improve the quality of the data set and provide more comprehensive and representative data support for the subsequent training of deep learning models.

[0048] In step S102, each fund data in the original fund data set is annotated to ensure that each data is associated with at least one relevant concept word. Through this annotation, a training fund data set is formed, which will be used for subsequent deep learning model training. By ensuring that each data has a clear concept word annotation, the effectiveness of subsequent model training can be improved. Concept word annotation helps to reveal the inherent characteristics and market trends of fund data.

[0049] In one embodiment, the step S102 includes:

[0050] Performing concept word annotation on each fund data in the original fund data set according to a pre-specified annotation rule to obtain multiple annotation results;

[0051] A labeling detection is performed on the plurality of labeling results, and the labeling results that are correctly detected are integrated into the training fund data set.

[0052] In this embodiment, each fund data in the original fund data set is labeled with concept words according to a predetermined labeling rule to obtain multiple labeling results. All the labeling results are tested to ensure accuracy, and then the labeled results that are tested correctly are integrated into the training fund data set. The accuracy and consistency of data labeling are further improved through screening. The strictly tested and screened data set provides high-quality data for subsequent model training.

[0053] Specifically, the predetermined labeling rules are usually used to label funds according to their investment industries, themes (such as food and beverage, culture and media, the Internet, etc.), investment concepts (such as ESG, hydrogen energy, virtual reality, etc.), investment markets (such as Nasdaq, SGX, etc.), and fund investment styles (such as growth and value). These rules help to systematically organize and understand fund data, and provide standardized input for subsequent data analysis and model training.

[0054] In step S103, data preprocessing includes the following steps:

[0055] Data cleaning: remove duplicate, incomplete or inaccurate data records to ensure data quality;

[0056] Data standardization: converting data in different formats or units into a unified standard format for easy analysis;

[0057] Data conversion: convert text data into a format that the model can process, for example, tokenize the text and encode and vectorize the text;

[0058] Missing value processing: fill or delete missing values ​​in the data set.

[0059] Preprocessing ensures the accuracy and reliability of the data set, provides a solid foundation for model training, and simplifies the model training process through standardization and feature extraction.

[0060] In step S104, the model is trained using pre-processed fund data, including feeding a large amount of annotated fund data so that the model can learn to identify different types of fund concepts. The accuracy and efficiency of the model are improved by adjusting the model parameters and structure. The present invention uses deep learning technology to analyze the data of fund names, thereby constructing a model that can automatically extract key concepts from fund names. The model obtains a high-accuracy concept extraction capability by learning historical fund names and related concepts. The model can not only accurately extract concepts from known fund names, but also identify and extract new fund concepts that have not appeared in historical data. This automated concept extraction function enables the model to automatically and effectively predict newly declared funds after completing initial training without the need for manual dictionary maintenance. Therefore, the model can quickly provide key concept information of declared funds, assist in the market issuance strategy and decision-making process of new fund products, and thus play an important role in the fund industry.

[0061] The traditional Trie dictionary matching method relies solely on keywords for matching, but fails to fully utilize the grammatical and semantic information in the fund name, which limits its accuracy in concept extraction. In contrast, deep learning models, such as the famous BERT pre-trained model, can more comprehensively capture and express the deep semantic information in the text. The uniqueness of the BERT model is that it can generate specific embeddings for each word in different contexts, thereby accurately capturing the different semantics that the same word may exhibit in different sentences. This ability of deep semantic understanding makes the BERT-based model perform better in fund name concept extraction, providing higher accuracy and adaptability.

[0062] Specifically, the deep learning model used in this example is RoBERTa-wwm, which is based on the RoBERTa model and is specially optimized and adjusted for Chinese text. These optimizations include:

[0063] Increased training data set: used a larger data set than the original BERT model and prolonged the training time;

[0064] Increase batch size: Use a larger batch size to improve training efficiency and model performance;

[0065] Text encoding optimization: BPE (Byte Pair Encoding) encoding method is used to effectively solve the problem of unregistered words (words not in the dictionary);

[0066] Optimizer parameter adjustment: The parameters of the optimizer algorithm were adjusted to improve the training effect;

[0067] Remove the NSP task: Remove the Next Sentence Prediction task in BERT to focus on more critical tasks;

[0068] Dynamic masking strategy: Different from BERT’s static masking, RoBERTa uses dynamic masking, applying different masking methods to the same data for multiple trainings, which improves the model’s ability to understand text;

[0069] Whole Word Masking Application: In the Whole Word Masking strategy, all components of the masked word will be masked together.

[0070] These improvements make RoBERTa-wwm perform better when processing Chinese text, especially in understanding complex semantics and context, providing strong support for fund concept extraction.

[0071] Combination Figure 2 As shown, in one embodiment, the step S104 includes:

[0072] Initialize the concept extraction model, and pass the surface information features output by the bottom Transformer layer in the concept extraction model, the syntactic information features output by the middle Transformer layer in the concept extraction model, and the semantic information features output by the top Transformer layer in the concept extraction model;

[0073] Input the surface information features output by the bottom Transformer layer, the syntactic information features output by the middle Transformer layer, and the semantic information features output by the top Transformer layer into an Attention layer;

[0074] Based on the Attention mechanism, the Attention layer is used to learn the importance values ​​of all the Transformer layers, and the output of the learned Attention layer is connected to a CRF layer to obtain the fund concept extraction model.

[0075] In this embodiment, a Transformer-based concept extraction model is initialized, surface information features are extracted from the bottom Transformer layer of the model, syntactic information features are extracted from the middle Transformer layer of the model, and semantic information features are extracted from the top Transformer layer of the model; the above three information features are input into an Attention layer; the Attention mechanism is then used to learn the importance of different Transformer layers in this layer, and finally the output processed by the Attention layer is connected to a CRF layer to complete the construction of the fund concept extraction model.

[0076] Specifically, the fund concept extraction task is defined as a sequence labeling problem in the field of natural language processing, and the fund concept words are extracted from the fund name text through named entity recognition (NER) technology. Although the BERT-BiLSTM-CRF structure is one of the mainstream models for the NER task, the model contains a large number of parameters that need to be trained from scratch, especially the BiLSTM layer and the CRF layer. In the fund name concept extraction task, due to the limited total amount of fund name data, this structure may not achieve the best effect. Therefore, the present invention removes the BiLSTM layer and adopts a more simplified BERT-CRF model structure. This adjustment enables the model to extract fund concepts more effectively in an environment with limited data volume, which is better than the traditional BERT-BiLSTM-CRF model.

[0077] The present invention uses RoBERTa-wwm, which is optimized based on the BERT model, to replace the BERT model. The ability of the RoBERTa-wwm model to extract semantic information is better than that of the BERT model. The obtained model structure is RoBERTa-wwm-CRF, and the effect of the model is further improved. In the RoBERTa model, taking RoBERTa-base as an example (such as Figure 2As shown), in this embodiment, the model has 12 layers of Transformer layers, and the meaning of the output expression of each layer of Transformer from low to high is different, wherein the bottom network represents surface information features, the middle network represents syntactic information features, and the high-level network represents semantic information features. When using the RoBERTa model for classification and sequence labeling tasks, the Embedding output by the last layer of Transformer that expresses high-level semantic information is generally used as the text representation input into the subsequent network. However, in the task of fund name concept extraction, the characteristics of the fund name are that the text length is medium and short, and the fund naming method has a relatively fixed structure, so grammatical information plays a certain role in this task. Therefore, the present invention uses the Transformer of all layers of the RoBERTa model to connect to an Attention layer, and through the Attention mechanism, the importance of each layer of Transformer is automatically learned during the training process, and then the Attention layer is followed by a CRF layer to obtain the RoBERTa-wwm-all-CRF model (i.e., the fund concept extraction model).

[0078] In one embodiment, the step S104 includes:

[0079] The learning rate preheating and decay settings of the fund concept extraction model are performed according to the following formula:

[0080]

[0081] Among them, lr represents the learning rate; lr0 represents the initial learning rate; lr1 represents the learning rate after the warm-up phase; lr2 represents the learning rate after the decay phase; step represents the number of training iterations; warmup steps Indicates the number of iterations in the warm-up phase; decay steps Indicates the number of iterations at the end of the decay phase.

[0082] In this embodiment, there are two characteristics when training the RoBERTa-wwm-all-CRF model: the first characteristic is that when training the RoBERTa-wwm-all-CRF model, a layered learning rate method is adopted. Since the parameters of the RoBERTa layer are pre-trained with a large amount of data, this layer uses a lower learning rate during the training process. On the contrary, the CRF layer is used as an initialization parameter and has not been pre-trained, and a higher learning rate is required to achieve rapid convergence. In particular, in the fund concept extraction task with a small amount of data, the rapid convergence of the CRF layer is crucial. Therefore, the learning rate of the CRF layer is set to more than ten times that of the RoBERTa layer to promote the optimization and improvement of the overall model. The second characteristic is that in the early stage of training the RoBERTa-wwm-all-CRF model, a smaller learning rate is selected to preheat the model to prevent damage to the pre-trained parameters of the RoBERTa layer and avoid premature instability of the untrained parameters of the CRF layer. As the training proceeds, the learning rate is gradually increased to accelerate the convergence of the model parameters. This method aims to balance the stability of the model with the need for rapid convergence to ensure that the entire training process is both effective and efficient.

[0083] In addition, in the early stage of model training, linear learning rate warm-up plus warm-up strategy learning rate is used; in the later stage of model training, the parameters have gradually stabilized and the learning rate needs to be reduced to avoid model parameter oscillation. In the later stage, reducing the learning rate can achieve better convergence, that is, using linear learning rate decay plus decay strategy learning rate. It should be noted that lr0, lr1, lr2, warmup are pre-set before training starts. steps and decay steps And other parameters.

[0084] In step S105, the trained fund concept extraction model is first used, and then the new fund data is processed: new fund name data is input into the model; the model analyzes the input fund name and predicts related fund concepts; the prediction results of the model are collected, and these results provide a basis for the next step.

[0085] Combination Figure 3 and Figure 4 As shown, in one embodiment, the step S105 includes:

[0086] The trained fund concept extraction model is used to perform concept prediction on the newly input fund data to obtain the probability that each text character of the newly input fund data belongs to each label, and the label with the largest probability is taken as the label of the corresponding character; wherein the labels include: concept start characters, concept middle characters, concept end characters, single-word concept characters, and non-concept characters.

[0087] In this embodiment, new fund data is input into the trained model; the model analyzes each input text character and predicts the probability of the character belonging to each label; based on the prediction results, the label with the highest probability is assigned to each character, and these labels include concept start characters (B, Begin), concept middle characters (M, Middle), concept end characters (E, End), single-word concept characters (S, Single) and non-concept characters (O, Outside).

[0088] For example, Figure 4 As shown in the figure, "small and medium-sized plates" correspond to "medium" for "B", "small" for "M", and "plate" for "E". For another example, "growth" corresponds to "mature" for "B" and "long" for "E".

[0089] In step S106, first, the model predicts the role of each character in the fund name, such as whether it is the beginning, middle, end of the concept, or an independent concept character. Then, based on these predictions, the character string is linked into a complete concept word. For example, a continuous sequence of "begin-middle-end" tags will be merged into a complete concept word. This process eventually produces a list of all recognized concept words, which constitute the concept extraction result of the fund name.

[0090] In one embodiment, the step S106 includes:

[0091] Concept words are extracted from each of the characters through the labels of each of the characters to obtain the concept extraction result of the fund name.

[0092] In this embodiment, the label of each character predicted by the analysis model is used; consecutive concept start (B), concept middle (M), and concept end (E) labels are connected in series to form a complete concept word, and the characters marked as single-word concepts (S) are directly regarded as independent concept words; the concept words extracted according to the above process constitute the final fund name concept extraction result.

[0093] Combination Figure 5 As shown, Figure 5 A schematic block diagram of a fund name concept extraction device based on deep learning provided by an embodiment of the present invention, wherein the fund name concept extraction device based on deep learning 500 includes:

[0094] The data construction unit 501 is used to collect the declared fund name data and the issued fund name data respectively to construct the original fund data set;

[0095] A data labeling unit 502 is used to label the concept words of each fund data in the original fund data set to obtain a training fund data set;

[0096] A data processing unit 503 is used to perform data preprocessing on the training fund data set to obtain fund preprocessing data;

[0097] A model training unit 504 is used to construct a fund concept extraction model, and use the fund pre-processing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model;

[0098] The model prediction unit 505 is used to use the trained fund concept extraction model to perform concept prediction on the newly input fund data to obtain a model prediction result;

[0099] The result output unit 506 is used to convert the model prediction result into concept words to obtain the fund name concept extraction result.

[0100] In this embodiment, the data construction unit 501 collects the declared fund name data and the issued fund name data respectively to construct the original fund data set; the data annotation unit 502 respectively annotates the concept words of each fund data in the original fund data set to obtain the training fund data set; the data processing unit 503 performs data preprocessing on the training fund data set to obtain fund preprocessing data; the model training unit 504 constructs a fund concept extraction model, and uses the fund preprocessing data to train and optimize the fund concept extraction model to obtain the trained fund concept extraction model; the model prediction unit 505 uses the trained fund concept extraction model to perform concept prediction on the newly input fund data to obtain the model prediction result; the result output unit 506 converts the model prediction result into concept words to obtain the fund name concept extraction result.

[0101] In one embodiment, the data construction unit 501 includes:

[0102] A first acquisition unit is used to acquire the fund name of the historically declared fund and obtain the declared fund name data;

[0103] A second acquisition unit is used to acquire the fund name of the historically issued fund and obtain the issued fund name data;

[0104] A fund sampling unit, used to perform multi-dimensional stratified sampling on the declared fund name data and the issued fund name data respectively, to obtain a plurality of fund sampling data; wherein the multi-dimensional stratified sampling includes fund type data sampling and declared year and month data sampling;

[0105] The sampling and merging unit is used to merge the multiple fund sampling data to obtain the original fund data set.

[0106] In one embodiment, the data annotation unit 502 includes:

[0107] A fund labeling unit, used to label each fund data in the original fund data set with a concept word according to a pre-specified labeling rule to obtain multiple labeling results;

[0108] The fund detection unit is used to perform tag detection on the multiple tag results, and integrate the tag results that are correctly detected into the training fund data set.

[0109] In one embodiment, the model training unit 504 includes:

[0110] A model initialization unit, used to initialize the concept extraction model, and pass the surface information features output by the bottom Transformer layer in the concept extraction model, pass the syntactic information features output by the middle Transformer layer in the concept extraction model, and pass the semantic information features output by the top Transformer layer in the concept extraction model;

[0111] A model learning unit, used for inputting the surface information features output by the bottom Transformer layer, the syntactic information features output by the middle Transformer layer, and the semantic information features output by the top Transformer layer into an Attention layer;

[0112] The model access unit is used to learn the importance values ​​of all the Transformer layers using the Attention layer based on the Attention mechanism, and access the output of the learned Attention layer to a CRF layer to obtain the fund concept extraction model.

[0113] In one embodiment, the model training unit 504 includes:

[0114] The model calculation unit is used to set the learning rate attenuation of the fund concept extraction model according to the following formula:

[0115]

[0116] Among them, lr represents the learning rate; lr0 represents the initial learning rate; lr1 represents the learning rate after the warm-up phase; step represents the number of training iterations; warmup steps Indicates the number of iterations in the warm-up phase; decay steps Indicates the number of iterations at the end of the decay phase.

[0117] In one embodiment, the model prediction unit 505 includes:

[0118] The label calculation unit is used to use the trained fund concept extraction model to perform concept prediction on the newly input fund data, obtain the probability of each text character of the newly input fund data being predicted to belong to each label, and take the label with the largest probability as the label of the corresponding character; wherein the labels include: concept start characters, concept middle characters, concept end characters, single-word concept characters, and non-concept characters.

[0119] In one embodiment, the result output unit 506 includes:

[0120] The result extraction unit is used to extract concept words from each of the characters through the labels of each of the characters to obtain the fund name concept extraction result.

[0121] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0122] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0123] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0124] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0125] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A fund name concept extraction method based on deep learning, characterized in that: include: Collect the data of the names of the declared funds and the names of the issued funds respectively to construct the original fund data set; Respectively label each fund data in the original fund data set with a concept word to obtain a training fund data set; Performing data preprocessing on the training fund data set to obtain fund preprocessing data; Constructing a fund concept extraction model, and using the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model; wherein the training optimization includes: increasing the training data set, increasing the batch size, using BPE encoding for text optimization, adjusting optimizer parameters, removing NSP tasks, using dynamic masks to apply different masking methods to the same data for multiple training, and applying full-word masking; Using the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain model prediction results; Performing concept word conversion on the model prediction results to obtain fund name concept extraction results; The construction of the fund concept extraction model includes: initializing the RoBERTa-wwm model, and using the surface information features output by the bottom Transformer layer in the RoBERTa-wwm model, the syntactic information features output by the middle Transformer layer in the RoBERTa-wwm model, and the semantic information features output by the top Transformer layer in the RoBERTa-wwm model; inputting the surface information features output by the bottom Transformer layer, the syntactic information features output by the middle Transformer layer, and the semantic information features output by the top Transformer layer into an Attention layer; using the Attention layer to learn the importance values ​​of all Transformer layers based on the Attention mechanism, and connecting the output of the learned Attention layer to a CRF layer to obtain the fund concept extraction model; The method uses the trained fund concept extraction model to perform concept prediction on the newly input fund data to obtain model prediction results, including: using the trained fund concept extraction model to perform concept prediction on the newly input fund data to obtain the probability that each text character of the newly input fund data belongs to each label, and taking the label with the largest probability as the label of the corresponding character; wherein the labels include: concept start characters, concept middle characters, concept end characters, single-word concept characters, and non-concept characters.

2. The fund name concept extraction method based on deep learning according to claim 1 is characterized in that: The data of the declared fund names and the issued fund names are collected separately to construct the original fund data set, including: Obtain the fund name of the historically declared fund and obtain the declared fund name data; Obtain the fund name of the historically issued fund and obtain the issuing fund name data; Perform multi-dimensional stratified sampling on the declared fund name data and the issued fund name data respectively to obtain multiple fund sampling data; wherein the multi-dimensional stratified sampling includes fund type data sampling and declaration year and month data sampling; The plurality of fund sampling data are merged to obtain the original fund data set.

3. The fund name concept extraction method based on deep learning according to claim 1 is characterized in that: The step of respectively labeling the concept words of each fund data in the original fund data set to obtain a training fund data set includes: Performing concept word annotation on each fund data in the original fund data set according to a pre-specified annotation rule to obtain multiple annotation results; A labeling detection is performed on the plurality of labeling results, and the labeling results that are correctly detected are integrated into the training fund data set.

4. The fund name concept extraction method based on deep learning according to claim 1 is characterized in that: The method of using the fund pre-processing data to train and optimize the fund concept extraction model to obtain the trained fund concept extraction model includes: The learning rate preheating and decay settings of the fund concept extraction model are performed according to the following formula: ; in, lr represents the learning rate; lr 0 represents the initial learning rate; lr 1 represents the learning rate after the warm-up phase; lr 2 represents the learning rate after the decay phase; step Indicates the number of training iterations; warmup steps Indicates the number of iterations in the warm-up phase; decay steps Indicates the number of iterations at the end of the decay phase.

5. The fund name concept extraction method based on deep learning according to claim 1 is characterized in that: The model prediction result is converted into a concept word to obtain a fund name concept extraction result, including: Concept words are extracted from each of the characters through the labels of each of the characters to obtain the concept extraction result of the fund name.

6. A fund name concept extraction device based on deep learning, characterized in that: include: A data construction unit, used to collect the declared fund name data and the issued fund name data respectively to construct the original fund data set; A data labeling unit, used to label the concept words of each fund data in the original fund data set to obtain a training fund data set; A data processing unit, used for performing data preprocessing on the training fund data set to obtain fund preprocessing data; A model training unit is used to construct a fund concept extraction model, and use the fund preprocessing data to train and optimize the fund concept extraction model to obtain a trained fund concept extraction model; wherein the training optimization includes: increasing the training data set, increasing the batch size, using BPE encoding for text optimization, optimizing optimizer parameters, removing NSP tasks, using dynamic masks to apply different masking methods to the same data for multiple training, and applying full-word masking; A model prediction unit, used to use the trained fund concept extraction model to perform concept prediction on newly input fund data to obtain a model prediction result; A result output unit, used to convert the model prediction result into concept words to obtain a fund name concept extraction result; The model training unit includes: a model initialization unit, which is used to initialize the concept extraction model and input the surface information features output by the bottom Transformer layer in the concept extraction model, the syntactic information features output by the middle Transformer layer in the concept extraction model, and the semantic information features output by the top Transformer layer in the concept extraction model; a model learning unit, which is used to input the surface information features output by the bottom Transformer layer, the syntactic information features output by the middle Transformer layer, and the semantic information features output by the top Transformer layer into an Attention layer; a model access unit, which is used to learn the importance values ​​of all Transformer layers using the Attention layer based on the Attention mechanism, and access the output of the learned Attention layer to a CRF layer to obtain the fund concept extraction model; The model prediction unit includes: a label calculation unit, which is used to use the trained fund concept extraction model to perform concept prediction on the newly input fund data, obtain the probability of each text character of the newly input fund data belonging to each label, and take the label with the largest probability as the label of the corresponding character; wherein, the label includes: concept start character, concept middle character, concept end character, single word concept character, and non-concept character.

7. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the fund name concept extraction method based on deep learning as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting fund name concepts based on deep learning as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Method and system for extracting knowledge data, computer device and storage medium

    CN109189848A