Element extraction method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence and finance, and relates to an element extraction method, which comprises the following steps: constructing an initial rule extraction model according to a business extraction demand and a predefined rule, and training the initial rule extraction model to obtain a trained rule extraction model; obtaining a service sample data set, and inputting the service sample data set into the rule extraction model to obtain a correct element recognition result; marking the business sample data set based on the correct element identification result, constructing a fine adjustment data set, and training a pre-trained element extraction model through the fine adjustment data set to obtain a final business element extraction model; and inputting the to-be-identified business text into the business element extraction model to obtain a business element extraction result. The invention further provides an element extraction device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the business sample data set can be stored in a block chain. According to the invention, the element extraction efficiency and accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a factor extraction method, device, computer equipment and storage medium. Background Art

[0002] In the field of financial investment, bonds are an important financing and investment tool, and the accuracy and timeliness of their information are crucial to investors and financial institutions. Bond issuance information usually contains a large amount of key data, such as the bond name, issuance start date, payment date, etc. These data are important factors that investment managers must consider when making investment decisions and risk management.

[0003] Currently, when investment managers enter new bond information, they need to manually extract this key information from the text segment of the bond issuance information. This step is not only tedious and complex, but also time-consuming and labor-intensive. Manual information extraction is not only inefficient but also prone to errors because the information in the text segment is often presented in different formats and arrangements, making it difficult to ensure the accuracy and completeness of the information.

[0004] In addition, with the development of the financial market, the number of new bond issuances continues to increase, and the amount of bond issuance information that investment managers need to process is also increasing. This further increases the burden of manual information extraction, making it difficult for investment managers to complete a large amount of information entry work in a short period of time. Therefore, the existing information extraction methods can no longer meet the requirements of the financial market for information processing speed and accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a factor extraction method, device, computer equipment and storage medium to solve the technical problems of low efficiency and easy errors in information extraction in related technologies.

[0006] In order to solve the above technical problems, the present application embodiment provides a factor extraction method, which adopts the following technical solution:

[0007] Constructing an initial rule extraction model according to business extraction requirements and predefined rules, and training the initial rule extraction model to obtain a trained rule extraction model;

[0008] Obtaining a business sample data set, inputting the business sample data set into the rule extraction model, and obtaining a correct element recognition result;

[0009] Annotating the business sample data set based on the correct element recognition result, constructing a fine-tuning data set, and dividing the fine-tuning data set into a training set and a test set;

[0010] Inputting the training set into a pre-trained factor extraction model to obtain factor prediction results;

[0011] Calculating the loss value between the element prediction result and the correct element recognition result according to a preset loss function, adjusting the model parameters of the element extraction model based on the loss value, continuing iterative training until the model converges, and outputting the extraction model to be verified;

[0012] Testing the extraction model to be verified by using the test set to obtain a test result, and when the test result meets a preset threshold condition, outputting the extraction model to be verified as a final business element extraction model;

[0013] The business text to be identified is obtained, and the business text to be identified is input into the business element extraction model to obtain a business element extraction result.

[0014] In order to solve the above technical problems, the embodiment of the present application further provides an element extraction device, which adopts the following technical solution:

[0015] A rule construction module is used to construct an initial rule extraction model according to business extraction requirements and predefined rules, and train the initial rule extraction model to obtain a trained rule extraction model;

[0016] A rule extraction module is used to obtain a business sample data set, input the business sample data set into the rule extraction model, and obtain a correct element recognition result;

[0017] A labeling module, used to label the business sample data set based on the correct element recognition result, construct a fine-tuning data set, and divide the fine-tuning data set into a training set and a test set;

[0018] A prediction module, used for inputting the training set into a pre-trained factor extraction model to obtain a factor prediction result;

[0019] An iterative module, used to calculate the loss value between the element prediction result and the correct element recognition result according to a preset loss function, adjust the model parameters of the element extraction model based on the loss value, continue iterative training until the model converges, and output the extraction model to be verified;

[0020] A testing module, used to test the extraction model to be verified through the test set to obtain a test result, and when the test result meets a preset threshold condition, output the extraction model to be verified as the final business element extraction model;

[0021] The element extraction module is used to obtain the business text to be identified, input the business text to be identified into the business element extraction model, and obtain the business element extraction result.

[0022] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0023] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned element extraction method when executing the computer-readable instructions.

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0025] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the element extraction method described above are implemented.

[0026] Compared with the prior art, this application has the following beneficial effects:

[0027] The present application provides a factor extraction method, which constructs an initial rule extraction model according to business extraction requirements and predefined rules, trains the initial rule extraction model, and uses the trained rule extraction model to extract factors from a business sample data set, thereby improving the factor recognition efficiency and accuracy of the data set, thereby improving the quality of the data set; annotates a business text data set with correct factor recognition results, constructs a fine-tuning data set, saves time and cost for data annotation, and improves the accuracy of the data set; trains the pre-trained factor extraction model with the fine-tuning data set to obtain a final business factor extraction model, and uses the business factor extraction model to perform factor recognition and extraction on business texts, thereby realizing automated factor extraction, improving factor extraction efficiency and accuracy, and thereby providing a more efficient and accurate technical means for information processing in the field of financial investment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0030] Figure 2 is a flow chart of an embodiment of a method for extracting elements according to the present application;

[0031] Figure 3 is a structural schematic diagram of an embodiment of a factor extraction device according to the present application;

[0032] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0034] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0036] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0037] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0038] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0039] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0040] It should be noted that the factor extraction method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the factor extraction device is generally set in the server / terminal device.

[0041] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0042] Continue to refer Figure 2 , shows a flowchart of an embodiment of the factor extraction method according to the present application, comprising the following steps:

[0043] Step S201, constructing an initial rule extraction model according to business extraction requirements and predefined rules, and training the initial rule extraction model to obtain a trained rule extraction model.

[0044] Among them, business extraction requirements refer to the requirements for key information or data items that need to be extracted from business data, documents, texts or other information sources in a specific business scenario in order to support business decision-making, process optimization, data analysis or automated processing. Predefined rules are extraction rules for predefined elements, which are used to guide how to accurately extract key information from text segments of information sources.

[0045] Specifically, multiple extraction fields are determined according to business extraction requirements, and an extraction element set is determined based on the multiple extraction fields; an initial dictionary is constructed based on the extraction element set, and the initial dictionary is synonymously expanded using a preset semantic co-occurrence network to obtain a keyword dictionary corresponding to the business extraction requirements; extraction rules for the extraction element set are constructed, and an initial rule extraction model is formed based on the extraction rules and the keyword dictionary.

[0046] In this embodiment, the extracted fields are specific data items or information fragments identified and extracted from the original data or text information. Taking bond issuance information as an example, the extracted fields usually involve multiple key information of the bonds and their issuance, including basic information (such as bond abbreviation, bond full name, bond code, ISIN code, bond type, etc.), issuing entity information (such as issuer name, issuer region, etc.), issuance details (such as issuance date, maturity date, bond term, issuance scale, issuance method, etc.), credit rating information, and trading market information.

[0047] Extraction elements are determined based on multiple extraction fields, and the extraction elements are combined to form an extraction element set. Extraction elements include industry terminology elements, entity elements, and rule extraction elements; among them, industry terminology elements refer to text fragments corresponding to industry terms; entity elements refer to text fragments corresponding to named entities; rule extraction elements refer to certain rules or algorithms, which can extract text fragments that meet the rules or algorithms, such as regular expressions.

[0048] In this embodiment, a keyword list corresponding to the business scenario is obtained, and keywords in the keyword list that are the same or similar to each extracted element in the extracted element set are queried as preliminary keywords, and an initial dictionary is constructed based on all preliminary keywords; a historical business data set is obtained, and words associated with each preliminary keyword are searched from the historical business data set as extended words based on a preset semantic co-occurrence network, and a keyword dictionary is constructed based on the preliminary keywords and the corresponding extended words. Among them, the keyword list can be provided by the business party, or collected from information sources such as professional books, academic papers, industry reports, and website content; the keyword dictionary also includes clear and accurate definitions and explanations for each keyword.

[0049] In this embodiment, extraction rules are constructed based on the data types and logical relationships of the extracted elements in the extraction element set. The logical relationships include the order relationship and position relationship of the extracted elements. The extraction rules include regular expressions, special extraction rules, date extraction rules, price extraction rules, etc. Different data types can correspond to different extraction rules. Most of the extracted elements can be extracted through regular expressions. In the process of regular expression extraction, according to the special needs of different extracted elements, logical processes are embedded in the processing of different extracted elements, including keyword priority (such as A1>A2), extraction priority, etc. The examples are as follows:

[0050] If A1 is available, take A1; if A1 is not available, take total scale;

[0051] If keyword extraction fails, the first occurrence of [ ] is used as the creditor's right name;

[0052] For dynamic identification of bonds above level A2, the maximum number of bonds to be identified can be set, such as 20.

[0053] For elements that cannot be extracted by regular expressions, special extraction rules are defined. The following is an example:

[0054] 1) Special forms of issuance time: expected to be issued in the early part of x month of x year, bookkeeping in the late part of xx month, etc.;

[0055] 2) Special form of payment time: payment in the early part of x month of x year;

[0056] 3) Special form of separation of month and day: bookkeeping on x month x day, payment on x day;

[0057] 4) Processing xy files: special price ranges such as 3.5-4.5%, etc.

[0058] For date and price, corresponding extraction rules are also constructed, as follows:

[0059] 1) Reading and standardization of different price units (100 million, 100 million, 10,000 yuan, and yuan);

[0060] 2) Reading of different price range formats (xx%-xx%, xx-xx%, etc.);

[0061] 3) Reading and standardization of different date formats (including different date separators and different date formats).

[0062] The constructed extraction rules are integrated with the keyword dictionary to form a rule-based initial rule extraction model, and the initial rule extraction model is trained.

[0063] By building an initial rule extraction model, the process of factor extraction can be standardized, facilitating subsequent data processing and analysis, saving a lot of time and cost.

[0064] In some optional implementations of this embodiment, the step of training the initial rule extraction model to obtain a trained rule extraction model includes:

[0065] Acquire a business data set, preprocess the business data set, and obtain preprocessed text data;

[0066] Input the preprocessed text data into the initial rule extraction model, call the keyword dictionary to match the preprocessed text data, and extract the preliminary element results;

[0067] Evaluate the preliminary element results to obtain evaluation results;

[0068] If the evaluation result does not meet expectations, the initial rule extraction model is iteratively optimized until the evaluation result meets expectations, thereby obtaining a trained rule extraction model.

[0069] Among them, the business data set is the collected business text data without element annotation. The preprocessing includes cleaning, normalization, word segmentation and standardized format. Cleaning includes removing irrelevant characters and special symbols, removing stop words, and filling or deleting missing values; normalization is to use the normalization method to scale the data value to the range of [0,1] to obtain the normalized text data, and use word segmentation technology to segment the normalized text data.

[0070] The preprocessed text data is input into the initial rule extraction model, and the keyword dictionary is called to locate the key information in the preprocessed text data. Then, the extraction rules in the initial rule extraction model are used to quickly and accurately match the key information according to the located information, and the preliminary element results are extracted. The preliminary element results are submitted to the annotation personnel, and the feedback results after the annotation personnel manually verify and correct the preliminary element results are received. The recognition accuracy of the preliminary element results is determined according to the feedback results. When the recognition accuracy is greater than or equal to the preset accuracy threshold, it is determined that the evaluation result meets the expectations; otherwise, it is determined that the evaluation result does not meet the expectations, and the extraction rules in the initial rule extraction model are adjusted and optimized. Training continues until the recognition accuracy is greater than or equal to the preset accuracy threshold, that is, a trained rule extraction model is obtained.

[0071] In this embodiment, the steps of inputting the preprocessed text data into the initial rule extraction model, calling the keyword dictionary to match the preprocessed text data, and extracting preliminary element results include:

[0072] Traverse the keyword dictionary based on the preprocessed text data to determine the element keyword position;

[0073] Based on the position of the element keywords, the extraction rules of the initial rule extraction model are used to extract the initial element results.

[0074] The keyword dictionary is used to locate the element keywords in the text data and obtain the element keyword position. Based on the location of the element keyword, the extraction rules are used to accurately match the element keywords and extract the identified preliminary element results.

[0075] Through keyword matching, texts containing specific keywords can be screened out, narrowing the scope of information extraction. Combined with extraction rules for precise matching and extraction of key information, efficient and accurate information extraction can be achieved.

[0076] By training the rule extraction model, the efficiency and accuracy of feature identification can be improved, thereby improving data quality and accuracy, and enhancing data availability and scalability.

[0077] Step S202, obtaining a business sample data set, inputting the business sample data set into a rule extraction model, and obtaining a correct element recognition result.

[0078] In this embodiment, the rule extraction model is used to extract elements from each business sample data in the acquired business sample data set to obtain a correct element recognition result corresponding to each business sample data.

[0079] In some optional implementations, the business sample data set may be obtained from a business data set for training an initial rule extraction model.

[0080] In this embodiment, factor extraction is performed through a rule extraction model, thereby achieving factor extraction through rule cold start.

[0081] It should be emphasized that in order to further ensure the privacy and security of the business sample data set, the above-mentioned business sample data set can also be stored in a node of a blockchain.

[0082] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0083] Step S203: annotate the business sample data set based on the correct element recognition result, construct a fine-tuning data set, and divide the fine-tuning data set into a training set and a test set.

[0084] In this embodiment, the correct element recognition result is used as the real element label, and the business sample data set is annotated by the real element label to obtain the annotated business sample data set, that is, the fine-tuning data set.

[0085] The fine-tuning data set is randomly divided into a training set and a test set according to a preset ratio. The preset ratio can be set according to actual needs. For example, the preset ratio is 8:2, that is, the ratio of the training set to the test set is 8:2.

[0086] Step S204, inputting the training set into the pre-trained factor extraction model to obtain the factor prediction result.

[0087] In this embodiment, the factor extraction model is pre-trained using an improved LLM (Large Language Model) with small parameters. The factor extraction model includes a BERT encoding network, a BiLSTM decoding network, and an output layer connected in sequence. The training set is input into the pre-trained factor extraction model, and the business sample data in the training set is processed in sequence by the BERT encoding network, the BiLSTM decoding network, and the output layer to obtain the factor prediction results of the model.

[0088] In some optional implementations, the step of inputting the training set into the pre-trained factor extraction model to obtain the factor prediction result includes:

[0089] Inputting the training set into a pre-trained feature extraction model, wherein the feature extraction model includes a BERT encoding network, a BiLSTM decoding network, and an output layer;

[0090] The business sample data in the training set is encoded through the BERT encoding network to obtain text semantic features;

[0091] The BiLSTM decoding network is used to capture the contextual information of text semantic features and obtain text sequence features;

[0092] The text sequence features are input into the output layer for decoding to obtain the feature prediction results.

[0093] In this embodiment, the BERT encoding network includes an embedding layer, a spatial extension layer and an encoder layer. Specifically, the training set is input into the BERT encoding network, and the business sample data in the training set is vectorized through the embedding layer to obtain a text embedding vector; the text embedding vector is input into the spatial extension layer, and the text embedding vector and the introduced prompt vector are concatenated to obtain a text extension feature vector; and the encoder layer performs multi-head attention calculation on the text extension feature vector to obtain text semantic features.

[0094] The business sample data is preprocessed through the embedding layer, including word segmentation, part-of-speech tagging, etc., and then the preprocessed sample data is converted into a text vector that the model can understand. Vector conversion includes position embedding, word embedding, and character embedding of the business sample data. Among them, position embedding provides information about the position of words in the sentence, which helps to understand the grammatical structure and contextual relationship of the sentence; word embedding captures the semantic information of the vocabulary, allowing the model to understand the meaning of the vocabulary in the context; character embedding captures character-level information, which helps the model process unknown vocabulary or spelling variants. By incorporating character-level information into the model, the model's ability to process complex data is improved. The text embedding vector includes the concatenation vector of the position embedding vector, word embedding vector, and character embedding vector.

[0095] The spatial expansion layer introduces a learnable prompt vector pool to spatially expand the text embedding vector to enrich the feature representation. Specifically, the text embedding vector is input into the spatial expansion layer, and the pre-trained prompt vector pool is obtained through the spatial expansion layer. The prompt vector pool contains S vectors, where S is a positive integer; the pre-trained k prompt vectors are obtained from the prompt vector pool, where k is a positive integer and is less than or equal to S; the text embedding vector is concatenated with the k prompt vectors to obtain the text expansion feature vector. By introducing the prompt vector, additional information or constraints are actually introduced on the basis of the original feature space, thereby expanding the feature space, allowing the model to capture more features related to the feature information, thereby improving the extraction accuracy; at the same time, the generalization ability of the model is enhanced, the computational efficiency is improved, and the training process of the model is optimized.

[0096] In the present embodiment, the semantics and context features of the text extension feature vector are extracted by the multi-head attention mechanism of the encoder layer to obtain the text semantic features. In some optional implementations, the encoder layer adopts multiple encoders connected in stacks, and the text extension feature vector is encoded in turn by multi-layer encoders to extract the semantics and context features of the business sample data. The input of the latter encoder is the output of the previous encoder and the text extension feature vector. After the processing of the encoder, the deep semantic feature representation of the business sample data can be obtained, and the output of the last layer of encoder is usually used as the text semantic feature.

[0097] Specifically, each encoder includes a multi-head self-attention layer and a feedforward network layer. The text extension feature vector is input into the multi-head self-attention layer for weight calculation to obtain the text attention fusion feature; the text attention fusion feature is optimized through the feedforward network layer to obtain the text semantic feature.

[0098] By encoding business sample data through the BERT encoding network, the model can capture more features related to element information, and use the multi-head attention mechanism to weightedly process the text extension feature vector, so that the model pays more attention to the key information of elements in the text, making the model more adaptable to complex text types, thereby improving the accuracy and robustness of element recognition.

[0099] In this embodiment, a BiLSTM decoding network is used to decode text semantic features to obtain text sequence features. The BiLSTM decoding network includes at least two connected LSTM layers, each LSTM layer processes information of different time scales to capture features of different time scales, and the input of the next LSTM layer is the output of the previous LSTM layer and the input text semantic features, wherein the specific number of LSTM layers can be set according to actual needs.

[0100] The output layer can use a CRF layer or a fully connected layer. When the output layer uses a CRF layer, the text sequence features are input into the CRF layer, and the optimal label sequence is obtained through the Viterbi decoding algorithm. The feature prediction result can be obtained based on the label sequence; when the output layer uses a fully connected layer, the text sequence features are input into the fully connected layer, and the classification probability distribution of the identified elements is output through the Softmax activation function, and the feature prediction result is obtained based on the classification probability distribution.

[0101] By using a pre-trained feature extraction model to perform feature identification and extraction, we can provide efficient and accurate feature recognition capabilities, improve the efficiency and accuracy of feature extraction, and reduce labor costs.

[0102] Step S205, calculate the loss value between the feature prediction result and the correct feature recognition result according to the preset loss function, adjust the model parameters of the feature extraction model based on the loss value, continue iterative training until the model converges, and output the extraction model to be verified.

[0103] In this embodiment, a loss function based on cross entropy is designed for the feature extraction task. Where n is the number of business sample data in the training set; y i is the true element label of the i-th business sample data, that is, the correct element recognition result; p(y i |x) is the conditional probability of the factor prediction result predicted by the model.

[0104] Specifically, the cross entropy loss function is used to calculate the loss value; according to the loss value, the gradient of the loss function to the model parameters is propagated back through the back propagation algorithm to update the network parameters of the spatial expansion layer, BiLSTM decoding network and output layer; it is determined whether the current iteration meets the convergence conditions. If the convergence conditions are met, the extraction model to be verified is obtained; if the convergence conditions are not met, iterative training is continued until the model converges to obtain the final extraction model to be verified. Among them, the convergence condition is met when the preset maximum number of iterations is reached, or the current loss value does not decrease compared to the loss value of the previous round.

[0105] When the convergence conditions are not met, the new network parameter values ​​are used to predict the training set through the forward propagation algorithm to obtain the prediction value of the new round of iteration; the cross entropy loss function is calculated using the new prediction value and the true feature label to obtain the loss value of this round of iteration; the loss value of this round is compared with the loss value of the previous round. If the loss function value decreases, it is considered that the model performance has been improved and the iterative optimization continues; if the loss value does not decrease for multiple consecutive rounds, the early stopping mechanism is triggered to stop the iteration, and the network parameters of the space expansion layer, BiLSTM decoding network and output layer when the iteration is stopped are obtained as the final training results, and the extraction model to be verified is output.

[0106] During the model training phase, the parameters of the main network of the BERT encoding network are fixed, and the focus is on training the prompt vector pool of the spatial extension layer, the BiLSTM decoding network, and the network parameters of the output layer. Fine-tuning during the model training process is achieved, making the model training process more efficient and faster, which can effectively shorten the training time and improve development efficiency. At the same time, it can significantly reduce the number of parameters required for the model, which will reduce the computational burden and memory usage of the model, thereby improving the running speed and efficiency of the model.

[0107] Step S206, testing the extraction model to be verified through the test set to obtain the test result, and when the test result meets the preset threshold condition, outputting the extraction model to be verified as the final business element extraction model.

[0108] When testing the model, the test set is input into the extraction model to be verified to obtain the model prediction feature extraction result; the evaluation index is calculated based on the model prediction feature extraction result and the corresponding real feature label to obtain the test result. When the test result is greater than or equal to the preset threshold condition, it means that the model meets expectations. Among them, the evaluation indicators include accuracy, precision, recall rate, F1 value, etc. The actual performance of the model can be comprehensively judged by the evaluation indicators, which can improve the accuracy and reliability of the model's feature extraction, thereby improving the effect and value of the model in practical applications and providing users with a better service experience.

[0109] Step S207, obtaining the business text to be identified, inputting the business text to be identified into the business element extraction model, and obtaining the business element extraction result.

[0110] The trained business element extraction model is used to perform element recognition and extraction on the business text to be identified, and the business element extraction results that can reflect the key information of the business are obtained.

[0111] In this embodiment, the business element extraction model is regularly adjusted and optimized to adapt to complex and changing financial business scenarios.

[0112] The present application constructs an initial rule extraction model according to business extraction requirements and predefined rules, trains the initial rule extraction model, and uses the trained rule extraction model to extract elements from a business sample data set, thereby improving the element recognition efficiency and accuracy of the data set, thereby improving the quality of the data set; annotates the business text data set with the correct element recognition results, constructs a fine-tuning data set, saves time and cost for data annotation, and improves the accuracy of the data set; trains the pre-trained element extraction model with the fine-tuning data set to obtain the final business element extraction model, and uses the business element extraction model to perform element recognition and extraction on the business text, thereby realizing automated element extraction and improving the accuracy and efficiency of element extraction.

[0113] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0114] The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium may be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0115] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0116] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a factor extraction device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0117] like Figure 3As shown, the element extraction device 300 described in this embodiment includes: a rule construction module 301, a rule extraction module 302, a labeling module 303, a prediction module 304, an iteration module 305, a testing module 306 and an element extraction module 307. Among them:

[0118] The rule construction module 301 is used to construct an initial rule extraction model according to business extraction requirements and predefined rules, and train the initial rule extraction model to obtain a trained rule extraction model;

[0119] The rule extraction module 302 is used to obtain a business sample data set, input the business sample data set into the rule extraction model, and obtain a correct element recognition result;

[0120] The annotation module 303 is used to annotate the business sample data set based on the correct element recognition result, construct a fine-tuning data set, and divide the fine-tuning data set into a training set and a test set;

[0121] The prediction module 304 is used to input the training set into the pre-trained factor extraction model to obtain the factor prediction result;

[0122] The iteration module 305 is used to calculate the loss value between the element prediction result and the correct element recognition result according to a preset loss function, adjust the model parameters of the element extraction model based on the loss value, continue iterative training until the model converges, and output the extraction model to be verified;

[0123] The testing module 306 is used to test the extraction model to be verified through the test set to obtain a test result, and when the test result meets a preset threshold condition, output the extraction model to be verified as the final business element extraction model;

[0124] The element extraction module 307 is used to obtain the business text to be identified, input the business text to be identified into the business element extraction model, and obtain the business element extraction result.

[0125] It should be emphasized that in order to further ensure the privacy and security of the business sample data set, the above-mentioned business sample data set can also be stored in a node of a blockchain.

[0126] Based on the above-mentioned feature extraction device 300, by constructing an initial rule extraction model, training the initial rule extraction model, and using the trained rule extraction model to extract features from a business sample data set, the feature recognition efficiency and accuracy of the data set can be improved, thereby improving the quality of the data set; by annotating the business text data set with the correct feature recognition results, a fine-tuning data set is constructed, saving time and cost for data annotation and improving the accuracy of the data set; the pre-trained feature extraction model is trained with the fine-tuning data set to obtain the final business feature extraction model, and the business text is subjected to feature recognition and extraction through the business feature extraction model to achieve automated feature extraction and improve the accuracy and efficiency of feature extraction.

[0127] In some optional implementations of this embodiment, the rule construction module 301 includes:

[0128] A determination submodule, used to determine a plurality of extraction fields according to business extraction requirements, and determine an extraction element set based on the plurality of extraction fields;

[0129] A dictionary construction submodule, used to construct an initial dictionary based on the extracted element set, and to expand the initial dictionary using a preset semantic co-occurrence network to obtain a keyword dictionary corresponding to the business extraction requirement;

[0130] The model building submodule is used to build the extraction rules of the extraction element set and form an initial rule extraction model based on the extraction rules and the keyword dictionary.

[0131] By building an initial rule extraction model, the process of factor extraction can be standardized, facilitating subsequent data processing and analysis, saving a lot of time and cost.

[0132] In some optional implementations, the rule construction module 301 further includes:

[0133] A preprocessing submodule is used to obtain a business data set, preprocess the business data set, and obtain preprocessed text data;

[0134] A matching submodule, used for inputting the pre-processed text data into the initial rule extraction model, calling a keyword dictionary to match the pre-processed text data, and extracting preliminary element results;

[0135] An evaluation submodule, used to evaluate the preliminary factor results to obtain evaluation results;

[0136] The optimization submodule is used to iteratively optimize the initial rule extraction model if the evaluation result does not meet expectations, until the evaluation result meets expectations, thereby obtaining a trained rule extraction model.

[0137] By training the rule extraction model, the efficiency and accuracy of feature identification can be improved, thereby improving data quality and accuracy, and enhancing data availability and scalability.

[0138] In this embodiment, the matching submodule includes:

[0139] A positioning unit, configured to traverse the keyword dictionary according to the preprocessed text data to determine the element keyword position;

[0140] The extraction unit is used to extract based on the element keyword position through the extraction rules of the initial rule extraction model to obtain preliminary element results.

[0141] Through keyword matching, texts containing specific keywords can be screened out, narrowing the scope of information extraction. Combined with extraction rules for precise matching and extraction of key information, efficient and accurate information extraction can be achieved.

[0142] In some optional implementations, the prediction module 304 includes:

[0143] An input submodule, used to input the training set into a pre-trained feature extraction model, wherein the feature extraction model includes a BERT encoding network, a BiLSTM decoding network and an output layer;

[0144] An encoding submodule, used to encode the business sample data in the training set through the BERT encoding network to obtain text semantic features;

[0145] A prediction submodule, used for capturing context information of the text semantic features through the BiLSTM decoding network to obtain text sequence features;

[0146] The decoding submodule is used to input the text sequence features into the output layer for decoding to obtain feature prediction results.

[0147] By using a pre-trained feature extraction model to perform feature identification and extraction, we can provide efficient and accurate feature recognition capabilities, improve the efficiency and accuracy of feature extraction, and reduce labor costs.

[0148] In some optional implementations of this embodiment, the BERT encoding network includes an embedding layer, a spatial expansion layer, and an encoder layer, and the encoding submodule includes:

[0149] An embedding unit, used for performing vector conversion on the business sample data in the training set through the embedding layer to obtain a text embedding vector;

[0150] An expansion unit, used for inputting the text embedding vector into the spatial expansion layer, concatenating the text embedding vector and the introduced prompt vector to obtain a text expansion feature vector;

[0151] The attention mechanism unit is used to perform multi-head attention calculation on the text extension feature vector through the encoder layer to obtain text semantic features.

[0152] By encoding business sample data through the BERT encoding network, the model can capture more features related to element information, and use the multi-head attention mechanism to weightedly process the text extension feature vector, so that the model pays more attention to the key information of elements in the text, making the model more adaptable to complex text types, thereby improving the accuracy and robustness of element recognition.

[0153] In some optional implementations of this embodiment, the iteration module 305 is further configured to:

[0154] Based on the loss value, a back propagation algorithm is used to return the gradient of the loss function to the model parameters to update the network parameters of the spatial expansion layer, the BiLSTM decoding network and the output layer.

[0155] Through fine-tuning during the model training process, the model training process can be made more efficient and faster, which can effectively shorten the training time and improve development efficiency. At the same time, it can significantly reduce the number of parameters required for the model, which will reduce the computational burden and memory usage of the model, thereby improving the running speed and efficiency of the model.

[0156] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0157] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0158] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0159] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the element extraction method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0160] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the feature extraction method.

[0161] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0162] By building an initial rule extraction model according to business extraction requirements and predefined rules, training the initial rule extraction model, and using the trained rule extraction model to extract elements from the business sample data set, the efficiency and accuracy of element recognition in the data set can be improved, thereby improving the quality of the data set; by annotating the business text data set with the correct element recognition results, a fine-tuning data set is constructed to save time and cost for data annotation and improve the accuracy of the data set; the pre-trained element extraction model is trained with the fine-tuning data set to obtain the final business element extraction model, and the business element extraction model is used to perform element recognition and extraction on the business text to achieve automated element extraction and improve the accuracy and efficiency of element extraction.

[0163] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the element extraction method as described above.

[0164] By building an initial rule extraction model according to business extraction requirements and predefined rules, training the initial rule extraction model, and using the trained rule extraction model to extract elements from the business sample data set, the efficiency and accuracy of element recognition in the data set can be improved, thereby improving the quality of the data set; by annotating the business text data set with the correct element recognition results, a fine-tuning data set is constructed to save time and cost for data annotation and improve the accuracy of the data set; the pre-trained element extraction model is trained with the fine-tuning data set to obtain the final business element extraction model, and the business element extraction model is used to perform element recognition and extraction on the business text to achieve automated element extraction and improve the accuracy and efficiency of element extraction.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0166] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A factor extraction method, characterized in that: The steps include: Constructing an initial rule extraction model according to business extraction requirements and predefined rules, and training the initial rule extraction model to obtain a trained rule extraction model; Obtaining a business sample data set, inputting the business sample data set into the rule extraction model, and obtaining a correct element recognition result; Annotating the business sample data set based on the correct element recognition result, constructing a fine-tuning data set, and dividing the fine-tuning data set into a training set and a test set; Inputting the training set into a pre-trained factor extraction model to obtain factor prediction results; Calculating the loss value between the element prediction result and the correct element recognition result according to a preset loss function, adjusting the model parameters of the element extraction model based on the loss value, continuing iterative training until the model converges, and outputting the extraction model to be verified; Testing the extraction model to be verified by using the test set to obtain a test result, and when the test result meets a preset threshold condition, outputting the extraction model to be verified as a final business element extraction model; The business text to be identified is obtained, and the business text to be identified is input into the business element extraction model to obtain a business element extraction result.

2. The factor extraction method according to claim 1, characterized in that: The step of constructing an initial rule extraction model according to business extraction requirements and predefined rules includes: Determine multiple extraction fields according to business extraction requirements, and determine an extraction element set based on the multiple extraction fields; Constructing an initial dictionary based on the extracted element set, and expanding the initial dictionary using a preset semantic co-occurrence network to obtain a keyword dictionary corresponding to the business extraction requirement; An extraction rule for the extraction element set is constructed, and an initial rule extraction model is formed based on the extraction rule and the keyword dictionary.

3. The factor extraction method according to claim 1, characterized in that: The step of training the initial rule extraction model to obtain a trained rule extraction model includes: Acquire a business data set, and preprocess the business data set to obtain preprocessed text data; Inputting the preprocessed text data into the initial rule extraction model, calling a keyword dictionary to match the preprocessed text data, and extracting preliminary element results; Evaluating the preliminary factor results to obtain evaluation results; If the evaluation result does not meet expectations, the initial rule extraction model is iteratively optimized until the evaluation result meets expectations, thereby obtaining a trained rule extraction model.

4. The factor extraction method according to claim 3, characterized in that: The step of inputting the preprocessed text data into the initial rule extraction model, calling a keyword dictionary to match the preprocessed text data, and extracting preliminary element results comprises: Traversing the keyword dictionary according to the preprocessed text data to determine the element keyword position; Based on the element keyword position, extraction is performed using the extraction rules of the initial rule extraction model to obtain preliminary element results.

5. The factor extraction method according to claim 1, characterized in that: The step of inputting the training set into the pre-trained factor extraction model to obtain the factor prediction result comprises: Inputting the training set into a pre-trained feature extraction model, wherein the feature extraction model includes a BERT encoding network, a BiLSTM decoding network, and an output layer; Encoding the business sample data in the training set through the BERT encoding network to obtain text semantic features; Capturing context information of the text semantic features through the BiLSTM decoding network to obtain text sequence features; The text sequence features are input into the output layer for decoding to obtain feature prediction results.

6. The factor extraction method according to claim 5, characterized in that: The BERT coding network includes an embedding layer, a spatial expansion layer and an encoder layer. The step of encoding the business sample data in the training set through the BERT coding network to obtain text semantic features includes: Performing vector conversion on the business sample data in the training set through the embedding layer to obtain a text embedding vector; Inputting the text embedding vector into the spatial extension layer, concatenating the text embedding vector and the introduced prompt vector to obtain a text extension feature vector; The encoder layer performs multi-head attention calculation on the text extension feature vector to obtain text semantic features.

7. The factor extraction method according to claim 6, characterized in that: The step of adjusting the model parameters of the factor extraction model based on the loss value comprises: Based on the loss value, a back propagation algorithm is used to return the gradient of the loss function to the model parameters to update the network parameters of the spatial expansion layer, the BiLSTM decoding network and the output layer.

8. A factor extraction device, characterized in that: include: A rule construction module is used to construct an initial rule extraction model according to business extraction requirements and predefined rules, and train the initial rule extraction model to obtain a trained rule extraction model; A rule extraction module is used to obtain a business sample data set, input the business sample data set into the rule extraction model, and obtain a correct element recognition result; A labeling module, used to label the business sample data set based on the correct element recognition result, construct a fine-tuning data set, and divide the fine-tuning data set into a training set and a test set; A prediction module, used for inputting the training set into a pre-trained factor extraction model to obtain a factor prediction result; An iterative module, used to calculate the loss value between the element prediction result and the correct element recognition result according to a preset loss function, adjust the model parameters of the element extraction model based on the loss value, continue iterative training until the model converges, and output the extraction model to be verified; A testing module, used to test the extraction model to be verified through the test set to obtain a test result, and when the test result meets a preset threshold condition, output the extraction model to be verified as a final business element extraction model; The element extraction module is used to obtain the business text to be identified, input the business text to be identified into the business element extraction model, and obtain the business element extraction result.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the element extraction method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the element extraction method according to any one of claims 1 to 7 are implemented.

Images