Financial text data processing method and device, equipment, medium and product

Abstract

The invention provides a financial text data processing method and device, equipment, a medium and a product, which can be applied to the fields of artificial intelligence and financial science and technology. The method comprises the steps of obtaining financial text data to be processed in response to a data processing instruction; word segmentation processing is carried out on the financial text data to be processed to obtain a keyword set, and the keyword set comprises a plurality of initial financial keywords; inputting the plurality of initial financial keywords into a target marker, and outputting a plurality of target keywords and a part-of-speech classification tag corresponding to each target keyword; wherein when the initial marker is trained, under the condition of forward transmission, a keyword intermediate vector generated by the initial marker is deleted, and under the condition of reverse transmission, the keyword intermediate vector is recalculated, so that the initial marker generates a target marker based on the recalculated keyword intermediate vector.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and financial technology, and specifically to a financial text data processing method, a financial text data processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Part-Of-Speech Tagging (POS) determines the part of speech of each word in a given corpus text sequence and tags it. It is the cornerstone of in-depth natural language processing. It provides a basis for high-level tasks such as machine translation, speech recognition, and information retrieval. Therefore, the research on the accuracy of part-of-speech tagging has attracted more and more attention.

[0003] However, when the related technology uses the part-of-speech tagging model to process data in the financial industry, it occupies a lot of memory and the accuracy of the part-of-speech tagging is poor. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a financial text data processing method, a financial text data processing device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to a first aspect of the present disclosure, a financial text data processing method is provided, comprising:

[0006] In response to the data processing instruction, obtaining financial text data to be processed;

[0007] Performing word segmentation processing on the financial text data to be processed to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords;

[0008] Inputting the plurality of initial financial keywords into a target tagger, and outputting a plurality of target keywords and a part-of-speech classification label corresponding to each of the target keywords;

[0009] Among them, when training the initial marker, in the case of forward transmission, the keyword intermediate vector generated by the above-mentioned initial marker is deleted, and in the case of reverse transmission, the above-mentioned keyword intermediate vector is recalculated so that the above-mentioned initial marker generates the above-mentioned target marker based on the recalculated keyword intermediate vector.

[0010] According to an embodiment of the present disclosure, before inputting the plurality of initial financial keywords into the target marker, the method further includes:

[0011] Processing the plurality of the initial financial keywords using a target attention network to obtain an attention weight corresponding to each of the initial financial keywords;

[0012] Based on the attention weights, the multiple initial financial keywords are sorted to obtain a sorted keyword set, so as to input the sorted keyword set into the target marker, wherein the sorted keyword set includes multiple sorted initial financial keywords.

[0013] According to an embodiment of the present disclosure, before the sorting process is performed, the following step is further included:

[0014] For each of the above-mentioned initial financial keywords, when the attention weight of the above-mentioned initial financial keywords is less than the preset weight threshold, the above-mentioned initial financial keywords are deleted to obtain a filtered keyword set, and the above-mentioned filtered keyword set is sorted, wherein the above-mentioned filtered keyword set includes multiple filtered initial financial keywords.

[0015] According to an embodiment of the present disclosure, a plurality of the above initial financial keywords are processed using a target attention network to obtain an attention weight corresponding to each of the above initial financial keywords, including:

[0016] For each of the initial financial keywords, feature mapping is performed on the initial financial keywords to obtain a query vector and a key vector of the initial financial keyword;

[0017] Generate a similarity score according to the query vector and the key vector, wherein the similarity score represents the degree of association between the initial financial keyword and other related keywords;

[0018] The similarity scores are normalized to obtain the attention weights of the initial financial keywords, wherein the attention weights represent the importance of the initial financial keywords to the other related keywords.

[0019] According to an embodiment of the present disclosure, the above-mentioned financial text data to be processed is segmented to obtain a keyword set, including:

[0020] Based on the data capacity of the financial text data to be processed, the financial text data to be processed is divided into a plurality of text data blocks;

[0021] The plurality of text data blocks are respectively segmented using a plurality of word segmenters in parallel to obtain a plurality of the initial financial keywords corresponding to each of the text data blocks.

[0022] According to an embodiment of the present disclosure, the training method of the target marker includes:

[0023] Obtain a financial text training set, where the financial text training set includes multiple training data blocks generated by at least one financial training text, the training data blocks include multiple financial training keywords, and the financial text training set further includes a part-of-speech tag corresponding to each of the financial training keywords;

[0024] For each of the training data blocks, input each of the financial training keywords into an initial tagger to obtain a predicted classification tag corresponding to the financial training keyword;

[0025] For each of the financial training keywords, generate a loss result according to the predicted classification tag and the part-of-speech tag corresponding to the predicted classification tag;

[0026] Iteratively adjust the network parameters of the initial tagger according to the loss result to obtain an intermediate tagger;

[0027] Perform a merging process on multiple intermediate taggers based on weight coefficients to obtain the trained target tagger.

[0028] According to an embodiment of the present disclosure, the initial tagger includes i tagging neurons.

[0029] According to an embodiment of the present disclosure, inputting each of the financial training keywords into the initial tagger to obtain a predicted classification tag corresponding to the financial training keyword includes:

[0030] Based on a context recognition method, use the m-th tagging neuron to process multiple financial training keywords to obtain an m-th keyword intermediate vector corresponding to each of the financial training keywords, where m < i;

[0031] In the process of forward propagation, use the (m + n)-th tagging neuron to process the keyword intermediate vector output by the previous tagging neuron to obtain an output result. When m + n < i, the output result represents the (m + n)-th keyword intermediate vector. When m + n = i, the output result represents the predicted classification tag. Any of the keyword intermediate vectors are deleted from the memory after being used by the next tagging neuron;

[0032] In the process of backpropagation, recalculate the deleted keyword intermediate vectors so that the p-th tagging neuron processes the keyword intermediate vector output by the previous tagging neuron based on the keyword intermediate vectors.

[0033] According to an embodiment of the present disclosure, the training method of the target tagger further includes:

[0034] Change the learning rate in training the target tagger to train the target tagger based on the changed learning rate to obtain a new target tagger; and / or

[0035] The financial text training set is randomly combined to obtain a new financial text training set, and the target marker is trained using the new financial text training set to obtain a new target marker.

[0036] A second aspect of the present disclosure provides a financial text data processing device, comprising:

[0037] An acquisition module, used for acquiring the financial text data to be processed in response to the data processing instruction;

[0038] A word segmentation module is used to perform word segmentation processing on the financial text data to be processed to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords;

[0039] A classification prediction module, used for inputting the plurality of initial financial keywords into a target tagger, and outputting a plurality of target keywords and a part-of-speech classification label corresponding to each of the target keywords;

[0040] Among them, when training the initial marker, in the case of forward transmission, the keyword intermediate vector generated by the above-mentioned initial marker is deleted, and in the case of reverse transmission, the above-mentioned keyword intermediate vector is recalculated so that the above-mentioned initial marker generates the above-mentioned target marker based on the recalculated keyword intermediate vector.

[0041] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0042] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0043] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0044] According to an embodiment of the present disclosure, during the patrol process of the target marker, the generated keyword intermediate vector is deleted from the memory during the forward transfer, and the keyword intermediate vector is recalculated during the backward transfer for use in the training of the target marker, thereby reducing the memory occupancy during the training process of the target marker and improving the training efficiency of the target marker. The target marker obtained in this way can also reduce the memory occupancy and improve the part-of-speech prediction efficiency during actual data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0046] Figure 1 The application scenario diagram of the financial text data processing method according to the embodiment of the present disclosure is schematically shown;

[0047] Figure 2 A flowchart of a method for processing financial text data according to an embodiment of the present disclosure is schematically shown;

[0048] Figure 3 A flowchart of a method for training a target marker according to an embodiment of the present disclosure is schematically shown;

[0049] Figure 4 The structure block diagram of the financial text data processing device according to the embodiment of the present disclosure is schematically shown;

[0050] Figure 5 A structural block diagram schematically shows a training device for a target marker according to an embodiment of the present disclosure; and

[0051] Figure 6 A block diagram of an electronic device suitable for implementing the above method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0053] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0054] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0055] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0056] Embodiments of the present disclosure provide a financial text data processing method, apparatus, device, medium, and product, the method comprising: obtaining to-be-processed financial text data in response to a data processing instruction; performing word segmentation processing on the to-be-processed financial text data to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords; inputting the plurality of initial financial keywords into a target tagger, and outputting a plurality of target keywords and a part-of-speech classification label corresponding to each target keyword; wherein, when training the initial tagger, in the case of forward transfer, a keyword intermediate vector generated by the initial tagger is deleted, and in the case of reverse transfer, the keyword intermediate vector is recalculated so that the initial tagger generates a target tagger based on the recalculated keyword intermediate vector.

[0057] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0058] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0059] It should be noted that the financial text data processing method and the financial text data processing device provided by the present disclosure can be used in the field of financial technology, such as banks and other financial institutions, and can also be used in any field other than the field of financial technology, such as hospitals and other health and safety fields. Therefore, the application field of the financial text data processing method and the financial text data processing device provided by the present disclosure is not limited.

[0060] Figure 1 The application scenario diagram of the financial text data processing method according to the embodiment of the present disclosure is schematically shown.

[0061] like Figure 1 As shown, the application scenario 100 according to this embodiment may include bank text data amount processing. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0062] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0063] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0064] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0065] It should be noted that the financial text data processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the financial text data processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The financial text data processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the financial text data processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0066] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0067] Figure 2 The flowchart of the financial text data processing method according to the embodiment of the present disclosure is schematically shown.

[0068] like Figure 2 As shown, the financial text data processing method of this embodiment includes operations S210 to S230, and the transaction processing method can be executed by an electronic device.

[0069] In operation S210, in response to the data processing instruction, financial text data to be processed is obtained;

[0070] In operation S220, the financial text data to be processed is segmented to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords;

[0071] In operation S230, a plurality of initial financial keywords are input into a target tagger, and a plurality of target keywords and a part-of-speech classification label corresponding to each target keyword are output;

[0072] Among them, when training the initial marker, in the case of forward transfer, the keyword intermediate vector generated by the initial marker is deleted, and in the case of reverse transfer, the keyword intermediate vector is recalculated so that the initial marker generates a target marker based on the recalculated keyword intermediate vector.

[0073] According to an embodiment of the present disclosure, the electronic device may be a mobile phone, a computer or other device. After a user inputs a corresponding operation on the electronic device, the electronic device may automatically respond to the operation and generate a corresponding data processing instruction.

[0074] According to an embodiment of the present disclosure, the financial text data to be processed may be text data related to finance obtained from news or social media, or may be text data obtained by performing text recognition on non-text data related to finance, such as financial text data to be processed by processing image type data using OCR recognition technology.

[0075] According to an embodiment of the present disclosure, part-of-speech classification tags may include names of people, places, organizations, payment amounts, payment times, financial policies, or other entity types. The target tagger may be a part-of-speech tagger built based on Neuro-Linguistic Programming (NLP) such as an LLM large language model, such as a Unigram tagger.

[0076] According to an embodiment of the present disclosure, after obtaining the financial text data to be processed, the financial text data to be processed is first segmented to obtain a plurality of initial financial keywords, and the plurality of initial financial keywords are input into a trained target marker to obtain a plurality of target keywords and a part-of-speech classification label corresponding to each target keyword.

[0077] According to an embodiment of the present disclosure, during the patrol process of the target marker, the generated keyword intermediate vector is deleted from the memory during the forward transfer, and the keyword intermediate vector is recalculated during the backward transfer for use in the training of the target marker, thereby reducing the memory occupancy during the training process of the target marker and improving the training efficiency of the target marker. The target marker obtained in this way can also reduce the memory occupancy and improve the part-of-speech prediction efficiency during actual data processing.

[0078] According to an embodiment of the present disclosure, before inputting a plurality of initial financial keywords into the target tagger, the method further includes:

[0079] Using the target attention network to process multiple initial financial keywords, obtaining the attention weight corresponding to each initial financial keyword;

[0080] Based on the attention weights, multiple initial financial keywords are sorted to obtain a sorted keyword set, so as to input the sorted keyword set into a target marker, wherein the sorted keyword set includes multiple sorted initial financial keywords.

[0081] According to an embodiment of the present disclosure, the attention weight of each initial financial keyword is calculated through a target attention network, and multiple initial financial keywords are sorted based on the size of the attention weight, thereby obtaining multiple sorted initial financial keywords with attention weights from large to small.

[0082] According to an embodiment of the present disclosure, by calculating the attention weight of each initial financial keyword, a target tagger can be provided with data input with importance differences, thereby enabling the target tagger to perform targeted part-of-speech classification predictions on the initial financial keywords based on the importance differences, thereby further improving the prediction accuracy of the part-of-speech classification of the target tagger.

[0083] According to an embodiment of the present disclosure, before the sorting process is performed, the following step is further included:

[0084] For each initial financial keyword, when the attention weight of the initial financial keyword is less than a preset weight threshold, the initial financial keyword is deleted to obtain a filtered keyword set, and the filtered keyword set is sorted, wherein the filtered keyword set includes multiple filtered initial financial keywords.

[0085] According to an embodiment of the present disclosure, the preset weight threshold may be specifically set according to requirements, for example, it may be 0.01.

[0086] According to an embodiment of the present disclosure, in order to further improve the prediction efficiency of the target marker, the initial financial keywords whose attention weights are less than a preset weight threshold can be deleted. The initial financial keywords are identified as invalid or less meaningful keywords by the target attention network. Therefore, deleting them can reduce the memory resources occupied by the target marker during prediction. At the same time, the initial financial keywords may not have much significance for the prediction of other keyword parts of speech. After deletion, the target marker can better understand the connection between other initial financial keywords, thereby further improving the prediction accuracy of the target marker.

[0087] According to an embodiment of the present disclosure, a plurality of initial financial keywords are processed using a target attention network to obtain an attention weight corresponding to each initial financial keyword, including:

[0088] For each initial financial keyword, feature mapping is performed on the initial financial keyword to obtain a query vector and a key vector of the initial financial keyword;

[0089] Generate a similarity score according to the query vector and the key vector, wherein the similarity score represents the degree of association between the initial financial keyword and other related keywords;

[0090] The similarity scores are normalized to obtain the attention weights of the initial financial keywords, where the attention weights represent the importance of the initial financial keywords to other related keywords.

[0091] According to an embodiment of the present disclosure, first, the initial financial keywords are mapped to two vector representations, namely, query vector Q and key vector K. This step is usually implemented by linear transformation, as shown in formula (1):

[0092] (1)

[0093] Among them, X is the initial financial keyword, , is a trainable weight matrix.

[0094] According to an embodiment of the present disclosure, the similarity score S between the query vector Q and the key vector K is calculated, usually using a dot product, as shown in formula (2). These similarity scores S represent the degree of association between the initial financial keywords and other related keywords.

[0095] (2)

[0096] Where T is the time coefficient, is a scaling factor to prevent the dot product value from being too large.

[0097] According to an embodiment of the present disclosure, the similarity score is normalized by a Softmax function to obtain the attention weight A, as shown in formula (3). These weights reflect the importance of each element to other elements.

[0098] (3)

[0099] According to an embodiment of the present disclosure, by calculating a similarity score S based on the query vector Q and the key vector K of each initial financial keyword and normalizing the similarity score S to obtain an attention weight, a target tagger can be provided with data input with importance differences, thereby enabling the target tagger to perform targeted part-of-speech classification predictions on the initial financial keywords based on the importance differences, thereby further improving the prediction accuracy of the part-of-speech classification of the target tagger.

[0100] According to an embodiment of the present disclosure, the financial text data to be processed is segmented to obtain a keyword set, including:

[0101] Based on the data capacity of the financial text data to be processed, dividing the financial text data to be processed into a plurality of text data blocks;

[0102] Multiple word segmenters are used to perform word segmentation processing on multiple text data blocks in parallel to obtain multiple initial financial keywords corresponding to each text data block.

[0103] According to an embodiment of the present disclosure, the financial text data to be processed is divided into a plurality of text data blocks based on the data capacity of the financial text data to be processed. For example, the financial text data to be processed can be divided into 256 text data blocks with substantially the same data capacity.

[0104] According to an embodiment of the present disclosure, a text data block is segmented using one word segmenter, and multiple word segmenters perform word segmentation in parallel, wherein the word segmenter may be a unary word segmenter.

[0105] According to an embodiment of the present disclosure, by dividing the financial text data to be processed into multiple text data blocks and using multiple word segmenters to process the multiple text data blocks in parallel, the efficiency of part-of-speech tagging can be improved.

[0106] Figure 3 The flowchart of the training method of the target marker according to the embodiment of the present disclosure is schematically shown.

[0107] According to the embodiments of the present disclosure, Figure 3 As shown, the training method of the target marker includes operations S310 to S350:

[0108] In operation S310, a financial text training set is obtained, wherein the financial text training set includes a plurality of training data blocks generated from at least one financial training text, the training data blocks include a plurality of financial training keywords, and the financial text training set also includes a part-of-speech tag corresponding to each financial training keyword;

[0109] In operation S320, for each training data block, each financial training keyword is input into an initial tagger to obtain a predicted classification label corresponding to the financial training keyword;

[0110] In operation S330, for each financial training keyword, a loss result is generated according to the predicted classification label and the part-of-speech label corresponding to the predicted classification label;

[0111] In operation S340, iteratively adjusting network parameters of the initial labeler according to the loss result to obtain an intermediate labeler;

[0112] In operation S350 , a plurality of intermediate markers are merged based on the weight coefficients to obtain a trained target marker.

[0113] According to an embodiment of the present disclosure, the part-of-speech tag may be a person's name, a place name, an organization, a payment amount, a payment time, a financial policy, or other entity tags.

[0114] According to an embodiment of the present disclosure, the loss result can be calculated by a binary classification loss function such as binary cross-entropy loss or Huber loss. The weight coefficient can be set based on the capacity size of the training data block, for example, set by the byte size of the training data block.

[0115] According to an embodiment of the present disclosure, for each training data block, an initial tagger is trained using the training data block. Based on the predicted classification labels and part-of-speech labels corresponding to each financial training keyword output by the initial tagger, its loss result is calculated, and thus the network parameters of the initial tagger are iteratively adjusted based on the loss result to obtain an intermediate tagger, where the training of multiple initial taggers can be performed in parallel.

[0116] According to an embodiment of the present disclosure, corresponding weight coefficients are set for each initial tagger based on the data sizes of different training data blocks, and thus multiple intermediate taggers are combined based on the weight coefficients of each initial tagger to obtain a target tagger.

[0117] According to an embodiment of the present disclosure, the initial tagger includes i tagging neurons.

[0118] According to an embodiment of the present disclosure, inputting each financial training keyword into the initial tagger to obtain a predicted classification label corresponding to the financial training keyword includes:

[0119] Based on a context recognition method, the m-th tagging neuron is used to process multiple financial training keywords to obtain an m-th keyword intermediate vector corresponding to each financial training keyword, where m < i;

[0120] In the process of forward propagation, the (m + n)-th tagging neuron is used to process the keyword intermediate vector output by the previous tagging neuron to obtain an output result. When m + n < i, the output result represents the (m + n)-th keyword intermediate vector. When m + n = i, the output result represents the predicted classification label. Any keyword intermediate vector is deleted from the memory after being used by the next tagging neuron;

[0121] In the process of backward propagation, the deleted keyword intermediate vector is recalculated so that the p-th tagging neuron processes the keyword intermediate vector output by the previous tagging neuron based on the keyword intermediate vector.

[0122] According to an embodiment of the present disclosure, the context identification method may be a context method. In the field of natural language processing, the context method is a commonly used technique for understanding and processing contextual information of a text. The context method can more accurately understand and interpret the meaning of a text by considering the contextual relationships in the text. The context method typically converts words into vector representations based on word embeddings technology. Word embedding is a technique for mapping words to a real vector space, and measures the semantic similarity between words by the distance in the vector space. By using word embedding technology, the context method can convert each word in the text into a vector representation, thereby being able to use the relationship between vectors to understand the context of the text.

[0123] According to an embodiment of the present disclosure, during the backward pass of the labeled neurons, when a labeled neuron enables activation checkpointing, only the input and output of the layer are saved in memory after the forward pass, and any intermediate vectors (i.e., keyword intermediate vectors) are discarded from the memory. During the backward pass, these intermediate vectors can be recalculated. The embodiment of the present disclosure applies activation checkpointing to each neural layer in the initial labeler, thereby minimizing the memory consumption of the training process.

[0124] According to an embodiment of the present disclosure, the training method of the target marker further includes:

[0125] Changing a learning rate in training a target marker to train the target marker based on the changed learning rate to obtain a new target marker; and / or

[0126] The financial text training set is randomly combined to obtain a new financial text training set, so as to train a target marker using the new financial text training set to obtain a new target marker.

[0127] According to an embodiment of the present disclosure, when the initial marker is trained to a certain stage (for example, the model parameters of the initial marker no longer change significantly), the learning rate in the training target marker is changed to train the target marker based on the changed learning rate to obtain a new target marker. Alternatively, the financial text training set is disrupted to perform secondary training on the target marker using the new financial text training set to obtain a new target marker.

[0128] In a specific embodiment, the learning rate may be reduced, for example, setting the learning rate to 2e-5 may obtain better training effect.

[0129] According to an embodiment of the present disclosure, after obtaining the target marker, the prediction accuracy of the target marker can be further improved by changing the learning rate or performing random combination processing on the financial text training set to perform secondary training on the target marker.

[0130] According to an embodiment of the present disclosure, after the target marker is trained, a set of test sets may be given to perform likelihood-based classification to test the prediction accuracy of the target marker. Among them, likelihood-based classification can be divided into three methods: conventional, calibrated, and normalized, as shown in the three formulas in formula (4):

[0131] (4)

[0132] in is the test task, s is the test text in the test set, and len represents the number of tags.

[0133] Figure 4 The structural block diagram of the financial text data processing device according to the embodiment of the present disclosure is schematically shown.

[0134] like Figure 4 As shown, the financial text data processing device 400 of this embodiment includes an acquisition module 410 , a word segmentation module 420 , and a classification prediction module 430 .

[0135] An acquisition module 410, configured to acquire the financial text data to be processed in response to the data processing instruction;

[0136] A word segmentation module 420 is used to perform word segmentation processing on the financial text data to be processed to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords;

[0137] The classification prediction module 430 is used to input a plurality of initial financial keywords into a target tagger, and output a plurality of target keywords and a part-of-speech classification label corresponding to each target keyword;

[0138] Among them, when training the initial marker, in the case of forward transfer, the keyword intermediate vector generated by the initial marker is deleted, and in the case of reverse transfer, the keyword intermediate vector is recalculated so that the initial marker generates a target marker based on the recalculated keyword intermediate vector.

[0139] According to an embodiment of the present disclosure, during the patrol process of the target marker, the generated keyword intermediate vector is deleted from the memory during the forward transfer, and the keyword intermediate vector is recalculated during the backward transfer for use in the training of the target marker, thereby reducing the memory occupancy during the training process of the target marker and improving the training efficiency of the target marker. The target marker obtained in this way can also reduce the memory occupancy and improve the part-of-speech prediction efficiency during actual data processing.

[0140] According to an embodiment of the present disclosure, the financial text data processing device 400 further includes:

[0141] An attention module, used for processing a plurality of initial financial keywords using a target attention network to obtain an attention weight corresponding to each initial financial keyword;

[0142] The sorting module is used to sort the multiple initial financial keywords based on the attention weights to obtain a sorted keyword set, so as to input the sorted keyword set into the target marker, wherein the sorted keyword set includes the multiple sorted initial financial keywords.

[0143] According to an embodiment of the present disclosure, the financial text data processing device 400 further includes:

[0144] The data screening module is used to delete each initial financial keyword when the attention weight of the initial financial keyword is less than a preset weight threshold, so as to obtain a screened keyword set, and to sort the screened keyword set, wherein the screened keyword set includes multiple screened initial financial keywords.

[0145] According to an embodiment of the present disclosure, the attention module includes:

[0146] A feature mapping unit, used for performing feature mapping processing on each initial financial keyword to obtain a query vector and a key vector of the initial financial keyword;

[0147] A calculation unit, configured to generate a similarity score according to the query vector and the key vector, wherein the similarity score represents the degree of association between the initial financial keyword and other related keywords;

[0148] The normalization unit is used to normalize the similarity scores to obtain the attention weight of the initial financial keyword, wherein the attention weight represents the importance of the initial financial keyword to other related keywords.

[0149] According to an embodiment of the present disclosure, the word segmentation module 420 includes:

[0150] A division unit, used for dividing the to-be-processed financial text data into a plurality of text data blocks based on the data capacity of the to-be-processed financial text data;

[0151] The word segmentation unit is used to perform word segmentation processing on multiple text data blocks in parallel using multiple word segmenters to obtain multiple initial financial keywords corresponding to each text data block.

[0152] Figure 5 The structural block diagram of the training device of the target marker according to the embodiment of the present disclosure is schematically shown.

[0153] like Figure 5 As shown, the training device 500 of the target marker includes a second acquisition module 510, a part-of-speech prediction module 520, a loss calculation module 530, an adjustment module 540, and a merging module 550.

[0154] A second acquisition module 510 is used to acquire a financial text training set, wherein the financial text training set includes a plurality of training data blocks generated by at least one financial training text, the training data blocks include a plurality of financial training keywords, and the financial text training set also includes a part-of-speech tag corresponding to each financial training keyword;

[0155] A part-of-speech prediction module 520 is used to input each financial training keyword into an initial tagger for each training data block to obtain a predicted classification label corresponding to the financial training keyword;

[0156] A loss calculation module 530, for generating a loss result for each financial training keyword according to the predicted classification label and the part-of-speech label corresponding to the predicted classification label;

[0157] An adjustment module 540, for iteratively adjusting the network parameters of the initial marker according to the loss result to obtain an intermediate marker;

[0158] The merging module 550 is used to merge multiple intermediate markers based on the weight coefficient to obtain a trained target marker.

[0159] According to an embodiment of the present disclosure, the initial marker includes i marker neurons.

[0160] According to an embodiment of the present disclosure, the part-of-speech prediction module 520 includes:

[0161] The unit is used to process multiple financial training keywords using the mth labeled neuron based on the context recognition method to obtain the mth keyword intermediate vector corresponding to each financial training keyword, where m <i;

[0162] The forward propagation unit is used to utilize the (m + n)-th token neuron to process the keyword intermediate vector output by the previous token neuron during forward propagation to obtain an output result. Wherein, when m + n < i, the output result represents the (m + n)-th keyword intermediate vector; when m + n = i, the output result represents the predicted classification label. Any keyword intermediate vector is deleted from the memory after being used by the next token neuron;

[0163] The backward propagation unit is used to recalculate the deleted keyword intermediate vector during backward propagation, so that the p-th token neuron processes the keyword intermediate vector output by the previous token neuron based on the keyword intermediate vector.

[0164] According to an embodiment of the present disclosure, the training device 500 of the target tokenizer further includes:

[0165] A learning rate adjustment module, configured to change the learning rate in the training of the target tokenizer, and train the target tokenizer based on the changed learning rate to obtain a new target tokenizer; and / or

[0166] A random combination module, configured to perform random combination processing on the financial text training set to obtain a new financial text training set, and train the target tokenizer using the new financial text training set to obtain a new target tokenizer.

[0167] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 410, the word segmentation module 420, the classification prediction module 430, or the second acquisition module 510, the part-of-speech prediction module 520, the loss calculation module 530, the adjustment module 540, and the merging module 550 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 410, the word segmentation module 420, the classification prediction module 430, or the second acquisition module 510, the part-of-speech prediction module 520, the loss calculation module 530, the adjustment module 540, and the merging module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 410, the word segmentation module 420, the classification prediction module 430, or the second acquisition module 510, the part-of-speech prediction module 520, the loss calculation module 530, the adjustment module 540, and the merging module 550 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0168] Figure 6 A block diagram of an electronic device suitable for implementing the above method according to an embodiment of the present disclosure is schematically shown.

[0169] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0170] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0171] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0172] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0173] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0174] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0175] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 601. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0178] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0179] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0181] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for processing financial text data, characterized in that: The method comprises: In response to the data processing instruction, obtaining financial text data to be processed; Performing word segmentation processing on the financial text data to be processed to obtain a keyword set, wherein the keyword set includes a plurality of initial financial keywords; Inputting the plurality of initial financial keywords into a target tagger, and outputting a plurality of target keywords and a part-of-speech classification label corresponding to each of the target keywords; Among them, when training the initial marker, in the case of forward transmission, the keyword intermediate vector generated by the initial marker is deleted, and in the case of reverse transmission, the keyword intermediate vector is recalculated so that the initial marker generates the target marker based on the recalculated keyword intermediate vector.

2. The method according to claim 1, characterized in that Before inputting the plurality of initial financial keywords into the target tagger, the method further includes: Processing the plurality of initial financial keywords using a target attention network to obtain an attention weight corresponding to each of the initial financial keywords; Based on the attention weights, the plurality of initial financial keywords are sorted to obtain a sorted keyword set, so as to input the sorted keyword set into the target marker, wherein the sorted keyword set includes a plurality of sorted initial financial keywords.

3. The method according to claim 2, characterized in that Before sorting, it also includes: For each of the initial financial keywords, when the attention weight of the initial financial keyword is less than a preset weight threshold, the initial financial keyword is deleted to obtain a filtered keyword set, so as to sort the filtered keyword set, wherein the filtered keyword set includes multiple filtered initial financial keywords.

4. The method according to claim 2, characterized in that: Processing the plurality of initial financial keywords using a target attention network to obtain an attention weight corresponding to each of the initial financial keywords includes: For each of the initial financial keywords, perform feature mapping processing on the initial financial keyword to obtain a query vector and a key vector of the initial financial keyword; Generate a similarity score according to the query vector and the key vector, wherein the similarity score represents the degree of association between the initial financial keyword and other related keywords; The similarity score is normalized to obtain the attention weight of the initial financial keyword, wherein the attention weight represents the importance of the initial financial keyword to the other related keywords.

5. The method according to claim 1, characterized in that The financial text data to be processed is segmented to obtain a keyword set, including: Based on the data capacity of the financial text data to be processed, dividing the financial text data to be processed into a plurality of text data blocks; A plurality of word segmenters are used to perform word segmentation processing on the plurality of text data blocks in parallel, so as to obtain a plurality of initial financial keywords corresponding to each of the text data blocks.

6. The method according to claim 1, characterized in that The target marker is trained as follows: Obtain a financial text training set, where the financial text training set includes a plurality of training data blocks generated from at least one financial training text, the training data blocks include a plurality of financial training keywords, and the financial text training set further includes a part-of-speech tag corresponding to each financial training keyword; For each training data block, input each financial training keyword into an initial tagger to obtain a predicted classification tag corresponding to the financial training keyword; For each financial training keyword, generate a loss result according to the predicted classification tag and the part-of-speech tag corresponding to the predicted classification tag; Iteratively adjust the network parameters of the initial tagger according to the loss result to obtain an intermediate tagger; Perform a merging process on multiple intermediate taggers based on weight coefficients to obtain the trained target tagger.

7. The method according to claim 6, characterized in that The initial tagger includes i tagging neurons; Inputting each financial training keyword into the initial tagger to obtain a predicted classification tag corresponding to the financial training keyword includes: Based on a context recognition method, use the m-th tagging neuron to process a plurality of financial training keywords to obtain an m-th keyword intermediate vector corresponding to each financial training keyword, where m < i; In the process of forward propagation, use the (m + n)-th tagging neuron to process the keyword intermediate vector output by the previous tagging neuron to obtain an output result. When m + n < i, the output result represents the (m + n)-th keyword intermediate vector. When m + n = i, the output result represents the predicted classification tag. Any keyword intermediate vector is deleted from the memory after being used by the next tagging neuron; In the process of backward propagation, recalculate the deleted keyword intermediate vector so that the p-th tagging neuron processes the keyword intermediate vector output by the previous tagging neuron based on the keyword intermediate vector.

8. The method according to claim 6 or 7, characterized in that: It further includes: Changing the learning rate in training the target tagger to train the target tagger based on the changed learning rate to obtain a new target tagger; and / or Performing a random combination process on the financial text training set to obtain a new financial text training set, and using the new financial text training set to train the target tagger to obtain a new target tagger.

9. A financial text data processing device, characterized in that: The device includes: An acquisition module for acquiring financial text data to be processed in response to a data processing instruction; A word segmentation module for performing word segmentation on the financial text data to be processed to obtain a keyword set, where the keyword set includes a plurality of initial financial keywords; A classification prediction module for inputting a plurality of initial financial keywords into a target tagger and outputting a plurality of target keywords and a part-of-speech classification tag corresponding to each target keyword; Among them, when training the initial tagger, in the case of forward propagation, the keyword intermediate vector generated by the initial tagger is deleted, and in the case of backward propagation, the keyword intermediate vector is recalculated so that the initial tagger generates the target tagger based on the recalculated keyword intermediate vector.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images