A mass credit short message key information automatic extraction analysis method

Abstract

The application relates to the technical field of data analysis, in particular to a key information automatic extraction and analysis method for massive credit short messages. The method replaces specific information in the short message text through preset semantic labels, and carries out word segmentation and vectorization processing on the short message text to obtain a structured text representation; according to each word segmentation in the semantic unit and the corresponding preset semantic label and template semantic anchor point, the semantic anchor point feature index is determined; the target semantic anchor point is screened out; for each semantic analysis window, the semantic description strength is determined according to the number of target semantic anchor points in the window and the semantic anchor point feature index; the template similarity is determined according to the semantic difference characteristics of the target semantic anchor points and the template semantic anchor points in the window; the regional attention weight of the window is determined according to the semantic description strength and the template similarity; the word vector of each text slice and the corresponding regional attention weight are input into a pre-trained language model, and the text key information is output, so that the accuracy of the key information is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, and particularly relates to a key information automatic extraction and analysis method for massive credit messages. BACKGROUND

[0002] In the field of financial credit, the massive credit messages contain key information such as user repayment reminders, credit changes, and overdue notifications. Accurate extraction of these information is crucial for risk control, customer service, and business analysis.

[0003] Currently, the related art relies on manual preset fixed templates to match the message format. However, credit messages have various text formats (such as large differences in notification templates from different institutions), frequent new scams or marketing messages, and other issues. Using fixed templates not only leads to high template matching missed judgment rate, but also cannot adapt to the semantic analysis needs of unstructured text. In addition, credit messages contain sensitive user information (such as card numbers and amounts), and existing solutions lack systematic design in privacy protection and compliance processing, posing a risk of data leakage. Moreover, pre-trained language models (PLM) are used to extract key information from message text. However, direct processing of message text has high computational cost and insufficient key information focusing ability, which seriously affects the efficiency of extracting key information. SUMMARY

[0004] To solve the technical problem of relying on manual preset fixed templates to match the message format, resulting in high computational cost and insufficient key information focusing ability, which affects the efficiency of extracting key information, the present application provides a key information automatic extraction and analysis method for massive credit messages, the technical solution of which is as follows:

[0005] The present application provides a key information automatic extraction and analysis method for massive credit messages, which includes the following steps:

[0006] Obtain the credit message text and the preset template containing the template semantic label and the corresponding template semantic anchor point. Replace the specific information in the message text with the preset semantic label, and perform word segmentation and vectorization processing on the message text to obtain a structured text representation.

[0007] Divide the structured text representation into semantic units based on punctuation marks as boundaries. Determine the semantic anchor point feature index of each word based on the semantic association relationship between the words in the semantic unit and the corresponding preset semantic label and template semantic anchor point. According to the semantic anchor point feature index, select the target semantic anchor point representing each preset semantic label from the words.

[0008] constructing a semantic analysis window based on semantic units associated with each preset semantic label; for each window, determining a semantic description strength according to the number of target semantic anchors and semantic anchor feature indicators within the window; determining a template similarity according to the semantic difference characteristics of the target semantic anchors and the template semantic anchors within the window; determining a regional attention weight of the window according to the semantic description strength and the template similarity;

[0009] The structured text representation is divided into multiple text slices by window, and the word vectors of each slice and the corresponding regional attention weight are input into a pre-trained language model to output text key information.

[0010] Further, before replacing the specific information in the short message text with the preset semantic label, a key information automatic extraction and analysis method for mass credit short messages further comprises:

[0011] unifying the encoding format of the short message text; converting various currency symbols into a standard form; converting Chinese numerals and special numeral expressions into Arabic numeral form.

[0012] Further, the specific information includes numerical data and institution identification information, and the replacing of the specific information in the short message text with the preset semantic label comprises:

[0013] numerical data is matched using regular expressions, and institution identification information is matched using a financial institution dictionary, and the matched information is replaced with a preset semantic label to form a standardized text.

[0014] Further, the short message text is subjected to word segmentation and vectorization processing to obtain a structured text representation, including:

[0015] The standardized text is subjected to word segmentation processing using a preset word segmentation tool, and stop words are removed to obtain a word segmentation sequence;

[0016] Each word in the word segmentation sequence is converted into a fixed-dimensional word vector to generate a corresponding word vector sequence;

[0017] The standardized text, the word segmentation sequence, and the word vector sequence jointly constitute the structured text representation.

[0018] Further, after the structured text representation is divided into semantic units based on punctuation marks as boundaries, it comprises:

[0019] If the first semantic unit does not contain the preset semantic label, the edge segmentation of the first semantic unit and the segmentation distance and semantic similarity of each preset semantic label in the adjacent semantic unit are calculated; a preset semantic label with the smallest segmentation distance and a semantic similarity greater than a preset threshold is preferentially selected as the preset semantic label of the first semantic unit; if the semantic similarity of all adjacent preset semantic labels is less than the preset threshold, the preset semantic label with the smallest segmentation distance is selected as the preset semantic label of the first semantic unit.

[0020] Further, the semantic anchor feature index determination process comprises:

[0021] For each segmented word in the semantic unit, the segmentation distance between the segmented word and the preset semantic label corresponding to the semantic unit is calculated as the label description distance of the segmented word.

[0022] From the preset template, a template semantic label consistent with the preset semantic label type is selected, and all template semantic anchors corresponding to the template semantic label are obtained; the word vector cosine similarity between the segmented word and each template semantic anchor is calculated, the maximum value in the cosine similarity is taken as the anchor semantic similarity coefficient of the segmented word, and the template semantic anchor corresponding to the maximum value is taken as the target template semantic anchor of the segmented word.

[0023] An anchor feature vector is constructed in combination with the segmentation distance and relative order relationship of the target template semantic anchor and the template semantic label corresponding thereto; a segmentation feature vector of each segmented word is constructed in combination with the segmentation distance and relative order relationship of each segmented word and the preset semantic label; the similarity between the segmentation feature vector of each segmented word and the corresponding anchor feature vector is calculated as the anchor position feature coefficient of each segmented word.

[0024] The product of the anchor semantic similarity coefficient and the anchor position feature coefficient is calculated as the template anchor matching degree index of each segmented word; the ratio of the template anchor matching degree index and the label description distance is calculated as the semantic anchor feature index.

[0025] Further, the filtering of the target semantic anchor representing each preset semantic label from each segmented word according to the semantic anchor feature index comprises:

[0026] For all segmented words corresponding to the same preset semantic label, the segmented words are sorted in descending order of the semantic anchor feature index of each segmented word to obtain a segmented word sorting list.

[0027] The difference between adjacent elements in the segmented word sorting list is calculated, and two target elements associated with the maximum difference value are obtained.

[0028] The target segmented word in the segmented word sorting list, whose semantic anchor feature index is not less than the larger semantic anchor feature index of the two target elements, is selected as the target semantic anchor representing the preset semantic label.

[0029] Further, the semantic description intensity determination process comprises:

[0030] For each target semantic anchor point, extract all interval word vectors between the target semantic anchor point and the associated preset semantic label, calculate the variance of all interval word vectors and perform normalization processing to obtain the modification feature parameter of the target semantic anchor point;

[0031] Statistically count the total number of interval words between each target semantic anchor point and the associated preset semantic label, and normalize the ratio of the total number of interval words to the corresponding modification feature parameter to obtain the modification index of the target semantic anchor point;

[0032] Sum the product of the modification index of all target semantic anchor points and their respective semantic anchor feature index to obtain the semantic description intensity.

[0033] Further, the template similarity determination process comprises:

[0034] From the preset template, extract target template semantic anchor points corresponding to each target semantic anchor point in the semantic analysis window; based on the target template semantic anchor points and their corresponding template semantic labels, and the semantic units associated with the template semantic labels, construct a reference window;

[0035] According to the text sequence, arrange the target semantic anchor points and the preset semantic labels in the semantic analysis window in sequence to form a first sequence; arrange the target template semantic anchor points and the template semantic labels in the reference window according to the same rule to form a second sequence;

[0036] Calculate the average value of the absolute value of the difference between the original sequence numbers of the elements with the same index in the first sequence and the second sequence in their respective windows, and normalize the average value to obtain the structural consistency index;

[0037] Extract all interval words between each target semantic anchor point and its associated preset semantic label in the semantic analysis window to form a first token set; extract all interval words between each target template semantic anchor point and its corresponding template semantic label in the reference window to form a second token set; calculate the average cosine similarity of the word vectors of all tokens in the first token set and the second token set as the modification consistency index;

[0038] Calculate the product of the structural consistency index and the modification consistency index as the template similarity of the semantic analysis window.

[0039] Further, the structured text representation is divided into multiple text slices according to the window, and the word vectors of each slice and the corresponding regional attention weight are input into the pre-trained language model to output the text key information, comprising:

[0040] The structured text representation is divided into multiple text slices by window, and each text slice contains corresponding regional attention weight;

[0041] Each word vector in the text slice is multiplied by the regional attention weight to obtain a modified word vector;

[0042] The self-attention weight of each word in the text slice is multiplied by the regional attention weight element by element to obtain a modified attention matrix; the modified word vector and the modified attention matrix are input into the subsequent layer of the pre-trained language model, and the text key information is output.

[0043] The present application has the following beneficial effects:

[0044] The present application replaces the numerical data and the institutional identification information by the preset semantic label, combines the semantic anchor point screening technology, accurately locates the semantic anchor point with strong association with the key information in the short message text, effectively extracts the core semantic features in the text, and then compares with the traditional preset template, realizes the generalization ability of the new short message format without relying on the fixed format, greatly improves the recall rate and accuracy of the text key information; through the semantic window construction and the regional attention weight setting, only the high-value text slice (the semantic unit containing the text key information) is input into the pre-trained language model, and the key semantic unit is focused by modifying the attention weight, the amount of subsequent invalid data processing is reduced, at the same time, combined with the template similarity semantic description intensity, the model attention distribution is effectively modified, irrelevant information interference is avoided, and the mass data processing efficiency is significantly improved on the premise of ensuring the extraction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0046] Figure 1 A flow chart of a key information automatic extraction and analysis method of a mass credit short message provided by an embodiment of the present application;

[0047] Figure 2 An example diagram of a semantic anchor point feature index determination process provided by an embodiment of the present application;

[0048] Figure 3 An example diagram of a semantic description intensity determination process provided by an embodiment of the present application;

[0049] Figure 4An example diagram of a template similarity determination process is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the following describes in detail the specific implementation, structure, features and effects of a key information automatic extraction and analysis method for mass credit short messages according to the present application, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0052] The following describes in detail the specific scheme of a key information automatic extraction and analysis method for mass credit short messages according to the present application.

[0053] Please refer to Figure 1 which shows a flowchart of a key information automatic extraction and analysis method for mass credit short messages according to an embodiment of the present application. The method comprises:

[0054] S101: Obtain credit short message text and a preset template containing template semantic labels and corresponding template semantic anchor points; replace specific information in the short message text with preset semantic labels, and perform word segmentation and vectorization processing on the short message text to obtain a structured text representation.

[0055] It should be understood that the collected credit short message data comes from multiple channels, and the credit short message usually contains numerical data such as amount, date, card number tail number / contract number, state words (such as loan / repayment / overdue / credit / interest rate), and characteristic nouns such as institution name. The original credit short message data is disorganized and contains sensitive information. Therefore, in order to ensure the integrity and accuracy of the data, protect user privacy, and accurately analyze the structured text representation representing key semantic information, a series of preprocessing is also needed for the original credit short message data.

[0056] It should be noted that the collection channel of the credit short message data is not specifically limited in the present embodiment, for example, it can be credit short message data collected from a credit institution short message gateway, user terminal log, credit short message database of a risk control system historical library, etc., while recording information such as short message sending time, institution ID, and user ID.

[0057] In the embodiment, a series of preprocessing is performed on the original credit short message data, including: constructing a unified data structure containing a unique identification of the short message, a desensitized user identification, a sender identification, a receiving timestamp, and a short message text content; performing privacy and compliance control, collecting only short message meta information and text content supporting key information extraction, not containing sensitive information directly associated with personal identity, and ensuring data security through desensitization, hashing, permission grading, and closed environment processing; using a high-concurrency message queue to realize real-time storage of streaming data, combining extensible object storage to store original data and meta information for offline analysis, forming a real-time and offline parallel storage architecture; and performing normalization processing on the short message text.

[0058] It should be noted that the unified data structure is used to integrate multi-source credit short message data.

[0059] It should be noted that the desensitization processing is achieved by removing the identity identification in the user associated information, the hashing processing converts the user unique identification into an irreversible character sequence, the permission grading mechanism limits the data operation range of different roles, and all data processing is performed in a closed and controlled environment.

[0060] It should be noted that the real-time storage uses a message queue with high concurrency processing capability to carry streaming credit short message data, ensuring low-latency access of data and supporting subsequent real-time analysis requirements; the offline storage uses an extensible object storage service to store original short messages and meta information processed in compliance for a long time, meeting the data requirements of offline analysis, template mining, and other scenarios.

[0061] It should be noted that the normalization processing on the short message text includes: unifying the short message text encoding format; converting various currency symbols to a standard form; converting Chinese numerals and special numeral expressions to Arabic numeral form.

[0062] It should be noted that the specific information includes numerical data and institution identification information, wherein the institution identification information refers to all institution names and associated expressions related to credit business, which are used to explicitly indicate business participants (such as fund providers and service providers) in the short message.

[0063] In the embodiment, regular expressions are used to match numerical data, and a financial institution dictionary is used to match institution identification information, and the matched information is replaced with a preset semantic label to form a standardized text.

[0064] It should be noted that the specific numerical data in the credit short message is determined according to the industry knowledge commonly used in the credit short message field, such as amount, date, number, and other numerical data.

[0065] The financial institution dictionary generally includes the full name, abbreviation and common alias of credit-related institutions such as banks, consumer finance companies and microfinance companies. The financial institution dictionary is the industry knowledge commonly used in the technical field, and will not be described herein.

[0066] It should be noted that the replaced preset semantic labels are different for different numerical data and institution identification information. For example, the preset semantic labels include an amount label, a date label, an institution label and a number label, which are respectively used to replace the amount value, date information, financial institution name and contract or card number in the short message text.

[0067] For example, the short message text is "Dear customer, your A bank credit card with tail number 1234, this period's bill has been issued, the amount to be paid is 3000 yuan, the payment date is December 20, 2024, please pay on time." Assuming that the amount label is <amt>, date tag is <date>, the institutional label is <bank>then match "3000 yuan" with a regular expression and replace it with <amt>, "A Bank" is replaced by matching the financial institution dictionary <bank>At the same time, the regular expression is used to replace "2024-12-20" with <date>, get "Dear customer, your account number 1234 is <bank>credit card, this month's bill is out, pay up <amt>Repayment date is <date>Please pay back in time.

[0068] The preset template is a set of predefined structured rules for guiding the semantic analysis of the short message text. The specific content of the preset template is summarized according to the public knowledge in the technical field, and the embodiment will not be described herein.

[0069] For example, the preset template includes <amt>(Amount) <date>(Date), and <bank>(BANK, institution) three types of template semantic tags and corresponding template semantic anchor points (such as <amt>corresponding "to be returned" "amount"; <date>corresponding to the "due date" "due date").

[0070] In this embodiment, the standardized text is processed by a preset word segmentation tool to remove stop words and obtain a word segmentation sequence; each word in the word segmentation sequence is converted into a fixed-dimensional word vector to generate a corresponding word vector sequence; the standardized text, the word segmentation sequence, and the word vector sequence jointly constitute the structured text representation.

[0071] It should be noted that the preset word segmentation tool can use a publicly available word segmentation tool, for example, the jieba word segmentation tool.

[0072] It should be noted that the specific method of word vector conversion is a well-known technical means to those skilled in the art, and will not be described in detail in this embodiment. For example, each word is converted into a fixed-dimensional (such as 100-300 dimensional) word vector using a word2vec model.

[0073] For example, the standardized text is processed by the jieba word segmentation tool to remove the stop words "of" "for" "please" and obtain the word segmentation sequence ["respectful", "customer", "tail number", "1234", "BANK", "credit card", "this period", "bill", "has been issued", "should be returned", "AMT", "repayment date", "DATE", "on time", "repayment"], and then the words are converted into 128-dimensional word vectors to form the structured text representation.

[0074] S102: dividing the structured text representation into semantic units based on punctuation marks as boundaries; determining semantic anchor feature indicators of each word based on semantic association relationships between the words in the semantic units and corresponding preset semantic labels and template semantic anchors; and selecting target semantic anchors representing each preset semantic label from the words based on the semantic anchor feature indicators.

[0075] It should be noted that the punctuation marks at least include a period, a semicolon, and an exclamation point.

[0076] For example, the structured text representation is divided into three semantic units based on punctuation marks as boundaries: semantic unit 1: "Dear customer, your tail number 1234 <bank>Credit card, this bill has been issued"; semantic unit 2: "should be paid back <amt>" ; semantic unit 3: "payment date is <date>Please pay back in time.

[0077] In the embodiment, if there is a first semantic unit not containing a preset semantic label, the edge tokenization of the first semantic unit and the tokenization distance and semantic similarity of each preset semantic label in the adjacent semantic unit are calculated; a preset semantic label with the minimum tokenization distance and greater than a preset threshold is preferentially selected as the preset semantic label of the first semantic unit; if the semantic similarity of all adjacent preset semantic labels is less than the preset threshold, the preset semantic label with the minimum tokenization distance is selected as the preset semantic label of the first semantic unit.

[0078] The edge tokenization is the tokenization at the beginning or end of the first semantic unit, for example, the edge tokenization of "warm prompt" can be "prompt" if the first semantic unit is "warm prompt".

[0079] The semantic similarity is used to indicate the cosine similarity of the word vector of the edge tokenization of the first semantic unit and the preset semantic label in the adjacent semantic unit.

[0080] It should be noted that the specific value of the preset threshold is determined according to the actual situation, and the embodiment is not limited specifically, for example, the preset threshold can be 0.6.

[0081] The tokenization distance refers to the interval number between the positions of two tokenizations in the tokenization sequence of the short message text, which is used to quantitatively represent the relative position relationship of different tokenizations in the text, wherein the smaller the tokenization distance between two tokenizations, the closer they are in the text, and the more closely the semantic association is.

[0082] It can be understood that the preset semantic label is also a tokenization in the text.

[0083] For example, the tokenizations obtained after the tokenization processing of the short message text are arranged in the original text order to form an ordered tokenization sequence, each tokenization corresponds to a unique position sequence number (increasing from 0 or 1) in the sequence, for any two tokenizations A and B, assuming that the position sequence number of A is i and the position sequence number of B is j, the tokenization distance of the two is│i-j│ (i.e. the absolute value of the position sequence number difference).

[0084] The semantic anchor feature index determination process is as shown in Figure 2 , which includes:

[0085] S102-1: For each tokenization in the semantic unit, the tokenization distance of the tokenization and the preset semantic label corresponding to the semantic unit is calculated as the label description distance of the tokenization.

[0086] It can be understood that the label description distance of the preset semantic label is not calculated.

[0087] For example, the "due date" with the preset semantic label of semantic unit 3 <date>The word segmentation sequence is adjacent, the word segmentation distance is 1, and therefore the label description distance is 1.

[0088] S102-2: From the preset template, a template semantic label consistent with the preset semantic label type is selected, and all template semantic anchor points corresponding to the template semantic label are obtained; a word vector cosine similarity between the word segmentation and each template semantic anchor point is calculated, the maximum value in the cosine similarity is taken as an anchor point semantic similarity coefficient of the word segmentation, and the template semantic anchor point corresponding to the maximum value is taken as a target template semantic anchor point of the word segmentation.

[0089] For example, in the preset template <date>The corresponding template semantic anchor point is "repayment date" "due date" on the date. The word vector cosine similarity between the segmented word (repayment date) and each template semantic anchor point is calculated. The cosine similarity with the template semantic anchor point (repayment date) is the maximum value, and the target template semantic anchor point of the segmented word (repayment date) is "repayment date".

[0090] It should be noted that the specific calculation method of the word vector cosine similarity is a well-known technical means to those skilled in the art, and will not be described here.

[0091] S102-3: Construct an anchor point feature vector combining the segmented word distance and relative order relationship of the target template semantic anchor point and its corresponding template semantic label; construct a segmented word feature vector of each segmented word combining the segmented word distance and relative order relationship of each segmented word and a preset semantic label; and calculate the similarity between the segmented word feature vector of each segmented word and the corresponding anchor point feature vector as the anchor point position feature coefficient of each segmented word.

[0092] It should be noted that if the template semantic label is before the target template semantic anchor point, the relative order relationship is in the negative direction; if the template semantic label is after the target template semantic anchor point, the relative order relationship is in the positive direction, for example, the target template semantic anchor point "repayment date" and its corresponding template semantic label <date>the word segmentation distance is 1 (preset in the template), and <date>If the "repayment date" is after the "due date" (the relative order relationship is positive), the anchor feature vector is (1, positive).

[0093] Similarly, if the preset semantic label is before the segmentation, the relative order relationship is negative; if the preset semantic label is after the segmentation, the relative order relationship is positive, for example, the segmentation "repayment date" and its corresponding preset semantic label <date>the word segmentation distance of " " is 1, and <date>After "repayment date" (with a positive relative word order), the word segmentation feature vector of "(repayment date)" is (1, positive).

[0094] It should be noted that the specific methods for measuring the similarity between two vectors are well-known techniques to those skilled in the art, and will not be elaborated upon in this embodiment. For example, the cosine similarity between the word segmentation feature vector and the corresponding anchor feature vector can be calculated.

[0095] S102-4: Calculate the product of the anchor point semantic similarity coefficient and the anchor point position feature coefficient, and use it as the template anchor point matching degree index for each word segmentation; calculate the ratio of the template anchor point matching degree index to the tag description distance, and use it as the semantic anchor point feature index.

[0096] It's important to understand that a higher semantic similarity coefficient for a word segment indicates a higher semantic similarity between that word and the target template's semantic anchor in the preset template, thus increasing its probability of serving as a semantic anchor in the current structured text representation. Furthermore, a higher anchor position feature coefficient indicates a closer "positional relationship" between that word and the target template's semantic anchor (e.g., both located one position before the semantic tag), meaning that the word spatially possesses the characteristics of a semantic anchor. Conversely, a greater tag description distance indicates a greater distance between that word and the key information (preset semantic tags), making it less likely to serve as a semantic anchor. Therefore, by comprehensively analyzing each word from the three dimensions of "semantic similarity, positional proximity, and conformity to template features," a better overall performance for a word suggests that it is more likely to be a true semantic anchor.

[0097] It should be noted that when calculating the semantic anchor feature index, if the label description distance is zero (i.e., the target semantic anchor and the preset semantic label are in the same position in the word segmentation sequence), the label description distance is directly assigned a preset minimum positive value (e.g., 0.001). The preset minimum positive value is a constant much less than 1, and its value does not affect the relative size ranking of the semantic anchor feature index. It is only used to ensure the mathematical validity of the division operation.

[0098] In this embodiment, for all word segments corresponding to the same preset semantic tag, they are sorted in descending order of semantic anchor feature index to obtain a word segmentation sorting list; the difference between adjacent elements in the word segmentation sorting list is calculated, and the two target elements associated with the largest difference are obtained; the target word in the word segmentation sorting list with a semantic anchor feature index not less than the larger semantic anchor feature index of the two target elements is selected, and the target word is used as the target semantic anchor representing the preset semantic tag.

[0099] It should be noted that the same preset semantic tag can correspond to multiple word segments.

[0100] For example, for "all words corresponding to the same preset semantic label", the words are sorted in descending order according to the semantic anchor feature indicators, and a word sorting list is obtained, which is assumed to be [1.8, 1.7, 1.0, 0.9]. The two elements with the largest adjacent element difference (i.e., "inflection point") are found, wherein the inflection point with the largest adjacent difference is 1.7 and 1.0 (difference 0.7). The target word with a semantic anchor feature indicator not less than 1.7 is selected as the target semantic anchor.

[0101] S103: Construct a semantic analysis window based on the semantic units associated with each preset semantic label; for each window, determine the semantic description strength according to the number of target semantic anchors in the window and the semantic anchor feature indicators; determine the template similarity according to the semantic difference characteristics of the target semantic anchors and the template semantic anchors in the window; determine the regional attention weight of the window according to the semantic description strength and the template similarity.

[0102] It should be understood that in the complex context of the short message text, the preset semantic label is often closely related to its context, and simply identifying the semantic label is not enough to analyze the key business features of the short message text. Since one label may correspond to multiple target semantic anchors, and the real business meaning needs to be analyzed in combination with the semantic label and the context, it is necessary to take the preset semantic label as the center, combine the surrounding semantic anchors, and establish an analysis window to further focus on the key information.

[0103] It should be noted that the analysis window needs to contain two types of semantic units: one type of semantic unit is the semantic unit divided according to the punctuation marks; the other type of semantic unit is the semantic unit to which the target semantic anchors corresponding to the preset semantic label belong (for example <amt>The corresponding target semantic anchor "should return" is in the "should return <amt>"semantic unit").

[0104] For example, if the preset semantic label <date>(date) is "Please return the completed form to <date>"front payment", the corresponding target semantic anchor point "due" where the sentence (i.e., semantic unit) is "bill is due", then <date>the analysis window must contain "please <date>"Please pay the outstanding balance by the due date" and "The bill is due" form the complete context "The bill is due. Please pay the outstanding balance by the due date." <date>"front payoff".

[0105] The modification indicators of all target semantic anchors are summed with the multiplication of the respective semantic anchor feature indicators to obtain a semantic description intensity.

[0106] It should be understood that the measurement of whether a target semantic anchor of a semantic analysis window can clearly and accurately describe the key information can be analyzed from two dimensions of "message context itself" and "similarity with the template".

[0107] The semantic description intensity determination process, as shown in Figure 3 , includes:

[0108] It should be understood that focusing on the message context itself can be achieved by focusing on the semantic quality of the window itself, that is, the more dense and strong the target semantic anchors in the window are, and the more modification words (such as "must" and "as soon as possible") there are, the more business urgency can be reflected, and the more sufficient the semantic description of the window to the preset semantic label is, and the stronger the business indication is.

[0109] S103-1: For each target semantic anchor, extract the word vectors of all interval tokens between the target semantic anchor and the associated preset semantic label, calculate the variance of all interval token word vectors and perform normalization processing to obtain the modification feature parameter of the target semantic anchor.

[0110] It should be understood that in the context of credit messages, the character distance between the semantic label and its corresponding semantic anchor does not simply reflect the strength of semantic relevance. When the character distance is large, there are usually more modification words indicating the same direction between them. These modification words are often used to emphasize urgency or importance, such as "must immediately", "please as soon as possible", "must be completed before × × day", etc. Unlike the rule that the relevance decreases as the distance increases in general text, in the context of credit messages, the existence of such modification words often gives the semantic anchor stronger business indication and urgency. Therefore, in the analysis process, higher weight should be given to such long-distance semantic anchors to reflect their actual importance in semantic interpretation and business risk prompt.

[0111] It should be noted that the specific calculation method of the variance of the word vector is a well-known technical means to those skilled in the art, and will not be described herein.

[0112] For example, assuming that a semantic analysis window is: "penalty <amt>, the interest is calculated by day", wherein the preset semantic label is <amt>, the target semantic anchor is "penalty interest", the window is <amt>The total number of associated target semantic anchor points is two for "penalty": "penalty" and <amt>There is no interval between the words, the word vector variance is 0, and the normalized modification feature parameter = 1.0 (the smaller the variance, the smaller the interference of interval word segmentation on the target semantic anchor point, and the higher the modification feature parameter); for "penalty interest": "penalty interest" and <amt>1 wordpiece, the variance of the word vector is small, and the normalized modification feature parameter = 0.8.

[0113] S103-2: Count the total number of interval wordpieces between each target semantic anchor point and the associated preset semantic label, normalize the ratio of the total number of interval wordpieces to the corresponding modification feature parameter to obtain the modification index of the target semantic anchor point.

[0114] It should be noted that after normalization, even if the variance is 0 (such as no interval wordpieces or interval wordpieces are completely consistent), the normalized parameter will be mapped to the maximum value (such as 1.0), and the modification feature parameter cannot be zero.

[0115] S103-3: Sum the product of the modification index of all target semantic anchor points and the respective semantic anchor point feature index to obtain the semantic description strength.

[0116] It should be noted that the more the total number of interval wordpieces of a certain target semantic anchor point, the more semantic clues related to the key information in the window; the greater the ratio, the stronger the "business indication strengthening effect" of the modifying words on the semantic anchor point (such as multiple groups of same-direction modifying words superimposed, highlighting the urgency of the repayment date), the greater the modification index.

[0117] The template similarity determination process is as shown in Figure 4 , which includes:

[0118] S103-4: Extract target template semantic anchor points corresponding to each target semantic anchor point in the semantic analysis window from the preset template; based on the target template semantic anchor points and their corresponding template semantic labels, and the semantic units associated with the template semantic labels, construct a reference window.

[0119] It should be noted that there are a large number of fixed structures in the credit SMS (such as the amount of money after "should be paid" and the date after "due date"), even if the overall SMS does not match the template, if the local window is similar to the structure of the preset template, it still has high key information value, therefore, in order to measure the matching degree of the semantic analysis window and the preset template, the structure of the analysis window and the modification words can be analyzed to see if they conform to the typical expression mode of the key information in the credit business.

[0120] For example, based on the extracted target template semantic anchor point (such as "should be paid"), find its corresponding template semantic label in the preset template <amt>In the preset template, find the one containing <amt>and semantic units of the target template semantic anchor. It is assumed that there is "should be returned" in the preset template <amt>Please pay on time" such as the expression, the target template semantic anchor, template semantic label and the associated semantic unit are integrated, build out the reference window "should pay <amt>Please pay on time.

[0121] S103-5: According to the text sequence, arrange the target semantic anchor points and preset semantic tags in the semantic analysis window in turn to form a first sequence; arrange the target template semantic anchor points and template semantic tags in the reference window according to the same rule to form a second sequence.

[0122] Text sequence, the arrangement order of the word segmentation in the text, maintains the order of the word segmentation in the original short message text, ignores irrelevant word segmentation (such as punctuation, auxiliary word), and only arranges the core word segmentation in turn.

[0123] For example, assume that the window content of the semantic analysis window is: "penalty <amt>, the core element: target semantic anchor point "penalty interest", the preset semantic label <amount>, the original word order (from left to right): "penalty interest" <amt>→ "penalty interest", then the first sequence: ["penalty", "penalty interest", <amt>, "penalty interest"] ; assume reference window window content: "shall be paid <amt>, overdue by the day interest", core element: target template semantic anchor point " should be returned" " overdue", template semantic label <amt>, the primitive sequence (from left to right): "should return" -> "should return" <amt>→ "overdue", then the second sequence: ["due", "overdue", <amt>, "overdue" ].

[0124] S103-6: Calculate the average value of the absolute value of the difference between the original sequence numbers of the elements with the same index in the first sequence and the second sequence in the respective corresponding window, normalize the average value, and obtain the structural consistency index.

[0125] It should be noted that if the number of elements is inconsistent, virtual elements are filled to make the sequence length equal.

[0126] It should be noted that before calculating the average value, the first sequence and the second sequence are first aligned in sequence, taking the "element index" of the sequence as the reference, and the elements with the same index in the first sequence and the second sequence are one-to-one corresponding, for example, the first element in the first sequence corresponds to the first element in the second sequence, the second element in the first sequence corresponds to the second element in the second sequence, and so on.

[0127] In this embodiment, the window corresponding to the elements of the first sequence is the semantic analysis window; and the window corresponding to the elements of the second sequence is the reference window.

[0128] The original sequence number refers to the real position sequence number of an element in a segmented sequence, which is not an index within the sequence.

[0129] For example, it is assumed that: the original sequence number of the element of the first sequence is "penalty for breach of contract" (the real position sequence number is 5), <amt>(True position number is 6), "Penalty interest" (True position number is 8); Original position number of the second sequence element: "Ownership" (True position number is 2) <amt>If the original sequence number difference is 3, the absolute value of the difference is: 1st position ("penalty" and "due"): |5-2|=3; 2nd position ("overdue" and "due"): |3-5|=2; 3rd position ("overdue" and "due"): |3-3|=0. Thus, the 1st position is the most significant position, the 2nd position is the second most significant position, and the 3rd position is the least significant position. <amt>With <amt>|6-3|=3; 3rd position ("penalty" vs. "overdue"): |8-5|=3.

[0130] S103-7: Extract all interval tokens between each target semantic anchor and its associated preset semantic label within the semantic analysis window, to form a first token set; extract all interval tokens between each target template semantic anchor and its corresponding template semantic label within the reference window, to form a second token set; calculate the average cosine similarity of the word vectors of all tokens in the first token set and the second token set as the modification consistency index.

[0131] It should be noted that in the case where the first token set or the second token set is empty, the average cosine similarity of the word vectors of all tokens in the first token set and the second token set is temporarily set to 1.

[0132] It should be noted that in the case where the number of elements in the first token set and the second token set is not the same, the set with fewer elements needs to be completed first (completion rule: repeat the elements already in the set until the number of elements in the first token set and the second token set is consistent), and then calculate the cosine similarity of the word vectors of the corresponding tokens.

[0133] In this embodiment, the cosine similarity of the word vectors of the corresponding tokens in the first token set and the second token set is calculated as the first similarity, and the average of all first similarities is calculated as the modification consistency index.

[0134] S103-8: Calculate the product of the structure consistency index and the modification consistency index as the template similarity of the semantic analysis window.

[0135] It should be noted that "cosine similarity" is used to measure the angle between two vectors. The smaller the angle, the closer the similarity is to 1 (the more similar the semantics); the larger the angle, the closer the similarity is to 0 (the less similar the semantics). Therefore, if the modification consistency index is larger, it means that the semantic pattern and word usage habit of the modifying words in the semantic analysis window are more consistent with the preset template (for example, the preset template "please as soon as possible" is more consistent with the semantic pattern "please as soon as possible" in the semantic analysis window). <date>, semantic analysis window also appears "please as soon as possible"); if the structural consistency index is greater, the order structure of the preset semantic label-target semantic anchor point in the semantic analysis window is more consistent with the template, and the key information expression of the credit message is more consistent, and the template similarity of the semantic analysis window is also higher.

[0136] In this embodiment, the product of the semantic description intensity and the template similarity is normalized as the window value score of the window; and the window value score is used as the regional attention weight for determining the window.

[0137] It should be understood that the greater the semantic description intensity of a certain semantic analysis window, the richer the semantic clues related to the key information in the semantic analysis window and the stronger the "business indication reinforcement" of the modifier words on the semantic anchor point, and the more important the semantic analysis window is to the overall structured text representation; the higher the template similarity of a certain semantic analysis window, the higher the matching degree of the semantic analysis window with the preset template, and the more important the semantic analysis window.

[0138] S104: divide the structured text representation into multiple text slices according to the window, input the word vectors of each slice and the corresponding regional attention weight into the pre-trained language model, and output the text key information.

[0139] It should be understood that the pre-trained language model has the same attention degree for all words in the text by default. When directly processing a large amount of credit messages, the pre-trained language model is easy to be distracted by irrelevant information (such as the greeting "Respected customer" and the redundant prompt "Please handle it in time"), and the accuracy of text key information extraction is reduced. It is known that the values of different text slices are different, and the high-value text slice contains the main key information, so the pre-trained language model should be guided to focus on the high-value text slice first. Therefore, the word vectors of the high-weight window can be amplified by the regional attention weight, the self-attention of the words in the high-weight window can be strengthened and focused, and finally the attention distribution can be adjusted to reduce irrelevant information interference and improve the recall rate of key information.

[0140] In this embodiment, the structured text representation is divided into multiple text slices according to the window, and each text slice contains a corresponding regional attention weight; each word vector in the text slice is multiplied by the regional attention weight to obtain a modified word vector; the self-attention weight of each word in the text slice is multiplied by the regional attention weight element by element to obtain a modified attention matrix; the modified word vector and the modified attention matrix are input into the subsequent layer of the pre-trained language model, and the text key information is output.

[0141] It should be noted that one text slice corresponds to one semantic analysis window, and contains the complete short message text segment in the semantic analysis window.

[0142] It should be noted that the text slice carries not only the regional attention weight, but also the start and end indexes, which are used to locate the real position of the text key information in the original short message text when the text key information is output subsequently.

[0143] It should be noted that the self-attention weight of each word is determined based on the self-attention matrix output by the subsequent (encoder) layer of the pre-trained language model, and the embodiment will not be described again.

[0144] It should be noted that the above-mentioned embodiment sequence of the application is only for description, and does not represent the advantages and disadvantages of the embodiment. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or may be advantageous.

[0145] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments.< / date> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / amt> < / date> < / date> < / date> < / date> < / date> < / amt> < / amt> < / date> < / date> < / date> < / date> < / date> < / date> < / date> < / amt> < / bank> < / date> < / amt> < / bank> < / date> < / amt> < / date> < / amt> < / bank> < / date> < / bank> < / amt> < / bank> < / date> < / amt>

Claims

1. A method for automatically extracting and analyzing key information of mass credit short messages, characterized in that, The method comprises: acquiring credit short message text and a preset template containing template semantic labels and corresponding template semantic anchors; replacing specific information in the short message text with preset semantic labels, and performing word segmentation and vectorization processing on the short message text to obtain a structured text representation; dividing the structured text representation into semantic units with punctuation marks as boundaries; determining semantic anchor feature indicators of each word based on the semantic association relationship between the words in the semantic units and the corresponding preset semantic labels and template semantic anchors; and selecting target semantic anchors representing each preset semantic label from the words according to the semantic anchor feature indicators; constructing a semantic analysis window based on the semantic units associated with each preset semantic label; determining the semantic description strength of each window according to the number of target semantic anchors in the window and the semantic anchor feature indicators; determining the template similarity according to the semantic difference characteristics of the target semantic anchors in the window and the template semantic anchors; and determining the regional attention weight of the window according to the semantic description strength and the template similarity; dividing the structured text representation into multiple text slices according to the windows, and inputting the word vectors of each slice and the corresponding regional attention weight into a pre-trained language model to output the key information of the text.

2. The method according to claim 1, wherein, Before replacing the specific information in the short message text with the preset semantic labels, the method further comprises: unifying the encoding format of the short message text; converting various currency symbols into a standard form; and converting Chinese numerals and special numeral expressions into Arabic numeral form.

3. The method of claim 2, wherein the key information of the mass credit short message is automatically extracted and analyzed, and the key information comprises the following information: The specific information includes numerical data and institution identification information, and the replacing of the specific information in the short message text with the preset semantic labels comprises: ​ matching the numerical data using a regular expression and matching the institution identification information using a financial institution dictionary, and replacing the matched information with the preset semantic labels to form a standardized text.

4. The method according to claim 3, wherein, The word segmentation and vectorization processing on the short message text to obtain the structured text representation comprises: performing word segmentation processing on the standardized text using a preset word segmentation tool, removing stop words, and obtaining a word segmentation sequence; converting each word in the word segmentation sequence into a fixed-dimensional word vector to generate a corresponding word vector sequence; the standardized text, the word segmentation sequence and the word vector sequence jointly constitute the structured text representation.

5. The method of claim 1, wherein the method further comprises: After dividing the structured text representation into semantic units with punctuation marks as boundaries, the method comprises: if there is a first semantic unit that does not contain a preset semantic label, calculating the word segmentation distance and semantic similarity of the edge word segmentation of the first semantic unit and each preset semantic label in the adjacent semantic unit; preferentially selecting a preset semantic label with the smallest word segmentation distance and a semantic similarity greater than a preset threshold as the preset semantic label of the first semantic unit; and if the semantic similarity of all adjacent preset semantic labels is lower than the preset threshold, selecting the preset semantic label with the smallest word segmentation distance as the preset semantic label of the first semantic unit.

6. The method of claim 5, wherein the key information of the mass credit short message is automatically extracted and analyzed, and The semantic anchor feature indicator determination process comprises: ​ for each word in the semantic unit, calculating the word segmentation distance between the word and the corresponding preset semantic label of the semantic unit as the label description distance of the word; From the preset template, select the template semantic label consistent with the preset semantic label type, and obtain all template semantic anchors corresponding to the template semantic label; calculate the word vector cosine similarity between the segmentation and each template semantic anchor, take the maximum value in the cosine similarity as the anchor point semantic similarity coefficient of the segmentation, and take the template semantic anchor corresponding to the maximum value as the target template semantic anchor of the segmentation; Combine the segmentation distance and relative order relationship of the target template semantic anchor and its corresponding template semantic label to construct an anchor point feature vector; combine the segmentation distance and relative order relationship of each segmentation and the preset semantic label to construct the segmentation feature vector of each segmentation; calculate the similarity between the segmentation feature vector of each segmentation and the corresponding anchor point feature vector as the anchor point position feature coefficient of each segmentation; Calculate the product of the anchor point semantic similarity coefficient and the anchor point position feature coefficient as the template anchor matching degree index of each segmentation; calculate the ratio of the template anchor matching degree index and the label description distance as the semantic anchor feature index.

7. The method of claim 6, wherein the key information of the mass credit short message is automatically extracted and analyzed. The target semantic anchor representing each preset semantic label is selected from each segmentation according to the semantic anchor feature index, including: For all segmentations corresponding to the same preset semantic label, sort the segmentations according to the semantic anchor feature index from large to small to obtain a segmentation sorting list; Calculate the difference value of adjacent elements in the segmentation sorting list, and obtain the two target elements associated with the maximum difference value; Select the target segmentation in the segmentation sorting list whose semantic anchor feature index is not less than the larger semantic anchor feature index of the two target elements, and take the target segmentation as the target semantic anchor representing the preset semantic label.

8. The method of claim 4, wherein the key information of the mass credit short message is automatically extracted and analyzed, and The semantic description intensity determination process includes: ​ For each target semantic anchor, extract the word vectors of all interval segmentations between the target semantic anchor and the associated preset semantic label, calculate the variance of the word vectors of all interval segmentations and perform normalization processing to obtain the modification feature parameter of the target semantic anchor; Statistically count the total number of interval segmentations between each target semantic anchor and the associated preset semantic label, and normalize the ratio of the total number of interval segmentations to the corresponding modification feature parameter to obtain the modification index of the target semantic anchor; Sum the product of the modification index of all target semantic anchors and their respective semantic anchor feature indexes to obtain the semantic description intensity.

9. The method of claim 6, wherein the key information of the mass credit short message is automatically extracted and analyzed, and the key information comprises at least one of the following: The template similarity determination process includes: ​ From the preset template, extract the target template semantic anchor corresponding to each target semantic anchor in the semantic analysis window; based on the target template semantic anchor and its corresponding template semantic label, and the semantic unit associated with the template semantic label, construct a reference window; According to the text order, arrange the target semantic anchors and preset semantic labels in the semantic analysis window in sequence to form a first sequence; arrange the target template semantic anchors and template semantic labels in the reference window according to the same rule to form a second sequence; Calculate the average value of the absolute value of the difference between the original sequence numbers of the elements with the same index in the first sequence and the second sequence in their respective windows, and normalize the average value to obtain the structural consistency index; Extract all interval words between each target semantic anchor point and its associated preset semantic label in the semantic analysis window to form a first token set; extract all interval words between each target template semantic anchor point and its corresponding template semantic label in the reference window to form a second token set; calculate the average cosine similarity of the word vectors of all tokens in the first token set and the second token set as the modification consistency index; Calculate the product of the structure consistency index and the modification consistency index as the template similarity of the semantic analysis window.

10. The method of claim 4, wherein the key information of the mass credit short message is automatically extracted and analyzed, and The structured text representation is divided into multiple text slices by window, and the word vectors of each slice and the corresponding regional attention weight are input into the pre-trained language model to output the text key information, including: ​ The structured text representation is divided into multiple text slices by window, and each text slice contains a corresponding regional attention weight; Each word vector in the text slice is multiplied by the regional attention weight to obtain a modified word vector; The self-attention weight of each token in the text slice is multiplied by the regional attention weight element by element to obtain a modified attention matrix; the modified word vector and the modified attention matrix are input into the subsequent layer of the pre-trained language model to output the text key information.

Images