A marketing knowledge text matching method based on multiple data sources

Abstract

The present invention provides a marketing knowledge text matching method based on multiple data sources in the field of text matching technology, comprising the following steps: Step S10, acquiring a large number of marketing knowledge texts including triple data and text data; Step S20, preprocessing each of the marketing knowledge texts to construct a text dataset; Step S30, creating a text matching model and training and optimizing the text matching model using the text dataset; Step S40, evaluating the performance of the text matching model; and Step S50, using the text matching model to perform similarity matching on the text data to be matched. The present invention has the advantage of significantly improving the accuracy and generalization capability of text matching.

Description

Technical Field

[0001] The present invention relates to the technical field of text matching, and in particular to a marketing knowledge text matching method based on multiple data sources. Background Art

[0002] With the rapid development of Internet information technology, massive amounts of text data have emerged. These text data usually contain useful similar information. Therefore, it is necessary to retrieve the similarity between text data, that is, to perform text similarity matching.

[0003] Text similarity matching plays a core role in the field of natural language processing. Matching retrieval information, intelligent question answering, machine translation, dialogue systems, interpretation recognition, etc. can all be summarized as text similarity matching problems. Their essence is to match the similarity between text data, extract similar information from it, and apply this similar information to search engines, question answering systems, text summarization, etc.

[0004] Traditional text similarity matching algorithms primarily rely on manually extracted features, analyzing features extracted from words, characters, and sentences within text data. These typically include methods like TF-IDF and Jaccard distance. TF-IDF uses word frequency as a feature to express the importance of vocabulary within text data, while Jaccard distance measures similarity by comparing the number of repeated words within text data.

[0005] However, traditional text similarity matching algorithms have the following problems:

[0006] 1. In-depth semantic understanding: There is a lack of consideration for the semantic understanding between texts. Only the organization and composition of the text are considered. The matching ability between texts has certain bottlenecks, making it difficult to correctly match the semantics of two texts. Matching is more based on the words themselves, while ignoring deeper semantic information and the potential semantics contained in the texts.

[0007] 2. Polysemy: Since the same word or sentence can express multiple semantic information, many words have polysemy in different contexts, which makes it difficult for traditional text similarity matching algorithms to distinguish.

[0008] 3. Structural issues in text: A phrase can be composed of multiple words in a certain structure. Even if the words in the text are the same, information with different structures will produce different semantics. For example, "machine learning" and "learning machine" have the same words, but the semantics are completely different when the order is reversed, and the degree of matching is also completely inconsistent.

[0009] 4. Hierarchical nature of long texts: Since text is composed in a hierarchical manner, that is, words are composed of phrases, phrases are composed of sentences, and sentences are composed of paragraphs, low-level information is gradually constructed into high-level information in a hierarchical manner. Traditional text similarity matching algorithms ignore the matching information between different levels.

[0010] Therefore, how to provide a marketing knowledge text matching method based on multiple data sources to improve the accuracy and generalization ability of text matching has become a technical problem that needs to be solved urgently. Summary of the Invention

[0011] The technical problem to be solved by the present invention is to provide a marketing knowledge text matching method based on multiple data sources to improve the accuracy and generalization ability of text matching.

[0012] The present invention is implemented as follows: a marketing knowledge text matching method based on multiple data sources, comprising the following steps:

[0013] Step S10: Acquire a large amount of marketing knowledge text including triple data and text data;

[0014] Step S20: pre-processing each of the marketing knowledge texts to construct a text dataset;

[0015] Step S30: creating a text matching model, and using the text dataset to train and optimize the text matching model;

[0016] Step S40: Evaluate the performance of the text matching model;

[0017] Step S50: Use the text matching model to perform similarity matching on the text data to be matched.

[0018] Furthermore, the step S10 is specifically as follows:

[0019] Acquire a large amount of triple data of marketing knowledge from a graph database, extract entity words and relation words from the triple data, and concatenate them;

[0020] Obtaining large amounts of marketing knowledge text data from relational databases;

[0021] The triple data and the text data are merged to obtain marketing knowledge text.

[0022] Furthermore, the step S20 specifically includes:

[0023] Step S21: removing noise data from each of the marketing knowledge texts;

[0024] Step S22: performing deduplication processing on each of the marketing knowledge texts;

[0025] Step S23: using a word segmentation tool to perform word segmentation on each of the marketing knowledge texts to obtain a plurality of phrases, and removing stop words from the phrases based on a preset stop word list;

[0026] Step S24: taking each of the phrases as a positive sample, and selecting a preset number of phrases in different fields as negative samples for each of the positive samples, thereby constructing a text dataset.

[0027] Furthermore, in step S21, the noise data at least includes a URL, a picture, an expression, an ID, special symbols, and a request time.

[0028] Furthermore, in step S30, the text matching model includes a vectorization module, a global relevance aggregation module and an output module; the input end of the global relevance aggregation module is connected to the vectorization module, and the output end is connected to the output module.

[0029] Furthermore, the vectorization module is used to vectorize triple data and text data;

[0030] The vectorization of triple data is specifically as follows:

[0031] Use the BERT pre-trained model to vectorize each metadata of the triple data to obtain the first word embedding of each metadata;

[0032] Create a parameter matrix, and use the parameter matrix to map each first word embedding to a specified text length to obtain a second word embedding;

[0033] Use the softmax function to requantize the weight of the second Word Embedding to obtain the third Word Embedding;

[0034] After comparing and fusing the first word embedding and the third word embedding, a pooling operation is performed to obtain a triple vector, completing the vectorization of the triple data;

[0035] The vectorization of text data is specifically as follows:

[0036] Based on word granularity, the text data is converted into BERT Embedding using the BERT pre-trained model;

[0037] Based on word granularity, the text data is converted into a fourth word embedding using the Word2Vec model;

[0038] The BERT Embedding and the fourth Word Embedding are fused together using average pooling and maximum pooling to obtain a text data vector, thereby completing the vectorization of the text data.

[0039] Furthermore, the global correlation aggregation module adopts a Bi-LSTM network to learn the mapping relationship between the vector sequence output by the matching vectorization module and the true matching degree;

[0040] The Bi-LSTM network selects cross entropy as the loss function and is trained by the back-propagation algorithm.

[0041] Furthermore, the output module is used to output a binary classification result through a sigmoid function.

[0042] Furthermore, the step S40 specifically includes:

[0043] Step S41: Evaluate the performance of the text matching model using the evaluation formula:

[0044] F1=(2*P*R) / (P+R);

[0045] P = TP / (TP + FN);

[0046] R = TP / (TP + FP);

[0047] Among them, F1 represents the performance score; P represents the precision; R represents the recall; TP represents that the actual intention is A and the predicted intention is A; FN represents that the actual intention is A and the predicted intention is not A; FP represents that the actual intention is not A and the predicted intention is A.

[0048] Step S42: determine whether the performance score is greater than a preset performance threshold. If so, proceed to step S50; if not, expand the text data set and proceed to step S30.

[0049] The advantages of the present invention are:

[0050] By acquiring marketing knowledge texts including triple data and text data, a text dataset is constructed. That is, feature capture is performed based on multiple data sources, and the similarities and differences of texts are compared from multiple dimensions to improve the robustness and generalization ability of the text matching model. By preprocessing the marketing knowledge texts, irrelevant data is prevented from interfering with the training of the text matching model. Different data structures of triple data and text data are vectorized separately to ensure the accuracy of feature extraction. Global correlation aggregation is performed through the Bi-LSTM network to overcome the long-range dependency problem of traditional recurrent neural networks. The performance of the text matching model is evaluated by combining precision and recall so that it can be further optimized, which ultimately greatly improves the accuracy and generalization ability of text matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] Figure 1 The present invention is a flowchart of a marketing knowledge text matching method based on multiple data sources.

[0053] Figure 2 It is a structural diagram of the BERT pre-training model of the present invention.

[0054] Figure 3 Schematic diagram of the structure of the Word2Vec model of the present invention.

[0055] Figure 4 Schematic diagram of the structure of the Bi-LSTM network of the present invention. DETAILED DESCRIPTION

[0056] The technical solution in the embodiments of the present application has the following overall ideas: feature capture based on multiple data sources to improve the robustness and generalization ability of the text matching model; preprocessing the marketing knowledge text to avoid interference from irrelevant data; vectorization based on different data structures of triple data and text data to ensure the accuracy of feature extraction; global correlation aggregation through the Bi-LSTM network to overcome the long-range dependency problem; and evaluating the performance of the text matching model by combining precision and recall rate so as to further optimize it, that is, using natural language processing technology and deep learning algorithms to complete the matching of marketing knowledge texts to achieve improved text matching accuracy and generalization ability.

[0057] Please refer to Figures 1 to 4 As shown, a preferred embodiment of the present invention is a marketing knowledge text matching method based on multiple data sources, comprising the following steps:

[0058] Step S10: Acquire a large amount of marketing knowledge text including triple data and text data;

[0059] Step S20: pre-processing each of the marketing knowledge texts to construct a text dataset;

[0060] Since the content of the marketing knowledge text is messy and not suitable for training data of the text matching model, some data may be irrelevant to subsequent analysis operations or may interfere with them. Therefore, preprocessing operations are required before training the text matching model.

[0061] Step S30: creating a text matching model based on multi-granularity and multi-features, and using the text dataset to train and optimize the text matching model;

[0062] Step S40: Evaluate the performance of the text matching model;

[0063] Step S50: Use the text matching model to perform similarity matching on the text data to be matched.

[0064] The step S10 is specifically as follows:

[0065] Obtaining a large amount of triple data of marketing knowledge from a graph database, extracting entity words and relation words from the triple data and concatenating them; the graph database is preferably Neo4J;

[0066] Obtaining large amounts of marketing knowledge text data from relational databases;

[0067] The triple data and the text data are merged to obtain marketing knowledge text.

[0068] The step S20 specifically includes:

[0069] Step S21: removing noise data from each of the marketing knowledge texts, which can effectively improve text processing efficiency;

[0070] Step S22: performing deduplication processing on each of the marketing knowledge texts;

[0071] Step S23: using a word segmentation tool to perform word segmentation on each of the marketing knowledge texts to obtain a plurality of phrases, and removing stop words from the phrases based on a preset stop word list;

[0072] Step S24: take each of the phrases as a positive sample, and select a preset number of phrases in different fields as negative samples for each of the positive samples, thereby constructing a text dataset; by constructing the positive samples and negative samples to train the text matching model, the generalization ability and discrimination ability can be effectively improved.

[0073] In step S21, the noise data at least includes a URL, a picture, an expression, an ID, special symbols, and a request time.

[0074] In step S30, the text matching model includes a vectorization module, a global relevance aggregation module and an output module; the input end of the global relevance aggregation module is connected to the vectorization module, and the output end is connected to the output module.

[0075] The vectorization module is used to vectorize the triple data and the text data to convert them into data that can be read by a computer. Since the triple data and the text data have different characteristics, they need to be vectorized separately;

[0076] The vectorization of triple data is specifically as follows:

[0077] Use the BERT pre-trained model to vectorize each metadata of the triple data to obtain the first word embedding of each metadata;

[0078] Create a parameter matrix and use it to map each first word embedding to a specified text length to obtain the second word embedding. The formula is:

[0079] O S =diag(S·W);

[0080] Use the softmax function to requantize the weight of the second Word Embedding to obtain the third Word Embedding. The formula is:

[0081] O S =softmax(O S );

[0082] After comparing and fusing the first word embedding with the third word embedding, a pooling operation is performed to obtain a triple vector, completing the vectorization of the triple data. The formula is:

[0083]

[0084] The triple data of marketing knowledge text consists of marketing keywords, marketing relationships, and marketing information, such as ("fusion package", "definition", "used for grade, promotion center marketing case, prize package, product configuration"); for triple data, if it is directly converted into a vector based on the BERT pre-trained model or Word2Vec model, it is impossible to extract the relationship characteristics between the keywords and marketing content in the triple data. It only represents their semantic information. In fact, relationship words are also one of the important information in the triple.

[0085] Since the traditional attention mechanism assigns different weights to enhance features based on contextual information and the relationships between different words in a sentence, but the traditional computing method focuses more on extracting the relationships between words, while the natural characteristics of the words themselves are not fully explored, its performance may be limited to a certain extent. Therefore, this application introduces a parameter matrix based on the characteristics of triple data to fully explore the unique relationship characteristics between the marketing relationship and the other two parts.

[0086] The softmax function is used to requantize the weights. That is, the Multi-Head Attention mechanism is used to assign weights to different words in a short text sequence, giving different words different weights and enhancing the accuracy of multi-intent recognition. Compared with other attention mechanisms, Multi-Head Attention can simultaneously obtain different words related to multiple intents, thereby obtaining more important word information from multiple angles.

[0087] The attention operation queries the mapping from Q to key-value pairs KV. Here, each attention head uses self-attention to find connections within the sequence. The calculation method of the Multi-Head Attention mechanism is as follows: First, Q, K, and V are linearly transformed; then they are input into the scaled dot product attention and calculated h times, each time as a head. The parameters between heads are not shared. Finally, the h scaled dot product attention results are spliced ​​from left to right and then linearly transformed to obtain the attention matrix X as the result of the multi-head attention. The formula is as follows:

[0088]

[0089] head i =Attention(QW i Q ,KW i K ,VW i V );

[0090]

[0091] Among them, Q, K, and V represent the Query matrix, Key matrix, and Value matrix respectively, and their values ​​are equal, and they are all vectorized output E; It refers to the square root of the dimension of the key vector, which plays a regulatory role and controls the inner product of Q and K from being too large; W is the parameter of the linear transformation, and W is different each time Q, K, and V undergo a linear transformation; h represents the number of heads, and i represents the i-th attention head.

[0092] The vectorization of text data is specifically as follows:

[0093] Based on word granularity, the text data is converted into BERT Embedding using the BERT pre-trained model;

[0094] Based on word granularity, the text data is converted into a fourth word embedding using the Word2Vec model;

[0095] The BERT Embedding and the fourth Word Embedding are fused together using average pooling and maximum pooling to obtain a text data vector, completing the vectorization of the text data. Since using a single pooling operation may ignore special values ​​in the features, the two pooling methods are combined to complement each other and the pooling results are spliced.

[0096] Traditional text feature extraction mainly relies on word granularity. Although it is relatively simple and has a fast calculation speed, the granularity considered is single. It only starts from the word granularity and misses the text interaction information at a higher granularity level, resulting in less than ideal text matching accuracy. Therefore, this application represents the text from the perspective of character granularity and word granularity.

[0097] Assume that a sentence pair is P = {p1, p2, ..., p m}, the final matrix V obtained after pooling p for:

[0098]

[0099]

[0100]

[0101] The essence of the BERT pre-training model (Bidirectional Encoder Representations from Transformers) is implemented on the basis of the bidirectional Transformer encoder, such as Figure 2 As shown, E1, E2, …, E N is the input vector of the BERT pre-training model. The vectors obtained by the bidirectional Transformer encoder are T1, T2, ..., T N .

[0102] The Word2Vec model uses the idea of ​​deep learning to simplify the processing of text content into vector operations in a multidimensional vector space through training, and expresses the similarity of text semantics through the similarity in the vector space; the Word2Vec model uses a distributed word vector representation method, which is essentially a neural probability model. There are two training methods: the CBOW model and the Skip-gram model. This application uses the Skip-gram model to train word vectors, such as Figure 3 shown.

[0103] The core idea of ​​the Skip-gram model is to predict W by k words in the context t The probability of occurrence is expressed mathematically as P(τ(W t-k ,W t-k+1 …W t-k-1 ,W t+k )W t ), the output layer of the network uses a hierarchical softmax function, and the objective optimization function of the two trainings is optimized by constructing a Huffman binary tree. The formula is as follows:

[0104]

[0105] Where C is the number of words in the short text.

[0106] The global correlation aggregation module uses a Bi-LSTM network to learn the mapping relationship between the vector sequence output by the matching vectorization module and the true matching degree;

[0107] The Bi-LSTM network selects cross entropy as the loss function and is trained by back propagation algorithm.

[0108] Since the obtained Semantic Vectors of the target text and the matching text only convey the interactive information of the texts and cannot effectively judge the degree of text matching, further processing is still required to obtain the correlation matching score. The lengths of the two matching degree vector sequences are not fixed. In order to map the variable-length sequences to a single-value matching degree, a Bi-LSTM network is used to learn the mapping relationship between the matching degree vector sequence and the actual matching degree.

[0109] LSTM is a carefully designed recurrent neural network that can effectively solve long-range dependency problems that general recurrent neural networks cannot overcome, such as Figure 4As shown in the figure, the key difference between LSTM and recurrent neural networks is that LSTM maintains the internal state of the network and continuously updates the internal state as the sequence inputs. In order to enhance the network's ability to remember information, especially the ability to remember long-range dependencies, whenever the output of the network at a previous moment is input to the network, LSTM does not directly affect the internal state of the network, but first filters it through a "forget gate" structure in order to retain more valuable information. Similarly, the input at this moment must also be filtered through the "input gate" first. The network then updates the network's state based on the internal state of the previous moment, combined with the filtered output of the previous moment and the input of this moment. Finally, the updated state of the network, combined with the filtered output of the previous moment and the input of this moment, jointly determines the output of this moment.

[0110] The output module is used to output the binary classification results through the sigmoid function. The formula is:

[0111] label=sigmoid(V final );

[0112] sigmoid=1 / (1+e -x ).

[0113] The step S40 specifically includes:

[0114] Step S41: Evaluate the performance of the text matching model using the evaluation formula:

[0115] F1=(2*P*R) / (P+R);

[0116] P = TP / (TP + FN);

[0117] R = TP / (TP + FP);

[0118] Among them, F1 represents the performance score (F1-score), and the higher the value, the better the recognition effect; P represents precision; R represents recall; TP represents that the actual intention is A and the predicted intention is A; FN represents that the actual intention is A and the predicted intention is not A; FP represents that the actual intention is not A and the predicted intention is A;

[0119] Predicted intention is A Predicted intention is not A The actual intention is A TP FN The actual intention is not A FP TN

[0120] Step S42: determine whether the performance score is greater than a preset performance threshold. If so, proceed to step S50; if not, expand the text data set and proceed to step S30.

[0121] In summary, the advantages of the present invention are:

[0122] By acquiring marketing knowledge texts including triple data and text data, a text dataset is constructed. That is, feature capture is performed based on multiple data sources, and the similarities and differences of texts are compared from multiple dimensions to improve the robustness and generalization ability of the text matching model. By preprocessing the marketing knowledge texts, irrelevant data is prevented from interfering with the training of the text matching model. Different data structures of triple data and text data are vectorized separately to ensure the accuracy of feature extraction. Global correlation aggregation is performed through the Bi-LSTM network to overcome the long-range dependency problem of traditional recurrent neural networks. The performance of the text matching model is evaluated by combining precision and recall so that it can be further optimized, which ultimately greatly improves the accuracy and generalization ability of text matching.

[0123] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A marketing knowledge text matching method based on multiple data sources, characterized by: The steps include: Step S10: obtaining a large amount of triple data of marketing knowledge from a graph database, extracting entity words and relation words from the triple data and concatenating them; obtaining a large amount of text data of marketing knowledge from a relational database; and merging the triple data and text data to obtain marketing knowledge text; Step S20: pre-processing each of the marketing knowledge texts to construct a text dataset; Step S30: creating a text matching model, and using the text dataset to train and tune the text matching model; the text matching model includes a vectorization module, a global relevance aggregation module, and an output module; the input end of the global relevance aggregation module is connected to the vectorization module, and the output end is connected to the output module; The vectorization module is used to vectorize triple data and text data; The vectorization of triple data is specifically as follows: Use the BERT pre-trained model to vectorize each metadata of the triple data to obtain the first word embedding of each metadata; Create a parameter matrix, and use the parameter matrix to map each first word embedding to a specified text length to obtain a second word embedding; Use the softmax function to requantize the weight of the second Word Embedding to obtain the third Word Embedding; After comparing and fusing the first word embedding and the third word embedding, a pooling operation is performed to obtain a triple vector, completing the vectorization of the triple data; The vectorization of text data is specifically as follows: Based on word granularity, the text data is converted into BERT Embedding using the BERT pre-trained model; Based on word granularity, the text data is converted into a fourth word embedding using the Word2Vec model; The BERT Embedding and the fourth Word Embedding are fused together using average pooling and maximum pooling to obtain a text data vector, completing the vectorization of the text data. The global correlation aggregation module uses a Bi-LSTM network to learn the mapping relationship between the vector sequence output by the matching vectorization module and the true matching degree; The Bi-LSTM network selects cross entropy as the loss function and is trained by the back propagation algorithm; The output module is used to output the binary classification result through the sigmoid function; Step S40: Evaluate the performance of the text matching model; Step S50: Use the text matching model to perform similarity matching on the text data to be matched.

2. The marketing knowledge text matching method based on multiple data sources according to claim 1, characterized in that: The step S20 specifically includes: Step S21: removing noise data of each marketing knowledge text including at least URL, picture, emoticon, ID, special symbols and request time; Step S22: performing deduplication processing on each of the marketing knowledge texts; Step S23: using a word segmentation tool to perform word segmentation on each of the marketing knowledge texts to obtain a plurality of phrases, and removing stop words from the phrases based on a preset stop word list; Step S24: taking each of the phrases as a positive sample, and selecting a preset number of phrases in different fields as negative samples for each of the positive samples, thereby constructing a text dataset.

3. The marketing knowledge text matching method based on multiple data sources according to claim 1, characterized in that: The step S40 specifically includes: Step S41: Evaluate the performance of the text matching model using the evaluation formula: ； ； ； in, represents the performance score; Indicates the accuracy rate; represents the recall rate; Indicates that the actual intention is A and the predicted intention is A; It means that the actual intention is A, but the predicted intention is not A; It means the actual intention is not A, but the predicted intention is A; Step S42: determine whether the performance score is greater than a preset performance threshold. If so, proceed to step S50; if not, expand the text data set and proceed to step S30.

Images