An intent type topic detection method and device based on transfer learning and a computer readable storage medium

Abstract

The present application relates to a kind of based on the intention type topic detection method, device and computer readable storage medium of migration learning, comprising: obtaining labeled native topic dataset and unlabeled derivative topic dataset and pre-processing;Balanced distribution self-adapting is used to feature adaptation is carried out to derivative topic data and native topic data;According to the topic propagation network of native topic and derivative topic, the network structure feature of user under corresponding topic is constructed using SDNE algorithm;According to the native topic data and derivative topic data after feature adaptation, and the structure feature of user under native topic and derivative topic, the intention type topic detection model is trained using migration learning;Intention type topic detection model is identified to derivative topic data by well-trained. The present application improves the performance and accuracy of model.

Description

Technical Field

[0001] This invention belongs to the field of social network technology applications, and in particular relates to an intent-based topic detection method, device, and computer-readable storage medium based on transfer learning. Background Technology

[0002] In today's social media era, users widely discuss and share various topics on online platforms, including some with specific intentions. Intentional topics refer to those discussed and shared on social media or other information dissemination platforms with a specific purpose or intent. These topics aim to mislead users, promote specific viewpoints, spread false information, or achieve other specific goals. Compared to ordinary discussion topics, intentional topics are characterized by concealment and deception, aiming to influence the audience's thinking, behavior, or beliefs. These intentional topics can cause inconvenience and distress to users and may also have a negative impact on social order and public safety.

[0003] As information spreads on social networks, a trending topic may spawn one or more derivative topics. These derivative topics develop from the original topic, typically by expanding, subdividing, or extending it. Derivative topics may explore certain aspects of the original topic more specifically and in greater depth, or they may address related but not core issues. Different users exhibit different behaviors in derivative and original topics due to factors such as interests, resulting in inconsistent data spatial distribution and domain adaptation issues.

[0004] In summary, existing intent-based topic detection methods suffer from the following technical problems: The scarcity of effective data within the topics. Intent-based topics often contain a large amount of irrelevant, anomalous, and noisy data, resulting in an imbalanced category distribution. This leads to uneven distribution of samples from different categories within the original and derived topics, preventing the model from simultaneously adapting to intent-based topic detection in both original and derived topics, thus resulting in low model performance and accuracy. Summary of the Invention

[0005] To address the problems existing in the background art, one aspect of the present invention provides an intent-based topic detection method based on transfer learning, comprising:

[0006] S1: Obtain labeled original topic datasets and unlabeled derived topic datasets and perform preprocessing;

[0007] S2: Adaptive feature matching between derived topic data and original topic data using balanced distribution;

[0008] S3: Construct the network structure features of users under the corresponding topic using the SDNE algorithm based on the topic propagation network of the original topic and the derived topic;

[0009] S4: Based on the original topic data and derived topic data after feature adaptation, as well as the network structure features of users under the original topic and derived topic, the intent-based topic detection model is trained using transfer learning;

[0010] S5: Identify derivative topic data using a trained intent-based topic detection model.

[0011] Preferably, the original topic data or derived topic data includes, but is not limited to: topic content text, user comment content text under the topic, and topic propagation network; the topic propagation network includes: the comment relationship network between users under the topic and the follow relationship network between users.

[0012] Preferably, step S3 includes: using the SDNE algorithm to perform representation learning on the comment relationship network and the attention relationship network respectively to extract the comment structure features W of user nodes. i Relationship structure features R with users i According to the user's structural characteristics W i Relationship structure features R with users i Obtain the user's network structure characteristics u i =μ1×W i +μ2×R i μ1 and μ2 represent the weight parameters.

[0013] Preferably, the preprocessing of the original topic data and the derived topic data includes: performing word segmentation and part-of-speech tagging on the topic content text and user comment text under the topic, removing words with useless parts of speech and stop words to obtain candidate keywords for the topic; using the TF-IDF algorithm to calculate the weight of each candidate keyword; extracting keywords with weights greater than a set threshold as the central keywords of the topic; obtaining the central keyword sequence of the original topic and the derived topic; and then using the Doc2vec algorithm to learn the central keyword sequence, the output feature vectors are: original topic content feature vector, derived topic content feature vector, original topic user content feature vector, and derived topic user content feature vector.

[0014] Preferably, the optimization objective of using balanced distribution adaptive feature matching to adapt derivative topic data and original topic data is expressed as:

[0015]

[0016] stA T XHX T A = I, 0 ≤ u ≤ 1

[0017] Where tr() represents the trace of the matrix, A represents the feature transformation matrix, X represents the combined data of the original topic and the derived topic; C represents the number of sample categories; λ represents the regularization parameter; H represents the center matrix; M c M0 represents the maximum mean difference matrix between classes; T represents the matrix transpose; u represents the balancing factor used to adjust the weights of marginal distribution differences and conditional distribution differences during the adaptation process; ||A|| F Let denote the Frobenius norm of matrix A; I denotes the identity matrix.

[0018] Preferably, the maximum mean difference matrix between the original topic and the derived topic includes:

[0019]

[0020] Among them, (M) o ) ij Let n represent the i-th row and j-th element of M0. s and n t D represents the number of samples for the original topic and the derived topic, respectively; s and D t These represent the original topic and the derived topic, respectively; x i and x j Let i and j represent the i-th and j-th data in X, respectively.

[0021] Preferably, the maximum mean difference matrix M c include:

[0022]

[0023] Among them, (M) c ) ij M represents c The element in the i-th row and j-th position, and ... and These represent the original topic and the derived topic of category c, respectively.

[0024] Preferably, the intent-based topic detection model includes a feature extractor, a label classifier, and a domain discriminator; the input features of the feature extractor are the original topic data or derived topic data after feature adaptation, and the network structure features of users under the original topic or derived topic; the output of the feature extractor is the extracted inter-domain invariant features of the original topic data and derived topic data; the domain discriminator determines whether it comes from the original topic or the derived topic through the extracted inter-domain invariant features; the label classifier classifies the extracted inter-domain invariant features to obtain the intent type of the topic.

[0025] Another aspect of the present invention provides an intent-based topic detection device based on transfer learning, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the intent-based topic detection device based on transfer learning performs the intent-based topic detection method based on transfer learning.

[0026] Another aspect of the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned intention-based topic detection method based on transfer learning.

[0027] The present invention has at least the following beneficial effects

[0028] This invention adapts features between derived topic data and original topic data through balanced distribution adaptation, reducing the marginal and conditional distribution differences between the two data. This enables the target model to accurately identify the topic intent type of both original and derived topics simultaneously. Furthermore, this invention extracts user network structure features based on the topic propagation network of both original and derived topic data, and performs transfer learning on the intent topic detection model using the feature-adapted original and derived topic data. This allows the intent topic recognition model to be applicable to the intent detection of both original and derived topics, improving the model's performance and accuracy. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0030] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0031] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0032] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0033] Please see Figure 1 One aspect of the present invention provides an intention-based topic detection method based on transfer learning, comprising:

[0034] S1: Obtain labeled original topic datasets and unlabeled derived topic datasets and perform preprocessing;

[0035] S2: Adaptive feature matching between derived topic data and original topic data using balanced distribution;

[0036] S3: Construct the network structure features of users under the corresponding topic using the SDNE algorithm based on the topic propagation network of the original topic and the derived topic;

[0037] S4: Based on the original topic data and derived topic data after feature adaptation, as well as the network structure features of users under the original topic and derived topic, the intent-based topic detection model is trained using transfer learning;

[0038] S5: Identify derivative topic data using a trained intent-based topic detection model.

[0039] In this embodiment, labeled native topic datasets and unlabeled derived topic datasets are obtained through the API interface provided by the social platform. The native topic data or derived topic data includes, but is not limited to: topic content text, user comment content text under the topic, and topic propagation network; the topic propagation network includes: comment relationship network between users under the topic and follow relationship network between users, etc., wherein the nodes in the comment relationship network are represented as user nodes, and the relationship between nodes indicates whether there is a comment relationship between users; the nodes in the follow relationship network are represented as user nodes, and the relationship between nodes indicates the friend relationship and / or follow relationship between users.

[0040] Preferably, the original topic data or derived topic data includes, but is not limited to: topic content text, user comment content text under the topic, and topic propagation network; the topic propagation network includes: the comment relationship network between users under the topic and the follow relationship network between users.

[0041] Preferably, step S3 includes: using the SDNE algorithm to perform representation learning on the comment relationship network and the attention relationship network respectively to extract the comment structure features W of user nodes. i Relationship structure features R with users i According to the user's structural characteristics W i Relationship structure features R with users i Obtain the user's network structure characteristics u i =μ1×W i +μ2×R i μ1 and μ2 represent the weight parameters.

[0042] In this embodiment, the SDNE algorithm is used to perform representation learning on the comment relationship network and the attention relationship network, respectively. SDNE uses a semi-supervised deep model with an opportunistic autoencoder for network embedding, where the input of the model is the adjacency matrix S of the i-th user node. i , S = {S1,S2,...,S} n Let} be the adjacency matrix of the network, and output the reconstructed adjacency matrix x. i By continuously reducing the error between the input and reconstructed matrices until it falls below a set threshold, the global structural features are obtained; the second-order similarity loss function of SDNE is defined as: Where ⊙ represents the Hadamarda product, If S i,j =0 then b i,j =1, otherwise b i,j =β>1. For the output features of each coding layer of the autoencoder For local features of node i; k = 1, 2, ..., K, where K represents the number of coding layers in the autoencoder; W (k) b represents the weight matrix of the k-th coding layer of the autoencoder; (k) The bias term of the k-th coding layer of the autoencoder; the first-order similarity loss function of SDNE is in, S represents the output representation of node i in the k-th coding layer; i,j This indicates whether nodes i and j are connected; a value of 1 indicates a connection, and a value of 0 indicates a connection. The minimum optimization objective function after the union is: L min =L 2nd +λL 1st +vL reg, where L reg Let L represent the regularization term, λ be the parameter controlling the first-order loss, and v be the parameter controlling the regularization term. Finally, train the model to obtain the minimum L. min Then, the output is used as the user's comment structure feature W. i Relationship structure features R with users i According to the user's structural characteristics W i Relationship structure features R with users i Obtain the user's network structure characteristics u i =μ1×W i +μ2×R i μ1 and μ2 represent weight parameters; the user's comment structure features W i Or the relational structure features of users R i It consists of the output representation of all coding layers of the autoencoder.

[0043] Preferably, the preprocessing of the original topic data and the derived topic data includes: performing word segmentation and part-of-speech tagging on the topic content text and user comment text under the topic, removing words with useless parts of speech and stop words to obtain candidate keywords for the topic; using the TF-IDF algorithm to calculate the weight of each candidate keyword; extracting keywords with weights greater than a set threshold as the central keywords of the topic; obtaining the central keyword sequence of the original topic and the derived topic; and then using the Doc2vec algorithm to learn the central keyword sequence, the output feature vectors are: original topic content feature vector, derived topic content feature vector, original topic user content feature vector, and derived topic user content feature vector.

[0044] Preferably, the optimization objective of using balanced distribution adaptive feature matching to adapt derivative topic data and original topic data is expressed as:

[0045]

[0046] stA T XHX T A = I, 0 ≤ u ≤ 1

[0047] Where tr() represents the trace of the matrix, A represents the feature transformation matrix, X represents the combined data of the original topic and the derived topic; C represents the number of sample categories; λ represents the regularization parameter; H represents the center matrix; M c M0 represents the maximum mean difference matrix between classes; T represents the matrix transpose; u represents the balancing factor used to adjust the weights of marginal distribution differences and conditional distribution differences during the adaptation process; ||A|| F Let denote the Frobenius norm of matrix A; I denotes the identity matrix.

[0048] In this embodiment, the balanced distribution adaptive representation is to find a transformation W that significantly reduces the differences in the marginal and conditional distributions of the original topic and derived topic data spaces after the transformation. The original topic content feature vector T is defined. S Derivative topic content feature vector T D Original topic user content feature vector U S and derivative topic user content feature vector U T Assume X S =X T Y S =Y T , P(X S )≠P(X T ), Q(Y S |X S )≠Q(Y T |X T We seek a transformation A that significantly reduces the differences in marginal and conditional distributions between the original and derived topic data spaces. Based on the feature vectors of topic content and user content obtained above, we assume the original topic data X... S =[T S U S [, Derivative topic X] T =[T T U T ], then the source domain D S ={X S ,P(X s )},X s ∈X S Among them, the source domain task is T. S ={Y s ,f(X s )},Y s ∈Y S f(X) s P(X) represents the classifier in the source domain and the classifier in the target domain. S ) represents the target domain D T ={X T ,P(X t )},X t ∈X T , P(X t ) represents the classifier over the target domain, which is defined by X. T Predict Y T .

[0049] By maximizing the divergence, a balance factor u is introduced, and the weights of marginal distribution differences and conditional distribution differences during the adaptation process are adjusted by continuously modifying the value of u. The final optimization objective is obtained by simplifying the process using a kernel function method.

[0050]

[0051] stA T XHX T A = I, 0 ≤ u ≤ 1

[0052] Finally, combining the two distances yields a total optimization objective:

[0053]

[0054] Where Φ is a Lagrange multiplier.

[0055] Preferably, the maximum mean difference matrix between the original topic and the derived topic includes:

[0056]

[0057] Among them, (M) o ) ij Let n represent the i-th row and j-th element of M0. s and n t D represents the number of samples for the original topic and the derived topic, respectively; s and D t These represent the original topic and the derived topic, respectively; x i and x j Let i and j represent the i-th and j-th data in X, respectively.

[0058] Preferably, the maximum mean difference matrix M c include:

[0059]

[0060] Among them, (M) c ) ij M represents c The element in the i-th row and j-th position, and ... and These represent the original topic and the derived topic of category c, respectively.

[0061] Preferably, the intent-based topic detection model includes a feature extractor, a label classifier, and a domain discriminator; the input features of the feature extractor are the original topic data or derived topic data after feature adaptation, and the network structure features of users under the original topic or derived topic; the output of the feature extractor is the extracted inter-domain invariant features of the original topic data and derived topic data; the domain discriminator determines whether it comes from the original topic or the derived topic through the extracted inter-domain invariant features; the label classifier classifies the extracted inter-domain invariant features to obtain the intent type of the topic.

[0062] In this embodiment, the intent-based topic detection model includes a feature extractor, a label classifier, and a domain discriminator; let f be the original topic-related features input to the feature extractor. i s, the original topic's tag is Among them, feature f i s This includes: the original topic data after feature adaptation and the network structure features of users under the original topic; similarly, the derived topic-related features input to the feature extractor are f. i T , where feature f i s This includes: derived topic data after feature adaptation and the network structure characteristics of users under derived topics;

[0063] Feature extractors, label classifiers, and domain discriminators are all multi-layered composite functions. For a feature extractor, its output is represented as:

[0064] G f (x i ;w,b)=sigmoid(wx i +b)

[0065] Where, x i Let w represent the i-th input sample, w represent the weight term of the feature extractor, b represent the bias term of the feature extractor, and sigmoid() represent the activation function; G f (x i w,b) represents the extracted inter-domain invariant features;

[0066] For a label classifier, its output is represented as:

[0067] G y (G f (x i ),v,c)=sigmoid(vG f (x i )+c)

[0068] Where v represents the weight term of the label classifier, and c represents the bias term of the label classifier; G y(G f (x i (), v, c) represent the predicted labels; the loss L of the label classifier y (G y (G f (x i )),y i (abbreviated as L) y ) is represented as:

[0069] L y (G y (G f (x i )),y i )=-logG y (G f (x i ))×y i

[0070] Among them, y i Indicates the true label of the sample;

[0071] The domain discriminator and the label classifier share input features, and the output is the sample source, i.e., whether the sample originates from the original topic or a derived topic. The domain discriminator loss is:

[0072] L d (G d (G f (x i )),d i )=-d i logG d (G f (x i )-(1-d i )logG d (G f (x i )

[0073] Where, d i Let L be the domain class label for the i-th sample, and let L be the domain discriminator loss. d (G d (G f (x i )),d i (abbreviated as L) d The feature extractor minimizes the label classifier error, continuously improving the classification accuracy of the features. Simultaneously, it maximizes the classification error of the domain discriminator, ensuring the features possess domain invariance. To guarantee the minimization of the overall loss function, a gradient inversion layer is added between the feature extractor and the domain discriminator. This layer remains unchanged during forward propagation and is automatically inverted during backward propagation. Therefore, the total loss function L = L... y -ηLd η represents the regularization parameter of the adversarial transfer learning model. The label classification loss and domain classification loss are co-optimized in the feature extractor to find domain-invariant features that allow the trained model on the original topic to transfer to derived topics, predicting the intent of the derived topics.

[0074] In summary, this invention adapts features between derived topic data and original topic data through balanced distribution adaptation, thereby reducing the marginal and conditional distribution differences between the original and derived topic data. This enables the target model to accurately identify the topic intent type of both original and derived topics simultaneously. Furthermore, this invention extracts user network structure features based on the topic propagation network of both original and derived topic data, and performs transfer learning on the intent topic detection model using the feature-adapted original and derived topic data. This allows the intent topic recognition model to be applicable to the intent detection of both original and derived topics, improving the model's performance and accuracy.

[0075] Another aspect of the present invention provides an intent-based topic detection device based on transfer learning, comprising a processor and a memory; the memory is used to store a computer program; the processor is connected to the memory and is used to execute the computer program stored in the memory, so that the intent-based topic detection device based on transfer learning performs the intent-based topic detection method based on transfer learning.

[0076] Another aspect of the present invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the aforementioned intention-based topic detection method based on transfer learning.

[0077] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An intent-based topic detection method based on transfer learning, characterized in that, include: S1: Obtain labeled original topic datasets and unlabeled derived topic datasets and perform preprocessing; S2: Adaptive feature matching between derived topic data and original topic data using balanced distribution; The optimization objective of adaptively matching features between derived topic data and original topic data using a balanced distribution is expressed as: in, Represents the trace of a matrix. This represents the feature transformation matrix, where X represents the combined data of the original topic and the derived topic. Indicates the number of categories in the sample; Indicates the regularization parameter; Represents the central matrix; This represents the matrix showing the maximum mean difference between classes. This represents the maximum mean difference matrix between the original topic and the derived topic; T represents the matrix transpose. This represents the balance factor, used to adjust the weights of marginal distribution differences and conditional distribution differences during the adaptation process; Representation matrix The Frobenius norm; Represents the identity matrix; The maximum mean difference matrix between the original topic and the derived topic includes: in, express The Line 1 One element, and These represent the number of samples for the original topic and the derived topic, respectively. and These represent the original topic and the derived topic, respectively. and They represent The first in The data and the first One data point; The maximum mean difference matrix include: in, express The Line number One element, and ... and These represent the original topic and the derived topic of category c, respectively. S3: Construct the network structure features of users under the corresponding topic using the SDNE algorithm based on the topic propagation network of the original topic and the derivative topic; S4: Based on the original topic data and derived topic data after feature adaptation, as well as the network structure features of users under the original topic and derived topic, the intent-based topic detection model is trained using transfer learning; S5: Identify derivative topic data using a trained intent-based topic detection model.

2. The intention-based topic detection method according to claim 1, characterized in that, The original topic data or derived topic data includes, but is not limited to: topic content text, user comment content text under the topic, and topic propagation network; The topic propagation network includes: the comment relationship network among users under the topic and the follow relationship network among users.

3. The intention-based topic detection method according to claim 2, characterized in that, Step S3 includes: using the SDNE algorithm to perform representation learning on the comment relationship network and the attention relationship network respectively to extract the comment structure features of user nodes. Relationship structure features with users Based on the user's structural characteristics Relationship structure features with users Obtain the user's network structure characteristics , and This represents the weighting parameter.

4. The intention-based topic detection method according to claim 2, characterized in that, Preprocessing of the original and derived topic data includes: segmenting and tagging the topic content text and user comments under the topics into words and parts of speech, removing words with useless parts of speech and stop words to obtain candidate keywords for the topics, using the TF-IDF algorithm to calculate the weight of each candidate keyword, extracting keywords with weights greater than a set threshold as the central keywords of the topics, obtaining the central keyword sequences of the original and derived topics, and then using the Doc2vec algorithm to learn the central keyword sequences, outputting feature vectors as: original topic content feature vector, derived topic content feature vector, original topic user content feature vector, and derived topic user content feature vector.

5. The intention-based topic detection method according to claim 1, characterized in that, The intent-based topic detection model includes a feature extractor, a label classifier, and a domain discriminator. The input features of the feature extractor are the original topic data or derived topic data after feature adaptation, as well as the structural features of users under the original topic or derived topic. The extractor outputs the inter-domain invariant features of the extracted original topic data and derived topic data; The domain discriminator determines whether a topic originates from a primary or secondary topic by extracting inter-domain invariant features; the label classifier classifies the extracted inter-domain invariant features to obtain the intent type of the topic.

6. An intent-based topic detection device based on transfer learning, characterized in that, It includes a processor and a memory; the memory is used to store computer programs; the processor is connected to the memory and is used to execute the computer programs stored in the memory, so that the intent-based topic detection device based on transfer learning performs the intent-based topic detection method according to any one of claims 1-5.

7. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the intent-based topic detection method according to any one of claims 1-5.

Images