Text topic determination method and device and storage medium

Abstract

The embodiment of the invention provides a text topic determination method and device and a storage medium, which can be applied to the technical field of natural language processing, in any segmented word in the method, according to the transmission weight of the segmented word and the underlying topic in a segmented word transmission matrix, a corresponding underlying topic vector when the segmented word vector of the segmented word is transmitted to the underlying topic is determined, and the underlying topic vector of the segmented word is transmitted to the underlying topic. A bottom-layer topic vector is used as an embedding anchor, the transmission direction of a plurality of segmented words is guided, key information is highlighted, and generation of a coherent fine-grained topic is facilitated; and for any (i-1) th layer of theme, according to the transmission weights of the (i-1) th layer of theme and the ith layer of theme in the theme transmission matrix, determining the corresponding ith layer of theme vector when the (i-1) th layer of theme vector of the segmented word is transmitted to the ith layer of theme, guiding the low-layer theme to be transmitted to the high-layer theme among different layers of themes, and realizing alignment of cross-layer themes, therefore, all layers of themes are kept consistent and hierarchical progressive semantically, and a hierarchical theme structure with interpretability and coverage is obtained.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, device and storage medium for determining text topics. Background Technology

[0002] With the advancement of digital transformation across various industries, customer service demands are rapidly increasing, making intelligent customer service systems a key tool for improving customer experience and operational efficiency. In practical applications, a hierarchical topic model is used to extract multi-level topics from texts across various professional fields. These extracted topics, along with their corresponding texts, are stored in a multi-level topic knowledge base. This multi-level topic approach enhances the knowledge retrieval and personalized response capabilities of the intelligent customer service system.

[0003] In related technologies, hierarchical topic models based on neural networks are often used to determine multi-level topics in text. However, most existing hierarchical topic models based on neural networks rely on unsupervised learning and lack the guidance of external semantic instructions. This can easily generate topics that are too semantically scattered or redundant, resulting in either too much or too little differentiation between topics. It is difficult to maintain a relatively balanced scale and reduces the accuracy of text topic determination. Summary of the Invention

[0004] This invention provides a method, device, and storage medium for determining text topics, which improves the accuracy of text topic determination.

[0005] On the one hand, embodiments of this application provide a method for determining text topics, the method comprising: Determine the segmentation vectors of each of the multiple words in the text to be processed; For any word segment, the underlying topic vector corresponding to the word segment vector when it is transmitted to the underlying topic is determined according to the transmission weight of the word segment and the underlying topic in the word segmentation transmission matrix; the underlying topic is the first-level topic; any transmission weight represents the transmission amount; For any (i-1)th layer topic, the i-th layer topic vector corresponding to the (i-1)th layer topic vector transmitted to the i-th layer topic is determined according to the transmission weights of the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix; i is an integer greater than or equal to 2. Based on the topic vector of each layer and the word segmentation vectors of the multiple words, the distribution matrix of each topic and the multiple words is inferred; Based on the distribution matrix of each topic and multiple word segments, the topic tags of the text to be processed in each topic are determined.

[0006] Optionally, the word segmentation transfer matrix is ​​trained in the following ways: Determine the training word vectors for each of the multiple training word segments in the training text; Determine the first distance from any trained word segmentation vector to the corresponding underlying topic vector of each underlying topic; Based on the first distance, the transmission weights of the training word and the underlying topic in the word segmentation transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same training word and multiple underlying topics is a first fixed value; the first fixed value is used to control the amount of transmission allowed for each training word to multiple underlying topics.

[0007] Optionally, the transmission weights between topics at layer i-1 and layer i in the topic transmission matrix are trained in the following manner, including: Determine the second distance from the (i-1)th layer topic vector corresponding to any (i-1)th layer topic to the i-th layer topic vector corresponding to each i-th layer topic; Based on the second distance, the transmission weights between the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same (i-1)th layer topic and multiple i-th layer topics is a second fixed value, which is used to control the amount of transmission allowed from each (i-1)th layer topic to the i-th layer topic.

[0008] Optionally, during training, a first transmission parameter is set to control the transmission smoothness between multiple training word segmentation vectors and multiple underlying topic vectors; a second transmission parameter is set to control the transmission smoothness between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

[0009] Optionally, until training requirements are met, including: The first transmission cost of the training process is determined based on the updated word segmentation transmission matrix, and the second transmission cost of the training process is determined based on the updated topic transmission matrix. Based on the first transmission cost and L-1 second transmission costs, the target loss value is determined; When the target loss value meets the training requirements, the word segmentation transfer matrix and L-1 topic transfer matrices are output; where L is the total number of levels of the multi-layer topics.

[0010] Optionally, determining the first transmission cost of the training process based on the updated word segmentation transmission matrix includes: Based on the first distance between multiple training word segmentation vectors and each underlying topic vector of the training text and the updated word segmentation transfer matrix, the first transfer cost between the multiple training word segmentation vectors and the multiple underlying topic vectors is determined. The determination of the second transmission cost of the training process based on the updated topic transfer matrix includes: Based on the second distance between multiple (i-1)th layer topic vectors and each i-th layer topic vector, and the updated topic transfer matrix, the second transfer cost of the training text between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors is determined.

[0011] Optionally, determining the target loss value based on the first transmission cost and L-1 second transmission costs includes: Based on the topic vector of each layer and multiple training word segmentation vectors, the distribution matrix of each topic and multiple training word segmentation is inferred; Based on the distribution matrix of each topic layer and multiple training words, the multiple training words, and the training text, the distribution matrix of the training text and each topic layer is inferred. Based on the distribution matrix of the training text and each layer of topics, and the distribution matrix of each layer of topics and multiple training word segments, the document is reconstructed to obtain the reconstruction loss value. The target loss value is determined based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0012] Optionally, determining the target loss value based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value includes: The divergence loss value is determined based on the difference between the distribution matrix of the training text and the topic prior matrix of each layer; wherein, the topic prior matrix is ​​the distribution template of the training text and the topic determined by Gaussian distribution. The target loss value is determined based on the divergence loss value, the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0013] Optionally, determining the topic tags of the text to be processed in each topic layer based on the distribution matrix of each topic layer and multiple word segments includes: From the distribution matrix of the underlying topic and multiple word segments, determine at least one word segment with a distribution probability greater than a first threshold under any underlying topic, and based on the at least one word segment, summarize the topic tag of the underlying topic; From the distribution matrix of the i-th layer topic and multiple word segments, determine at least one word segment with a distribution probability greater than the second threshold under any i-th layer topic, and based on the at least one word segment, summarize the topic tag of the i-th layer topic.

[0014] On one hand, embodiments of this application provide a text topic determination device, the device comprising: The word segmentation vector module is used to determine the word segmentation vectors of multiple words in the text to be processed. The underlying topic module is used to determine the underlying topic vector corresponding to the word segmentation vector when it is transmitted to the underlying topic, based on the transmission weights of the word segmentation and the underlying topic in the word segmentation transmission matrix, for any word segmentation; the underlying topic is the first-level topic; any transmission weight represents the transmission amount; The high-level topic module is used to determine the i-th level topic vector corresponding to the (i-1)th level topic vector of the word segmentation when it is transmitted to the i-th level topic, according to the transmission weights of the (i-1)th level topic and the i-th level topic in the topic transmission matrix; i is an integer greater than or equal to 2. The distribution matrix module is used to infer the distribution matrix of each topic and multiple words based on the topic vector of each layer and the word vector of the multiple words; The topic tag module is used to determine the topic tags of the text to be processed in each topic based on the distribution matrix of each topic and multiple word segments.

[0015] Optionally, the underlying theme module is specifically used for: Determine the training word vectors for each of the multiple training word segments in the training text; Determine the first distance from any trained word segmentation vector to the corresponding underlying topic vector of each underlying topic; Based on the first distance, the transmission weights of the training word and the underlying topic in the word segmentation transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same training word and multiple underlying topics is a first fixed value; the first fixed value is used to control the amount of transmission allowed for each training word to multiple underlying topics.

[0016] Optionally, the high-level topic module is specifically used for: Determine the second distance from the (i-1)th layer topic vector corresponding to any (i-1)th layer topic to the i-th layer topic vector corresponding to each i-th layer topic; Based on the second distance, the transmission weights between the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same (i-1)th layer topic and multiple i-th layer topics is a second fixed value, which is used to control the amount of transmission allowed from each (i-1)th layer topic to the i-th layer topic.

[0017] Optionally, during training, a first transmission parameter is set to control the transmission smoothness between multiple training word segmentation vectors and multiple underlying topic vectors; a second transmission parameter is set to control the transmission smoothness between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

[0018] Optionally, the distribution matrix module is specifically used for: The first transmission cost of the training process is determined based on the updated word segmentation transmission matrix, and the second transmission cost of the training process is determined based on the updated topic transmission matrix. Based on the first transmission cost and L-1 second transmission costs, the target loss value is determined; When the target loss value meets the training requirements, the word segmentation transfer matrix and L-1 topic transfer matrices are output; where L is the total number of levels of the multi-layer topics.

[0019] Optionally, the distribution matrix module is specifically used for: Based on the first distance between multiple training word segmentation vectors and each underlying topic vector of the training text and the updated word segmentation transfer matrix, the first transfer cost between the multiple training word segmentation vectors and the multiple underlying topic vectors is determined. The determination of the second transmission cost of the training process based on the updated topic transfer matrix includes: Based on the second distance between multiple (i-1)th layer topic vectors and each i-th layer topic vector, and the updated topic transfer matrix, the second transfer cost of the training text between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors is determined.

[0020] Optionally, the distribution matrix module is specifically used for: Based on the topic vector of each layer and multiple training word segmentation vectors, the distribution matrix of each topic and multiple training word segmentation is inferred; Based on the distribution matrix of each topic layer and multiple training words, the multiple training words, and the training text, the distribution matrix of the training text and each topic layer is inferred. Based on the distribution matrix of the training text and each layer of topics, and the distribution matrix of each layer of topics and multiple training word segments, the document is reconstructed to obtain the reconstruction loss value. The target loss value is determined based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0021] Optionally, the distribution matrix module is specifically used for: The divergence loss value is determined based on the difference between the distribution matrix of the training text and the topic prior matrix of each layer; wherein, the topic prior matrix is ​​the distribution template of the training text and the topic determined by Gaussian distribution. The target loss value is determined based on the divergence loss value, the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0022] Optionally, the topic tag module is specifically used for: From the distribution matrix of the underlying topic and multiple word segments, determine at least one word segment with a distribution probability greater than a first threshold under any underlying topic, and based on the at least one word segment, summarize the topic tag of the underlying topic; From the distribution matrix of the i-th layer topic and multiple word segments, determine at least one word segment with a distribution probability greater than the second threshold under any i-th layer topic, and based on the at least one word segment, summarize the topic tag of the i-th layer topic.

[0023] On one hand, embodiments of this application provide a computer device, including: Memory, used to store program instructions; The processor is used to call program instructions stored in the memory and execute the steps of the above-described text topic determination method according to the obtained program.

[0024] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer to perform the steps of the above-described text topic determination method.

[0025] On one hand, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described text topic determination method.

[0026] In this embodiment, the segmentation vectors of multiple words in the text to be processed are determined. For any word, the underlying topic vector corresponding to the segmentation vector when it is transmitted to the underlying topic is determined according to the transmission weight of the word and the underlying topic in the segmentation transmission matrix. The underlying topic is the first-level topic. Each transmission weight represents the transmission amount, which is equivalent to using the underlying topic vector as an embedding anchor to guide the potential transmission direction of multiple words, suppress the dominant effect of high-frequency general words such as "method" and "research", highlight key information, and facilitate the generation of coherent fine-grained topics.

[0027] For any (i-1)th level topic, the i-th level topic vector corresponding to the (i-1)th level topic vector transmitted to the i-th level topic is determined according to the transmission weights of the (i-1)th level topic and the i-th level topic in the topic transmission matrix. i is an integer greater than or equal to 2. Between different levels of topics, the transmission of lower-level topics to higher-level topics is guided to achieve cross-level topic alignment, so that the topics of each level maintain semantic consistency and hierarchical progression. Thus, based on the topic vector of each level and the word vectors of multiple words, the distribution matrix of each level of topics and multiple words can be inferred, and the topic tags of the text to be processed in each level of topics can be determined, resulting in a hierarchical topic structure with good interpretability and coverage. This alleviates the problem of difficulty in grasping the distinguishability between topics of the same level and the loose structure between topics of different levels in unsupervised modeling. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic diagram of a system architecture provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for determining a text topic provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a text topic determination device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0031] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0032] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0033] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0034] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0036] The following explains the technical terms used in the embodiments of this application: Latent Dirichlet Allocation (LDA): A classic generative topic model that assumes each document is a mixture of multiple topics, each represented by a word distribution.

[0037] Hierarchical Topic Model (HTM): An extension of LDA, allowing topics to have a multi-level structure, from macro topics to micro topics, expressing hierarchical semantics.

[0038] Topic Collapse: Semantic redundancy and insufficient differentiation between lower-level topics lead to a blurred hierarchical structure.

[0039] Unsupervised learning: Model training does not rely on human labels, but discovers patterns through the distribution of the data itself.

[0040] Embedding: Mapping discrete objects (such as words or topics) to a continuous vector space to facilitate the calculation of semantic distance.

[0041] Entropy-regularized Optimal Transport: Entropy regularization is added when calculating the optimal transport (OT) to make the transport matrix sparse and easy to solve.

[0042] Sinkhorn Algorithm: An iterative algorithm for efficiently solving the entropy-regularized optimal transport problem.

[0043] VAE (Variational Autoencoder): A generative model used to reconstruct data from latent variables while performing probabilistic inference.

[0044] KL Divergence (KL divergence / Kullback-Leibler divergence): A metric that measures the difference between two probability distributions. In VAE, it is used to measure the difference between the prior distribution and the variational distribution.

[0045] The following is a brief introduction to the system architecture diagram applicable to the technical solutions of the embodiments of this application. It should be noted that the process described below is only for illustrating the embodiments of this application and is not intended to limit the scope of the application.

[0046] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least a terminal device 101 and a server 102. The number of terminal devices 101 can be one or more, and the number of servers 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and servers 102.

[0047] Terminal device 101 has an application pre-installed with a defined text theme. This application can be a client application, a web application, a mini-program application, etc. Terminal device 101 can be a smartphone, tablet, laptop, desktop computer, smart home appliance, smart voice interaction device, smart in-vehicle device, etc., but is not limited to these.

[0048] Server 102 is the backend server of the application. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, but it is not limited to these.

[0049] It should be noted that the method in the embodiments of this application can be executed independently by the terminal device 101 or the server 102, or jointly executed by the terminal device 101 and the server 102.

[0050] In the embodiments of this application, the terminal device 101 and the server 102 can be directly or indirectly communicatively connected through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and the embodiments of this application do not limit this.

[0051] Based on Figure 1 the system architecture diagram shown below, the embodiments of this application provide a process of a text topic determination method. The process of this method can be executed by Figure 1 the terminal device 101 shown below, or by the server 102, or by the interaction between the terminal device 101 and the server 102. As Figure 2 shown below, it includes the following steps: Step 201, determine the respective token vectors of multiple tokens in the text to be processed.

[0052] Specifically, eliminate invalid words such as "de" and "děi" in the text to be processed, only retain valid words, then remove duplicates from all valid words to obtain multiple tokens, and map each token to a continuous vector space, that is, encode it as a token vector, which can also be called token embedding; the text to be processed can be one or multiple, and this application does not make specific limitations on this.

[0053] Step 202, for any token, determine the corresponding underlying topic vector when the token vector of the token is transmitted to the underlying topic according to the transmission weight between the token and the underlying topic in the token transmission matrix.

[0054] In the embodiments of this application, the underlying topic is the first layer of topics from bottom to top in the topic tree, that is, the topic with the finest granularity; any transmission weight represents the amount of transmission.

[0055] Specifically, the token transmission matrix is the data transmission rule between multiple training tokens and multiple underlying topics output in the training process; each element in the token transmission matrix is used to guide how much of its own information a training token vector transmits to the underlying topic vector of an underlying topic. For example, a training token vector has information in N dimensions, and the transmission weight guides how many dimensions of information this training token vector can transmit to any underlying topic.

[0056] In some embodiments, the training word vectors of multiple training words in the training text are determined; a first distance is determined from any training word vector to the underlying topic vector corresponding to each underlying topic; based on the first distance, the transmission weights of the training words and the underlying topics in the word transmission matrix are updated until the training requirements are met.

[0057] In this embodiment of the application, the sum of the transmission weights of the same training word and multiple underlying topics is a first fixed value; the first fixed value is used to control the amount of transmission allowed for each training word to multiple underlying topics.

[0058] Specifically, the first distance from any training word segmentation vector to the underlying topic vector corresponding to each underlying topic can be obtained by calculating the squared Euclidean distance between the training word segmentation vector and the underlying topic vector. The specific calculation formula is shown in the following formula (1): (1) in, This represents the j-th trained word segmentation vector; This represents the k-th underlying topic vector; This represents the squared Euclidean distance between the j-th training word segmentation vector and the k-th underlying topic vector.

[0059] Alternatively, the first distance can be calculated by measuring the cosine distance, standardized Euclidean distance, or Mahalanobis distance between the trained word segmentation vector and the underlying topic vector; however, this application does not impose specific limitations on this.

[0060] Furthermore, before transmission, discrete measures are defined for the training word segmentation vectors and the underlying topic vectors, respectively, to quantify the qualitative relationship between the training word segmentation vectors and the underlying topic vectors. The discrete measure determined for the training word segmentation vectors is shown in the following formula (2): (2) in, Indicates the number of training word segments; This indicates that the word segmentation vectors are trained. The Dirac unit mass centered on; The discrete measure representing the training word segmentation vectors, that is, used to constrain each training word segmentation vector to only transmit its own information. For example, if the training word segmentation vector is a 16-dimensional vector, and there are a total of 4 training word segmentation vectors, then each training word segmentation vector can only incorporate information from its own 16 dimensions. Information from each dimension is transmitted to the underlying topic vector.

[0061] The discrete measure determined for multiple underlying topic vectors is shown in the following formula (3): (3) in, Indicates the number of bottom-level topics (first-level topics); This indicates the total amount of data transmitted from multiple training word segmentation vectors to the k-th underlying topic vector; Represents the underlying topic vector Dirac unit mass centered on; It represents the discrete measure of the underlying topic vector, that is, the amount of transmission of multiple training segments received by each underlying topic vector.

[0062] Based on the first distance between each training word segmentation vector and each underlying topic vector, the transmission weights of the training word segmentation vector and the underlying topic vector in the word segmentation transmission matrix are adjusted accordingly. When the first distance is too large, the transmission weights are appropriately reduced.

[0063] Furthermore, based on the above formula (3), the transmission weights of the multiple training word segmentation vectors received by the multiple underlying topic vectors are a single weight vector. , This indicates that the first underlying topic vector receives the sum of the transmission weights from multiple trained word segmentation vectors. Indicates the first The last (i.e., the last) bottom-level topic vector receives the sum of the transmission weights of multiple training word segmentation vectors, and the sum of all elements in this weight vector is 1. It should be noted that before the information of the training word segmentation vectors themselves is transmitted to the bottom-level topics, the bottom-level topics are simply symbols without semantic information or other meaning. Therefore, the first transmission between the multiple training word segmentation vectors and the multiple bottom-level topics is performed according to the initialized word segmentation transmission matrix. In the initialized word segmentation transmission matrix, the transmission weights received by each bottom-level topic vector from the multiple training word segmentation vectors are... ,Right now .

[0064] For example, the initial segmentation transfer matrix between the four trained segmentation vectors and the two underlying topics is shown in Table 1: Table 1

[0065] Table 1 shows the training word segmentation vectors. The transmission between the two underlying topics is respectively This indicates that the word segmentation vectors were trained. Able to transmit itself The dimensional information is transmitted to two underlying topics; and the underlying topics The sum of the transfer weights with the four trained word segmentation vectors is ; Underlying theme The sum of the transfer weights of the four trained word segmentation vectors is also... .

[0066] In this embodiment of the application, by setting discrete measures for the training word segmentation and the underlying topic respectively, the transmission volume of the training word segmentation itself and the received transmission volume of the underlying topic can be initially constrained, which can avoid excessive bias towards certain underlying topics at the beginning of transmission and ensure the fairness of transmission.

[0067] Step 203: For any (i-1)th layer topic, determine the i-th layer topic vector corresponding to the (i-1)th layer topic vector transmitted to the i-th layer topic according to the transmission weights of the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix.

[0068] Specifically, i is an integer greater than or equal to 2. The topic model in this technical solution is a hierarchical topic model, meaning the topics are a multi-level topic tree, with the entire topic tree progressing from macro-level topics to micro-level topics from top to bottom. The number of levels in the topic tree and the number of topics at each level are pre-defined by hyperparameters in this hierarchical topic model. If the total number of levels in the topic tree is L, then... The i-1th layer topic can represent any layer topic from the bottom layer topic to the L-1 layer topic.

[0069] In some embodiments, a second distance is determined between the (i-1)th layer topic vector corresponding to any (i-1)th layer topic and the i-th layer topic vector corresponding to each i-th layer topic; based on the second distance, the transmission weights between the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix are updated until the training requirements are met.

[0070] In this embodiment of the application, the sum of the transmission weights of the same (i-1)th layer topic and multiple i-th layer topics is a second fixed value. The second fixed value is used to control the amount of transmission allowed from each (i-1)th layer topic to the i-th layer topic.

[0071] Specifically, the second distance is calculated by taking the squared Euclidean distance between the topic vectors of the (i-1)th layer and the topic vectors of the ith layer. The specific calculation formula is shown in the following formula (4): (4) in, This represents the k-th (i-1)-th layer topic vector; Indicates the first The i-th layer topic vector; Represents the relationship between the k-th (i-1)th layer topic vector and the... The second distance between the i-th layer topic vectors can also be calculated, such as the cosine distance, standardized Euclidean distance, or Mahalanobis distance between the (i-1)-th layer topic vectors and the i-th layer topic vectors; however, this application does not impose specific limitations on this.

[0072] Furthermore, before transmission, discrete measures are defined for the (i-1)th layer topic vector and the i-th layer topic vector, respectively, to quantify the qualitative relationship between the (i-1)th layer topic vector and the i-th layer topic vector. The discrete measure determined for the (i-1)th layer topic vector is shown in the following formula (5): (5) in, This represents the total number of topics at level i-1; This represents the k-th (i-1)-th level topic vector. Dirac unit mass centered on; The discrete measure representing the topic vectors of the (i-1)th layer is used to constrain each (i-1)th layer topic vector to only transmit its own information. For example, if the topic vector of the (i-1)th layer is a vector with 16 dimensions, and there are a total of 4 topic vectors of the (i-1)th layer, then each topic vector of the (i-1)th layer can only contain information from its own 16 dimensions. Information from each dimension is transmitted to the i-th layer topic vector.

[0073] The discrete measure determined for multiple i-th layer topic vectors is shown in the following formula (6): (6) in, This represents the total number of topics in the i-th layer; Indicates the first Each layer i-th topic vector receives the total transmission from multiple layer i-1 topic vectors; Represents the topic vector of the i-th layer Dirac unit mass centered on; This represents the discrete measure of the i-th layer topic vector, which is used to constrain the transmission volume of multiple training segments received for each i-th layer topic vector.

[0074] Furthermore, based on the above formula (6), the transmission weights of the multiple (i-1)th layer topic vectors received by the multiple i-th layer topic vectors are a weight vector. Furthermore, the sum of all elements in this weight vector is 1. It should be noted that before the information of the (i-1)th layer topic vector itself is transmitted to the i-th layer topic, the i-th layer topic is a symbol without semantic information or other meaning. Therefore, the initial transmission between multiple (i-1)th layer topic vectors and the i-th layer topic is performed according to the initialized topic transmission matrix. In the initialized topic transmission matrix, the transmission weight of each i-th layer topic receiving multiple (i-1)th layer topic vectors is... ,Right now .

[0075] For example, the initial word segmentation transfer matrix between the four (i-1)th layer topic vectors and the two i-th layer topics is shown in Table 2: Table 2

[0076] In Table 1, the topic vectors of the (i-1)th layer The transmission weights of the two i-th layer topics are respectively This indicates that the topic vector of the (i-1)th layer... Able to transmit itself The dimensional information is then assigned to two i-th layer topics, thus initially forming the i-th layer topic vector; and the i-th layer topic... The sum of the transmission weights of the four (i-1)th layer topic vectors is The i-th layer topic The sum of the transmission weights of the four (i-1)th layer topic vectors is also... .

[0077] In this embodiment of the application, by setting a discrete measure between the (i-1)th layer topic vector and the i-th layer topic, the amount of information transmitted by each (i-1)th layer topic vector to itself and the information received by each i-th layer topic are constrained, thus ensuring fairness at the beginning of the transmission.

[0078] In some embodiments, a first transmission cost for the training process is determined based on the updated word segmentation transmission matrix, and a second transmission cost for the training process is determined based on the updated topic transmission matrix; a target loss value is determined based on the first transmission cost and L-1 second transmission costs; when the target loss value meets the training requirements, the word segmentation transmission matrix and L-1 topic transmission matrices are output.

[0079] In some embodiments, a first transmission cost between multiple training word vectors and multiple underlying topic vectors is determined based on a first distance between multiple training word vectors and each underlying topic vector of the training text and an updated word transfer matrix; a second transmission cost of the training process is determined based on the updated topic transfer matrix, including: determining a second transmission cost between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors of the training text based on a second distance between multiple (i-1)th layer topic vectors and each i-th layer topic vector and an updated topic transfer matrix.

[0080] In the embodiments of this application, L represents the total number of levels of the multi-layered topic.

[0081] Specifically, for the first transmission plan from multiple training word segmentation vectors to the underlying topic vectors in each iteration, the entropy regularization optimal transmission problem is followed, that is, the first transmission plan in each round of iterative training follows the following calculation formulas (7) and (8): (7) in, Indicates the first transmission cost; This represents the squared Euclidean distance (i.e., the first distance) between the j-th trained word segmentation vector and the k-th underlying topic vector. This represents the transfer weight between the j-th trained word segmentation vector and the k-th underlying topic vector, which is the transfer weight adjusted based on the first distance. This represents the first transmission control parameter, which controls the smoothness of transmission between multiple training word segmentation vectors and multiple underlying topic vectors.

[0082] In the first transmission plan, the distribution of each training word segmentation vector and each underlying topic vector in the transmission matrix of the first transmission plan is constrained by the following formula (8) as the first constraint: (8) in, Indicates a column of 1 dimensional vector; Represents a V-dimensional vector consisting of only one column; This represents the sum of the transfer weights that can be transferred for each row in the word segmentation transfer matrix, i.e., for each trained word segmentation vector (i.e., each row) in the constraint word segmentation transfer matrix. ; This represents all columns in the word segmentation transmission matrix, that is, the sum of the transmission weights received by multiple underlying topic vectors (i.e., all columns of the word segmentation transmission matrix) in the constraint word segmentation transmission matrix. .

[0083] The entropy regularization in the first transmission plan is a "structural simplification" of the original transmission problem, which is "naturally compatible" with the iterative mechanism of the Sinkhorn algorithm. Entropy regularization transforms hard constraints into a decomposable convex optimization problem, while Sinkhorn, based on the core logic of "Iterative Scaling," precisely utilizes the decomposition structure of the transmission problem, achieving a unity of efficiency, parallelism, stability, and sparsity from the bottom up. Therefore, in order to efficiently solve and obtain the first transmission plan with the minimum cost, the first transmission plan is optimized using the Sinkhorn algorithm shown in the following formula (9) to obtain the optimal word segmentation transmission matrix: (9) in, Represents the word segmentation transfer matrix; Represents the discrete measure of the trained word segmentation vectors; Represents a discrete measure of the underlying topic vector; This represents the first transmission control parameter.

[0084] The first transmission cost obtained from the first transmission plan is calculated as shown in the following formula (10): (10) in, Indicates the first transmission cost; This represents the squared Euclidean distance (i.e., the first distance) between the j-th trained word segmentation vector and the k-th underlying topic vector. The squared Euclidean distance between the j-th trained word segmentation vector and the k-th underlying topic vector; This represents the optimal transfer weight between the j-th training word segmentation vector and the k-th underlying topic vector.

[0085] For the second transmission plan of multiple (i-1)th layer topic vectors to the i-th layer topic vector each time, the entropy regularization optimal transmission problem is also followed, that is, the second transmission plan in each round of iterative training follows the following calculation formulas (11) and (12): (11) in, Indicates the first transmission cost; Represents the relationship between the k-th (i-1)th layer topic vector and the... The squared Euclidean distance (i.e., the second distance) between the i-th layer topic vectors; Represents the relationship between the k-th (i-1)th layer topic vector and the... The weights transferred between the i-th layer topic vectors are the weights adjusted based on the second distance. This represents the second transmission control parameter, which controls the transmission smoothness between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

[0086] In the second transmission plan, the distribution of each (i-1)th layer topic vector and each (i)th layer topic vector in the second transmission plan is constrained by the following formula (12) as the second constraint: (12) Among them, among them, Indicates a column of 1 dimensional vector; Let $\mathbf{i-1}$ represent the sum of the transfer weights that can be transferred for each row in the topic transfer matrix of the (i-1)th layer topic vector and the i-th layer topic vector, i.e., the sum of the transfer weights for each row of the (i-1)th layer topic segmentation vector in the topic transfer matrix. ; This represents the sum of the transmission weights received by all columns in the topic transfer matrix, i.e., the constraints on the sum of the transmission weights received by multiple i-th layer topic vectors (i.e., all columns of the topic transfer matrix). .

[0087] Similarly, the second transmission plan is optimized using the Sinkhorn algorithm to obtain the optimal word segmentation transmission matrix. The specific calculation formula is shown in the following formula (13): (13) in, This represents the topic transfer matrix between the (i-1)th layer topic vector and the ith layer topic vector; Represents the discrete measure of the topic vector at the (i-1)th layer; This represents the discrete measure of the topic vector at the i-th layer; This represents the second transmission control parameter, which controls the transmission smoothness between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

[0088] The second transmission cost obtained from the second transmission plan is calculated as shown in the following formula (14): (14) in, This represents the second transmission cost; L represents the redemption cost of the number of topics, i.e., the total number of levels. Represents the relationship between the k-th (i-1)th layer topic vector and the... The second distance of the i-th layer topic vectors; Represents the relationship between the k-th (i-1)th layer topic vector and the... The optimal transfer weights between the i-th layer topic vectors.

[0089] For example, for a topic tree with a three-level topic hierarchy, the high-level topics (including the middle-level topics and the top-level topics) are constructed based on the low-level topic vectors, and the low-level topic vectors are constructed based on the training word segmentation vectors. Therefore, it is necessary to construct a first transmission plan between the training analysis vectors and the low-level topics, as well as two second transmission plans between the low-level topic vectors and the middle-level topics, and between the middle-level topic vectors and the top-level topics, and so on, in order to construct the topic hierarchy of the file.

[0090] In the embodiments of this application, the first transmission plan uses the bottom-level topic vector as the semantic anchor point of multiple training word segmentation vectors, and the second transmission plan uses the i-th layer topic vector as the semantic anchor point of multiple (i-1)-th layer topic vectors, guiding the training word segmentation vectors or the (i-1)-th layer topic vectors to be distributed to the topic vectors of the next higher level. Compared with the unsupervised training methods in the prior art, this can enhance the interpretability of the hierarchical topic model and the consistency of the hierarchical topic structure, and induce a stable and sparse word segmentation transmission matrix and topic transmission matrix.

[0091] Step 204: Based on the topic vector of each layer and the word segmentation vectors of multiple words, infer the distribution matrix of each topic and multiple words.

[0092] Specifically, according to the VAE method, the correlation between each topic vector and each training word in each layer is calculated using the following formula (15), thereby obtaining the distribution matrix of each layer's topics and multiple training words: (15) in, This indicates that the i-th training word segment and the k-th topic are in the same position. Layer correlation; This represents the k-th (i-1)-th layer topic vector; This represents the i-th word in the training text; It's a hyperparameter.

[0093] Furthermore, based on the first transmission plan and the second transmission plan between the (i-1)th layer topic vector and the i-th layer topic vector, normalized training documents and the i-th layer topic distribution matrix are generated, and the specific calculation formula is shown in the following formula (16): (16) in, This represents the distribution matrix of the training documents and the topics in the i-th layer; Used for Rescaling; This represents the optimal topic transfer matrix between the (i-1)th layer topic vector and the ith layer topic vector; This represents the distribution matrix of the documents to be trained and the topics of the i-th layer.

[0094] In some embodiments, based on the topic vector of each layer and multiple training word vectors, the distribution matrix of each layer of topics and multiple training word vectors is inferred; based on the distribution matrix of each layer of topics and multiple training word vectors, multiple training word vectors and training text, the distribution matrix of training text and each layer of topics is inferred; based on the distribution matrix of training text and each layer of topics, and the distribution matrix of each layer of topics and multiple training word vectors, the document is reconstructed to obtain the reconstruction loss value; based on the first transmission cost, L-1 second transmission costs and the reconstruction loss value, the target loss value is determined.

[0095] In some embodiments, the divergence loss value is determined based on the difference between the distribution matrix of the training text and the topic prior matrix of each layer; wherein the topic prior matrix is ​​a distribution template of the training text and the topic determined by a Gaussian distribution; and the target loss value is determined based on the divergence loss value, the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0096] Specifically, after deriving the distribution matrix of each layer's topic vector and the training text, and the distribution matrix of each layer's topic and multiple training words, the training text is reconstructed based on the decoding method in VAE. The reconstructed training text is then compared with the real training text to obtain the reconstruction loss value.

[0097] Furthermore, the distribution matrix of the training text and each layer of topics is compared with the prior matrix of the training text and topics determined by Gaussian distribution to obtain the divergence loss value; the divergence loss value and the reconstruction loss value are both obtained based on the decoding method in the VAE method, and the specific calculation formula is shown in the following formula (17): (17) in, This indicates that the loss value is obtained through the decoding method, which is the sum of the reconstruction loss value and the divergence loss value; Indicates the total number of levels in the topic hierarchy; Represents the training text; This represents the distribution matrix of the topic vectors and multiple trained word segmentation vectors at the (i-1)th layer; This represents the distribution matrix of the training text and the topic vectors of the (i-1)th layer; This represents the prior topic matrix, i.e., the true distribution matrix; This represents the distribution matrix of training texts and topics at each layer, obtained by training the training texts using the model.

[0098] Step 205: Based on the distribution matrix of each topic and multiple word segments, determine the topic tags of the text to be processed in each topic.

[0099] In some embodiments, from the distribution matrix of the underlying topic and multiple word segments, at least one word with a distribution probability greater than a first threshold under any underlying topic is determined, and the topic tag of the underlying topic is summarized based on the at least one word; from the distribution matrix of the i-th layer topic and multiple word segments, at least one word with a distribution probability greater than a second threshold under any i-th layer topic is determined, and the topic tag of the i-th layer topic is summarized based on the at least one word.

[0100] Specifically, after obtaining the word segmentation transfer matrix and L-1 topic transfer matrices during the training phase, where L is pre-set by the model's hyperparameters, multiple word segments of the text to be processed are input into the pre-trained hierarchical topic model. Each word segment transmits its own information to the bottom-level topic vector according to the word segmentation transfer matrix. The bottom-level topic vector transmits its own information to the next level of topic vector according to the topic transfer matrix of its corresponding level, and so on, to obtain the topic hierarchy. At this point, the topic vectors in the topic hierarchy are still not specific topic labels, such as entertainment, science, etc.

[0101] The similarity between each topic vector and each word in each topic layer is calculated using the formula (15) above, thereby obtaining the distribution probability of each word under each topic. For any bottom-level topic, at least one word with a distribution probability greater than the first threshold is obtained, and the topic label of the bottom-level topic is summarized based on the semantics of at least one word, such as artificial intelligence. For any i-th level topic, at least one word with a distribution probability greater than the second threshold is obtained, and the topic label of the i-th level topic is summarized based on at least one word. Alternatively, after determining the topic label of the (i-1)-th level topic, the distribution matrix of the i-th level topic vector and the (i-1)-th level topic vector is calculated, and at least one topic label of the (i-1)-th level topic with a distribution probability greater than the third threshold under the i-th level topic is obtained. Based on the topic label of at least one (i-1)-th level topic, the topic label of the i-th level topic is summarized. The first threshold, the second threshold, and the third threshold are set according to actual needs, and this application does not make specific limitations on them.

[0102] The word segmentation vectors of multiple words in the text to be processed are determined. For any word segmentation, the underlying topic vector corresponding to the word segmentation vector when it is transmitted to the underlying topic is determined according to the transmission weight of the word segmentation and the underlying topic in the word segmentation transmission matrix. The underlying topic is the first-level topic. Each transmission weight represents the transmission amount, which is equivalent to using the underlying topic vector as an embedding anchor to guide the potential transmission direction of multiple words, suppress the dominant effect of high-frequency general words such as "method" and "research", highlight key information, and facilitate the generation of coherent fine-grained topics.

[0103] For any (i-1)th level topic, the i-th level topic vector corresponding to the (i-1)th level topic vector transmitted to the i-th level topic is determined according to the transmission weights of the (i-1)th level topic and the i-th level topic in the topic transmission matrix. i is an integer greater than or equal to 2. Between different levels of topics, the transmission of lower-level topics to higher-level topics is guided to achieve cross-level topic alignment, so that the topics of each level maintain semantic consistency and hierarchical progression. Thus, based on the topic vector of each level and the word vectors of multiple words, the distribution matrix of each level of topics and multiple words can be inferred, and the topic tags of the text to be processed in each level of topics can be determined, resulting in a hierarchical topic structure with good interpretability and coverage. This alleviates the problem of difficulty in grasping the distinguishability between topics of the same level and the loose structure between topics of different levels in unsupervised modeling.

[0104] Based on the same technical concept, embodiments of this application provide a structural schematic diagram of a text topic determination device, such as... Figure 3 As shown, the text topic determination device 300 includes: The word segmentation vector module 301 is used to determine the word segmentation vectors of multiple words in the text to be processed. The underlying topic module 302 is used to determine the underlying topic vector corresponding to the word segmentation vector when it is transmitted to the underlying topic, according to the transmission weight of word segmentation and underlying topic in the word segmentation transmission matrix for any word segmentation; the underlying topic is the first-level topic; any transmission weight represents the transmission amount; The high-level topic module 303 is used to determine the i-th level topic vector corresponding to the transmission of the i-th level topic vector of the word segmentation to the i-th level topic according to the transmission weight of the i-th level topic and the i-th level topic in the topic transmission matrix for any i-th level topic; i is an integer greater than or equal to 2. The distribution matrix module 304 is used to infer the distribution matrix of each topic and multiple words based on the topic vector of each layer and the word vector of the multiple words; The topic tag module 305 is used to determine the topic tags of the text to be processed in each topic based on the distribution matrix of each topic and multiple word segments.

[0105] Optionally, the underlying theme module 302 is specifically used for: Determine the training word vectors for each of the multiple training word segments in the training text; Determine the first distance from any trained word segmentation vector to the corresponding underlying topic vector of each underlying topic; Based on the first distance, the transmission weights of the training word and the underlying topic in the word segmentation transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same training word and multiple underlying topics is a first fixed value; the first fixed value is used to control the amount of transmission allowed for each training word to multiple underlying topics.

[0106] Optionally, the high-level topic module 303 is specifically used for: Determine the second distance from the (i-1)th layer topic vector corresponding to any (i-1)th layer topic to the i-th layer topic vector corresponding to each i-th layer topic; Based on the second distance, the transmission weights between the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same (i-1)th layer topic and multiple i-th layer topics is a second fixed value, which is used to control the amount of transmission allowed from each (i-1)th layer topic to the i-th layer topic.

[0107] Optionally, during training, a first transmission parameter is set to control the transmission smoothness between multiple training word segmentation vectors and multiple underlying topic vectors; a second transmission parameter is set to control the transmission smoothness between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

[0108] Optionally, the distribution matrix module 304 is specifically used for: The first transmission cost of the training process is determined based on the updated word segmentation transmission matrix, and the second transmission cost of the training process is determined based on the updated topic transmission matrix. Based on the first transmission cost and L-1 second transmission costs, the target loss value is determined; When the target loss value meets the training requirements, the word segmentation transfer matrix and L-1 topic transfer matrices are output; where L is the total number of levels of the multi-layer topics.

[0109] Optionally, the distribution matrix module 304 is specifically used for: Based on the first distance between multiple training word segmentation vectors and each underlying topic vector of the training text and the updated word segmentation transfer matrix, the first transfer cost between the multiple training word segmentation vectors and the multiple underlying topic vectors is determined. The determination of the second transmission cost of the training process based on the updated topic transfer matrix includes: Based on the second distance between multiple (i-1)th layer topic vectors and each i-th layer topic vector, and the updated topic transfer matrix, the second transfer cost of the training text between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors is determined.

[0110] Optionally, the distribution matrix module 304 is specifically used for: Based on the topic vector of each layer and multiple training word segmentation vectors, the distribution matrix of each topic and multiple training word segmentation is inferred; Based on the distribution matrix of each topic layer and multiple training words, the multiple training words, and the training text, the distribution matrix of the training text and each topic layer is inferred. Based on the distribution matrix of the training text and each layer of topics, and the distribution matrix of each layer of topics and multiple training word segments, the document is reconstructed to obtain the reconstruction loss value. The target loss value is determined based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0111] Optionally, the distribution matrix module 304 is specifically used for: The divergence loss value is determined based on the difference between the distribution matrix of the training text and the topic prior matrix of each layer; wherein, the topic prior matrix is ​​the distribution template of the training text and the topic determined by Gaussian distribution. The target loss value is determined based on the divergence loss value, the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

[0112] Optionally, the topic tag module 305 is specifically used for: From the distribution matrix of the underlying topic and multiple word segments, determine at least one word segment with a distribution probability greater than a first threshold under any underlying topic, and based on the at least one word segment, summarize the topic tag of the underlying topic; From the distribution matrix of the i-th layer topic and multiple word segments, determine at least one word segment with a distribution probability greater than the second threshold under any i-th layer topic, and based on the at least one word segment, summarize the topic tag of the i-th layer topic.

[0113] In this embodiment, the segmentation vectors of multiple words in the text to be processed are determined. For any word, the underlying topic vector corresponding to the segmentation vector when it is transmitted to the underlying topic is determined according to the transmission weight of the word and the underlying topic in the segmentation transmission matrix. The underlying topic is the first-level topic. Each transmission weight represents the transmission amount, which is equivalent to using the underlying topic vector as an embedding anchor to guide the potential transmission direction of multiple words, suppress the dominant effect of high-frequency general words such as "method" and "research", highlight key information, and facilitate the generation of coherent fine-grained topics.

[0114] For any (i-1)th level topic, the i-th level topic vector corresponding to the (i-1)th level topic vector transmitted to the i-th level topic is determined according to the transmission weights of the (i-1)th level topic and the i-th level topic in the topic transmission matrix. i is an integer greater than or equal to 2. Between different levels of topics, the transmission of lower-level topics to higher-level topics is guided to achieve cross-level topic alignment, so that the topics of each level maintain semantic consistency and hierarchical progression. Thus, based on the topic vector of each level and the word vectors of multiple words, the distribution matrix of each level of topics and multiple words can be inferred, and the topic tags of the text to be processed in each level of topics can be determined, resulting in a hierarchical topic structure with good interpretability and coverage. This alleviates the problem of difficulty in grasping the distinguishability between topics of the same level and the loose structure between topics of different levels in unsupervised modeling.

[0115] Based on the same technical concept, embodiments of this application provide a computer device, which can be... Figure 1 The server shown is as follows: Figure 4 As shown, it includes at least one processor 401 and a memory 402 connected to at least one processor. In this embodiment, the specific linking medium between the processor 401 and the memory 402 is not limited. Figure 4 Taking the connection between processor 401 and memory 402 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0116] In this embodiment of the application, the memory 402 stores instructions executed by at least one processor 401. By executing the instructions stored in the memory 402, at least one processor 401 can perform the steps of the above-described text topic determination method.

[0117] The processor 401 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It determines the text topic by running or executing instructions stored in the memory 402 and retrieving data stored in the memory 402. Optionally, the processor 401 may include one or more processing modules. The processor 401 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0118] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0119] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 402 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0120] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described text topic determination method.

[0121] Based on the same inventive concept, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described text topic determination method.

[0122] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the topic of a text, characterized in that, include: Determine the segmentation vectors of each of the multiple words in the text to be processed; For any word segment, the underlying topic vector corresponding to the word segment vector when it is transmitted to the underlying topic is determined according to the transmission weight of the word segment and the underlying topic in the word segmentation transmission matrix; the underlying topic is the first-level topic; any transmission weight represents the transmission amount; For any (i-1)th layer topic, the i-th layer topic vector corresponding to the (i-1)th layer topic vector transmitted to the i-th layer topic is determined according to the transmission weights of the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix; i is an integer greater than or equal to 2. Based on the topic vector of each layer and the word segmentation vectors of the multiple words, the distribution matrix of each topic and the multiple words is inferred; Based on the distribution matrix of each topic and multiple word segments, the topic tags of the text to be processed in each topic are determined.

2. The method as described in claim 1, characterized in that, The word segmentation transfer matrix is ​​obtained by training in the following ways: Determine the training word vectors for each of the multiple training word segments in the training text; Determine the first distance from any trained word segmentation vector to the corresponding underlying topic vector of each underlying topic; Based on the first distance, the transmission weights of the training word and the underlying topic in the word segmentation transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same training word and multiple underlying topics is a first fixed value; the first fixed value is used to control the amount of transmission allowed for each training word to multiple underlying topics.

3. The method as described in claim 1, characterized in that, The transmission weights between topics in layer i-1 and layer i in the topic transmission matrix are obtained through training in the following manner: Determine the second distance from the (i-1)th layer topic vector corresponding to any (i-1)th layer topic to the i-th layer topic vector corresponding to each i-th layer topic; Based on the second distance, the transmission weights between the (i-1)th layer topic and the i-th layer topic in the topic transmission matrix are updated until the training requirements are met; wherein, the sum of the transmission weights of the same (i-1)th layer topic and multiple i-th layer topics is a second fixed value, which is used to control the amount of transmission allowed from each (i-1)th layer topic to the i-th layer topic.

4. The method as described in claim 2 or 3, characterized in that, During training, a first transmission parameter is set to control the smoothness of transmission between multiple training word segmentation vectors and multiple underlying topic vectors; a second transmission parameter is set to control the smoothness of transmission between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors.

5. The method as described in claim 2 or 3, characterized in that, Until the training requirements are met, including: The first transmission cost of the training process is determined based on the updated word segmentation transmission matrix, and the second transmission cost of the training process is determined based on the updated topic transmission matrix. Based on the first transmission cost and L-1 second transmission costs, the target loss value is determined; When the target loss value meets the training requirements, the word segmentation transfer matrix and L-1 topic transfer matrices are output; where L is the total number of levels of the multi-layer topics.

6. The method as described in claim 5, characterized in that, The determination of the first transmission cost of the training process based on the updated word segmentation transmission matrix includes: Based on the first distance between multiple training word segmentation vectors and each underlying topic vector of the training text and the updated word segmentation transfer matrix, the first transfer cost between the multiple training word segmentation vectors and the multiple underlying topic vectors is determined. The determination of the second transmission cost of the training process based on the updated topic transfer matrix includes: Based on the second distance between multiple (i-1)th layer topic vectors and each i-th layer topic vector, and the updated topic transfer matrix, the second transfer cost of the training text between multiple (i-1)th layer topic vectors and multiple i-th layer topic vectors is determined.

7. The method as described in claim 5, characterized in that, The step of determining the target loss value based on the first transmission cost and L-1 second transmission costs includes: Based on the topic vector of each layer and multiple training word segmentation vectors, the distribution matrix of each topic and multiple training word segmentation is inferred; Based on the distribution matrix of each topic layer and multiple training words, the multiple training words, and the training text, the distribution matrix of the training text and each topic layer is inferred. Based on the distribution matrix of the training text and each layer of topics, and the distribution matrix of each layer of topics and multiple training word segments, the document is reconstructed to obtain the reconstruction loss value. The target loss value is determined based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

8. The method as described in claim 7, characterized in that, The step of determining the target loss value based on the first transmission cost, L-1 second transmission costs, and the reconstruction loss value includes: The divergence loss value is determined based on the difference between the distribution matrix of the training text and the topic prior matrix of each layer; wherein, the topic prior matrix is ​​the distribution template of the training text and the topic determined by the Gaussian distribution. The target loss value is determined based on the divergence loss value, the first transmission cost, L-1 second transmission costs, and the reconstruction loss value.

9. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 8.

Images