Method for training a topic recognition model, method and device for recognizing a document topic

Abstract

The application is suitable for the field of artificial intelligence technology, and provides a training method of a topic identification model, a document topic identification method and device, which comprises: inputting a to-be-trained document into a topic identification model to obtain frozen vector information; determining a first transmission cost matrix and a second transmission cost matrix according to the frozen vector information of each to-be-trained document, each preset topic embedding vector and each preset word embedding vector; the first transmission cost matrix represents the transmission cost of each document to each topic, and the second transmission cost matrix represents the transmission cost of each topic to each word; the topic embedding vector represents the semantic feature of the topic, and the word embedding vector represents the semantic feature of the word; determining an embedding transmission plan loss, a double semantic relationship reconstruction loss and a structure perception distribution alignment loss according to the first transmission cost matrix and the second transmission cost matrix; and performing model training according to the embedding transmission plan loss, the double semantic relationship reconstruction loss and the structure perception distribution alignment loss.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a training method for a topic recognition model, a method and apparatus for document topic recognition. Background Technology

[0002] In areas such as freight supervision and risk assessment, inspection records often contain certain risk points. Staff need to identify the specific risk points in the inspection records to facilitate subsequent supervision.

[0003] Current risk topic identification methods mostly rely on manual judgment or rule base matching, which wastes a lot of manpower and time, resulting in low efficiency in topic identification. Summary of the Invention

[0004] This application provides a training method for a topic recognition model, a method and apparatus for identifying document topics, which can improve the training accuracy of the model and thus improve the accuracy of topic recognition.

[0005] In a first aspect, embodiments of this application provide a method for training a topic recognition model, including: The document to be trained is input into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector, a first transmission cost matrix and a second transmission cost matrix are determined; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. Based on the first transmission cost matrix and the second transmission cost matrix, the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss are determined; wherein, the embedding transmission plan loss represents the document-to-topic transmission loss and the topic-to-vocabulary transmission loss, the dual semantic relation reconstruction loss represents the loss between the actual vocabulary distribution in the document and the predicted vocabulary distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution. The topic recognition model is trained based on the embedded transport plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0006] Secondly, embodiments of this application provide a method for identifying document topics, including: Obtain the document to be recognized; The document to be identified is input into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; The topic corresponding to the document to be identified is determined based on the topic distribution vector of the document to be identified; wherein the topic identification model is the topic identification model described in the first aspect.

[0007] Thirdly, embodiments of this application provide a training apparatus for a topic recognition model, comprising: The document input module is used to input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; The matrix determination module is used to determine a first transmission cost matrix and a second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. The loss determination module is used to determine the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss based on the first transmission cost matrix and the second transmission cost matrix; wherein, the embedding transmission plan loss represents the document-to-topic transmission loss and the topic-to-vocabulary transmission loss, the dual semantic relation reconstruction loss represents the loss between the actual vocabulary distribution in the document and the predicted vocabulary distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution; The model training module is used to train the topic recognition model based on the embedded transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0008] Fourthly, embodiments of this application provide a document topic identification device, including: The document acquisition module is used to acquire the document to be recognized. The vector output module is used to input the document to be identified into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; The topic determination module is used to determine the topic corresponding to the document to be identified based on the topic distribution vector of the document to be identified; wherein the topic identification model is the topic identification model according to any one of claims 1-11.

[0009] Fifthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0011] In a seventh aspect, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the method described in the first aspect above.

[0012] In this embodiment, the document to be trained is input into the topic recognition model, and feature extraction is performed on the document to obtain frozen vector information. Multiple topics and words are pre-set. Based on the frozen vector information of each document to be trained, the topic embedding vector of each topic, and the word embedding vector of each word, a first transmission cost matrix and a second transmission cost matrix are determined. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word, thus introducing the embedding transmission plan algorithm and facilitating the calculation of the embedding transmission plan loss. Furthermore, the dual semantic relation reconstruction loss and structure-aware distribution alignment loss can be calculated to enable multi-dimensional training of the topic recognition model, improving the model's training accuracy and consequently, the topic recognition accuracy. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of distance and semantic mismatch in the embedding space provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the training method of the topic recognition model provided in the embodiments of this application; Figure 3 This is a flowchart illustrating the training method of the topic recognition model provided in the embodiments of this application; Figure 4 This is a flowchart illustrating the training method of the topic recognition model provided in the embodiments of this application; Figure 5This is a flowchart illustrating the training method of the topic recognition model provided in the embodiments of this application; Figure 6 This is an overall framework diagram of model training provided in the embodiments of this application; Figure 7 This is a flowchart illustrating the document topic identification method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the training device for the topic recognition model provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the document topic identification device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0016] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0017] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0018] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0019] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0020] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0021] The training method for the topic recognition model provided in this application embodiment can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, robots, and personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of terminal device.

[0022] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows: Subject: Refers to the risk category involved in the document. In this application embodiment, the document may refer to customs inspection records. For customs inspection records, the subject may include risks related to price declaration, risks related to misdeclaration of specifications, risks related to procedural violations of declaration, etc.

[0023] ETP: Embedding Transport Plan.

[0024] Sinkhorn: Optimal transfer neighborhood algorithm.

[0025] DSR: Dual Semantic-relation Reconstruction.

[0026] SADA: Structure-Aware Distribution Alignment.

[0027] Customs inspection records typically integrate declaration elements, on-site inspection facts, anomaly findings, handling opinions, mode of transport, cargo description, and subsequent regulatory clues. They serve as crucial textual evidence for freight supervision, risk assessment, and enforcement review. These records are characterized by their high concentration of technical terminology, lengthy texts, dispersed evidence distribution, and the potential for a single record to correspond to multiple risk points. In identifying risk themes, the system not only needs to determine whether a record involves a particular type of risk but also needs to explain the source of the risk evidence, its relevance to specific inspection facts, and whether further manual review is required.

[0028] Current topic identification solutions typically involve manual judgment, rule bases, keyword dictionaries, or template matching. These methods can clearly express some regulatory rules, facilitating auditing and interpretation, but require continuous rule maintenance and struggle to cover long-tail expressions, complex semantics, and novel risk patterns. Alternatively, intelligent agent methods based on pre-trained language models, large language models, and retrieval enhancement can be used. These methods can combine domain knowledge bases to generate "label-evidence-suggestion" outputs, demonstrating good usability in practical business applications. However, their underlying semantic organization often relies on the original embedding space or retrieval results, lacking explicit calibration for the structure of potential risk topics.

[0029] Furthermore, directly embedding large language models into long texts for topic modeling and risk label recognition in customs inspection records still presents key challenges. Large language model embeddings are typically optimized for retrieval or semantic matching objectives, and their geometric neighborhood is not necessarily equivalent to the latent topic neighborhood. For example... Figure 1 As shown, Figure 1 This diagram illustrates distance and semantic mismatch in the embedding space. Records that are close in distance in the embedding space may not share the same risk topic, while records with the same fine-grained topic semantics may be separated by distance due to retrieval training objectives. Figure 1 The cone-shaped region in the diagram represents the topic neighborhood of the document within the black box. However, the white box within the dashed ellipse does not belong to the same topic as the black box. If clustering, topic inference, or label-assisted determination is performed directly based on the cone-shaped region, noisy neighbors and outliers may be propagated into the topic inference process, affecting the accuracy of topic recognition.

[0030] General Description of Embodiments in this Application According to one embodiment of this application, a method for training a topic recognition model is provided.

[0031] Figure 2 A flowchart illustrating the training method for a topic recognition model, as shown below. Figure 2 As shown, a method for training a topic recognition model according to an embodiment of this application may include: Step 210: Input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Step 220: Determine the first transmission cost matrix and the second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. Step 230: Based on the first transmission cost matrix and the second transmission cost matrix, determine the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss; wherein, the embedding transmission plan loss represents the transmission loss from document to topic and the transmission loss from topic to word, the dual semantic relation reconstruction loss represents the loss between the actual word distribution in the document and the predicted word distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution. Step 240: Train the topic recognition model based on the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0032] Steps 210-240 are described in detail below.

[0033] In step 210, the document to be trained can be a pre-collected document dataset used for model training, such as customs inspection records generated within a historical period. Frozen vector information is vector data obtained by encoding the complete document to be trained, with fixed parameters that no longer participate in gradient updates. The frozen vector information fully preserves the global contextual semantics of the document to be trained, and can serve as the underlying semantic input for subsequent nonlinear projection, neighborhood construction, and transmission cost calculation in the topic recognition model, providing a stable original semantic representation for all subsequent loss calculation modules.

[0034] In this embodiment, the topic recognition model can use an open-source network architecture to initialize the document embedding representation, i.e., generate frozen vector information. This embodiment does not specifically limit the process of generating frozen vector information; for example, feature extraction methods can be used to obtain frozen vector information. Alternatively, encoding operations can be performed on the training documents that have undergone cleaning and desensitization processing to output frozen vector information with fixed parameters that cannot be trained or adjusted, thus completing the initial semantic representation acquisition process of the model. For example, a general text embedding model that supports ultra-long context input can be used to generate frozen vector information. This general text embedding model maintains a frozen parameter state throughout the process, only outputting static vector results and not participating in the gradient backpropagation update of the topic recognition model.

[0035] By obtaining the frozen vector information corresponding to the document to be trained, the semantic association of all paragraphs and fields in the long text document can be completely preserved, providing a complete and uninterrupted original semantic basis for subsequent semantic alignment calculations of the document, topic, and vocabulary layers.

[0036] In step 220, multiple topics and multiple words are pre-set. Each topic corresponds to a pre-set topic embedding vector, and each word corresponds to a pre-set word embedding vector. The pre-set topic embedding vector is pre-initialized vector data used to represent the inherent semantic features of each type of potential risk topic, which can establish semantic association between the document and the topic; the pre-set word embedding vector is pre-initialized vector data used to represent the inherent semantic features of each word in the text, which can establish semantic association between the topic and the word.

[0037] By combining the frozen vector information of each training document, the topic embedding vectors corresponding to each topic, and the word embedding vectors corresponding to each word, the first and second transmission cost matrices can be calculated. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word. For example, an embedding transmission plan can be introduced to transform semantic alignment into an optimal transmission problem, and the document-to-topic transmission cost matrix and the topic-to-word transmission cost matrix can be calculated separately. The document-to-topic transmission cost matrix is ​​the first transmission cost matrix, and the topic-to-word transmission cost matrix is ​​the second transmission cost matrix.

[0038] The first transmission cost matrix is ​​a matrix generated based on frozen vector information and preset topic embedding vectors. Each element in the first transmission cost matrix corresponds to the semantic distance cost between a single document and a single topic, quantifying the degree of loss in assigning semantic weights to a document for its corresponding topic. The second transmission cost matrix is ​​a matrix generated based on preset topic embedding vectors and preset word embedding vectors. Each element in the second transmission cost matrix corresponds to the semantic distance cost between a single topic and a single word, quantifying the degree of loss in assigning semantic weights to a topic for its corresponding word. In other words, using frozen vector information from all documents to be trained, preset topic semantic vectors, and preset word semantic vectors as input, the semantic distance costs for document-to-topic and topic-to-word matching are calculated separately, generating two independent cost matrices and completing the construction of the basic data for optimal transmission cost calculation.

[0039] For example, the L2 distance can be used as the semantic distance calculation rule. A first transmission cost matrix is ​​generated by traversing all documents and all topics, and a second transmission cost matrix is ​​generated by traversing all topics and all words. Each value in the matrix corresponds to the spatial distance loss between the two sets of vectors. Besides the L2 distance, cosine distance, Manhattan distance, etc., can also be used as the calculation rule for the transmission cost between vectors, as long as the cost matrix can objectively quantify the semantic differences between the two sets of vectors.

[0040] By generating two sets of transmission cost matrices based on frozen vector information, topic embedding vectors, and word embedding vectors respectively, a unified quantization loss standard can be established for documents, topics, and words in the same embedding space, providing complete cost input conditions for subsequent solving the transmission plan matrix based on entropy regularization optimal transmission algorithm.

[0041] In step 230, the loss function for model training may include at least three types, namely, embedding transmission plan loss, dual semantic relation reconstruction loss, and structure-aware distribution alignment loss. The embedding transport plan loss is a comprehensive loss term obtained by superimposing the transport plan matrices obtained from two sets of transport cost matrices through an entropy-regularized optimal transport algorithm. The embedding transport plan loss constrains the overall loss of the two-layer semantic transport process from document to topic and from topic to vocabulary, respectively, and can guide the semantic representations of document, topic, and vocabulary to tend to be unified and aligned. The dual semantic relation reconstruction loss is a cross-entropy loss term obtained by taking the original bag-of-words representation of the document and the document-topic distribution and topic-vocabulary distribution derived from the transport plan matrix as input. The dual semantic relation reconstruction loss is used to constrain the vocabulary distribution output by the model to fit the real vocabulary composition of the document, ensuring that the topic modeling results fit the objective lexical features of the original text and avoiding the topic semantics from deviating from the original content of the document. The structure-aware distribution alignment loss is an alignment loss term calculated based on the reference distribution constructed from the topic-consistent neighborhood after refinement and filtering and the learning distribution obtained by the model. The structure-aware distribution alignment loss is used to constrain the semantic relations of the document neighborhood learned by the model to fit the real risk topic clustering structure and eliminate the neighborhood semantic mismatch noise brought about by the retrieval-oriented embedding space.

[0042] This embodiment solves for the optimal transmission-related loss based on two sets of transmission cost matrices, solves for the text reconstruction-related loss by combining the original bag-of-words representation of the document, and solves for the distribution alignment-related loss based on the calibration neighborhood after topic consistency filtering. The results are summarized into three independent model optimization loss terms, which fully cover the three optimization objectives of topic modeling, text reconstruction, and local semantic structure calibration.

[0043] For example, the Sinkhorn iterative algorithm can be used to solve the entropy regularization optimal transport problem, calculate the sub-losses corresponding to the document-topic transport plan and the topic-vocabulary transport plan respectively, and sum them to obtain the complete embedding transport plan loss; the cross-entropy loss function can be used to calculate the deviation between the original bag of words and the reconstructed vocabulary distribution to obtain the dual semantic relation reconstruction loss; the KL divergence can be used to calculate the difference between the reference distribution and the learned distribution to obtain the structure-aware distribution alignment loss.

[0044] By simultaneously solving for three types of losses—embedded transport plan loss, dual semantic relation reconstruction loss, and structure-aware distribution alignment loss—it can simultaneously constrain three major optimization directions: document-topic-vocabulary semantic alignment, original text information restoration, and local topic neighborhood structure calibration, thus fully covering all optimization constraints required by the topic recognition model.

[0045] In step 240, the embedding transport plan loss unifies the semantic space of documents, topics, and words; the dual semantic relation reconstruction loss ensures that the topic modeling fits the original document's lexical facts; and the structure-aware distribution alignment loss corrects topic-irrelevant noise associations in the original embedding neighborhood. These three loss terms work together to iteratively update all trainable parameters of the topic recognition model. For example, the three loss terms can be weighted and combined to obtain the overall optimization objective of the model. Based on the total loss value, the gradient backpropagation algorithm is used to iteratively update the trainable network parameters, topic embedding vectors, and word embedding vectors within the topic recognition model, completing the full training process of the topic recognition model.

[0046] By fusing three types of loss terms to iteratively train the topic recognition model, the coherence, diversity, and document-topic distribution discriminability of the output topics can be improved simultaneously. This reduces the interference of noisy neighborhoods in the original embedding space on the topic inference results, and outputs a document topic distribution representation with interpretable topic structure that is suitable for industry risk label recognition tasks.

[0047] In one implementation, a topic recognition model is trained based on the embedding transport plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss, including: performing a weighted summation of the embedding transport plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss to train the topic recognition model.

[0048] Specifically, independent weight coefficients can be pre-configured for different types of loss terms, and then the superposition operation is performed. The model is then trained using backpropagation based on the total loss function of the superposition operation. The impact of three constraint objectives—embedding transport plan loss, dual semantic relation reconstruction loss, and structure-aware distribution alignment loss—on the parameter updates of the topic recognition model is balanced. The constraint priority of each type of loss during the model iterative optimization process can be adjusted according to the weight coefficients, enabling the model to simultaneously consider multiple optimization objectives: document topic vocabulary semantic alignment, original text information restoration, and local topic neighborhood structure calibration. The configured weight coefficients directly affect the construction of the overall training objectives of the topic recognition model, providing a unified total loss benchmark for subsequent gradient backpropagation operations.

[0049] The formula for the total loss function can be expressed as: L=L ETP +L DSR +λL SADA ; Where L is the total loss function, L ETP Characterizing the embedded transmission plan loss, L DSR L represents the loss for reconstructing dual semantic relations. SADA The structure-aware distribution alignment loss is represented by λ, which is a preset weighting coefficient.

[0050] In this embodiment, by performing weighted summation on the embedding transmission plan loss, dual semantic relation reconstruction loss, and structure-aware distribution alignment loss before training the topic recognition model, the optimization priority of the three types of semantic constraints can be flexibly adjusted. This simultaneously takes into account the semantic alignment of document topic vocabulary, the restoration of lexical information of the original text, and the calibration of topic-consistent neighborhood structure, thereby improving the coherence and discriminative stability of the topics output by the model.

[0051] In this embodiment, the document to be trained is input into the topic recognition model, and feature extraction is performed on the document to obtain frozen vector information. Multiple topics and words are pre-set. Based on the frozen vector information of each document to be trained, the topic embedding vector of each topic, and the word embedding vector of each word, a first transmission cost matrix and a second transmission cost matrix are determined. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word, thus introducing the embedding transmission plan algorithm and facilitating the calculation of the embedding transmission plan loss. Furthermore, the dual semantic relation reconstruction loss and structure-aware distribution alignment loss can be calculated to enable multi-dimensional training of the topic recognition model, improving the model's training accuracy and consequently, the topic recognition accuracy.

[0052] Figure 3 A flowchart illustrating the training method for a topic recognition model, as shown below. Figure 3As shown, a method for training a topic recognition model according to an embodiment of this application may include: Step 310: Input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Step 320: Determine the first transmission cost matrix based on the frozen vector information of each document to be trained and each preset topic embedding vector; Step 330: Determine the second transmission cost matrix based on each preset topic embedding vector and each preset word embedding vector; Step 340: Based on the first transmission cost matrix and the second transmission cost matrix, determine the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss; wherein, the embedding transmission plan loss represents the transmission loss from document to topic and the transmission loss from topic to word, the dual semantic relation reconstruction loss represents the loss between the actual word distribution in the document and the predicted word distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution. Step 350: Train the topic recognition model based on the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0053] Steps 310, 340 and 350 have been described in detail above and will not be repeated here. Steps 320 and 330 will be described in detail below.

[0054] In step 320, the frozen vector information participates in the calculation of the first transmission cost matrix, the preset word embedding vector participates in the calculation of the second transmission cost matrix, and the preset topic embedding vector participates in the calculation of both the first and second transmission cost matrices. This embodiment does not specifically limit the execution order of steps 320 and 330.

[0055] The elements within the first transport cost matrix quantify the semantic space difference loss between a single document and a single topic. Each row in the first transport cost matrix corresponds to a document, and each column corresponds to a topic. For example, for each training document and each topic, the semantic matching probability between the document and the topic is determined based on the frozen vector information of the training document and the topic embedding vector corresponding to the topic. This semantic matching probability is then used as an element in the first transport cost matrix. For instance, the closer the semantics between the frozen vector information of the training document and the topic embedding vector corresponding to the topic, the higher the semantic matching probability.

[0056] In step 330, the elements within the second transmission cost matrix quantify the semantic space difference loss between a single topic and a single word. Each row in the second transmission cost matrix corresponds to a topic, and each column corresponds to a word. For example, for each topic and each word, the semantic matching probability between the topic and the word is determined based on the topic embedding vector and the word embedding vector, and this semantic matching probability is used as an element in the second transmission cost matrix. For example, the closer the semantics between the topic embedding vector and the word embedding vector, the higher the semantic matching probability.

[0057] This embodiment splits the vector matching calculation process into two independent sets, generating the first transmission cost matrix and the second transmission cost matrix respectively, and completing the construction of the basic cost data required for solving the optimal transmission problem in two layers.

[0058] For example, the vector L2 distance can be used as the rule for calculating semantic loss. This involves iterating through all the frozen vector information corresponding to all the documents to be trained and all the preset topic embedding vectors, calculating the vector distance between each pair and filling the corresponding positions in the matrix to generate the first transmission cost matrix. Then, iterate through all the preset topic embedding vectors and all the preset word embedding vectors, calculating the vector distance between each pair and filling the corresponding positions in the matrix to generate the second transmission cost matrix. Besides the vector L2 distance, cosine distance, Manhattan distance, etc., can also be used as the rule for calculating the transmission loss between vectors, as long as the calculated values ​​can objectively reflect the degree of semantic difference between the two sets of vectors. The generation processes of the two cost matrices are independent of each other and share the preset topic embedding vectors as the intermediate calculation carrier, which can unify the measurement standards of the embedding space of documents, topics, and words. The two matrix calculation processes can be executed serially or in parallel synchronously with allocated computing power.

[0059] In this embodiment, by splitting the document-topic and topic-vocabulary vector matching processes into two sets, the first transmission cost matrix and the second transmission cost matrix are generated respectively. This enables hierarchical quantification of the loss standard of the two-layer semantic transmission, providing hierarchical and clear cost input for solving the embedded transmission plan loss, and ensuring that the semantic alignment calculation logic of the document, topic, and vocabulary layers is carried out in an orderly manner.

[0060] In one implementation, determining a first transmission cost matrix based on the frozen vector information of each training document and each preset topic embedding vector includes: for each training document and each topic, determining first transmission cost information from the training document to the topic based on the frozen vector information of the training document and the preset topic embedding vector corresponding to the topic; wherein the first transmission cost information is a row in the first transmission cost matrix, and the first transmission cost information represents the transmission cost from the document to the topic; and determining the first transmission cost matrix based on the first transmission cost information from each training document to each topic.

[0061] Specifically, the first transmission cost matrix comprises multiple rows, each row representing a first transmission cost information. That is, the first transmission cost matrix is ​​a two-dimensional matrix formed by concatenating the first transmission cost information corresponding to all training documents row by row. The first transmission cost information is a set of transmission cost values ​​obtained by matching each training document with all topics. Each first transmission cost information corresponds to an independent row of data within the first transmission cost matrix, capable of centrally carrying the semantic transmission loss of a single document corresponding to all topics, providing a single row of cost input for subsequent optimization of the document-to-topic transmission plan matrix through entropy regularization.

[0062] The vector distance calculation is performed on each training document and each topic category to obtain a complete set of transmission cost data corresponding to a single document, which is the first transmission cost information. Then, the first transmission cost information corresponding to all documents is concatenated in a row-ordered manner to form a complete first transmission cost matrix, thus completing the quantization and collection of all pairwise paired semantic losses on the document side and the topic side.

[0063] In this embodiment, the first transmission cost information is calculated for each document and each type of topic, and then all the first transmission cost information is integrated to generate the first transmission cost matrix. This can completely collect the semantic transmission loss of all document-topic pairings, and provide regular and hierarchical cost input data for solving the optimal transmission of document-to-topic branches.

[0064] In one implementation, determining the first transmission cost information from the document to the topic based on the frozen vector information of the document to be trained and the preset topic embedding vector corresponding to the topic includes: compressing the frozen vector information of the document to be trained using a preset nonlinear projection head to obtain the latent vector information of the document to be trained; wherein the latent vector information represents a low-dimensional vector mapped from the frozen vector information; and determining the first transmission cost information from the document to the topic based on the latent vector information of the document to be trained and the preset topic embedding vector corresponding to the topic.

[0065] Specifically, the frozen vector information is a semantic vector generated by a long-context pre-trained embedding model after encoding the document to be trained. Its parameters are fixed and it does not participate in gradient updates. This vector serves as the original input data for a pre-defined nonlinear projection head. The pre-defined nonlinear projection head is a trainable mapping network structure built from a multilayer perceptron. This pre-defined nonlinear projection head can perform nonlinear spatial mapping compression operations on the frozen vector information, eliminating redundant semantic dimensions in the original frozen vector information and generating a low-dimensional representation adapted for optimal transmission computation, thus obtaining the latent vector information of the document to be trained.

[0066] The latent vector information is a low-dimensional continuous spatial vector obtained by mapping and compressing the frozen vector information through a preset nonlinear projection head. It can adapt to the preset embedding space metric standard of the topic embedding vector, eliminate the calculation deviation caused by the dimensionality mismatch of the original frozen vector, and is specifically used to perform distance calculation with the preset topic embedding vector to generate the first transmission cost information. This embodiment does not specifically limit the architecture of the nonlinear projection head.

[0067] The preset topic embedding vector and latent vector information can maintain dimensionality consistency. The first transmission cost information is generated by pairwise operations on the latent vector information and the preset topic embedding vector. In this embodiment, a spatial compression transformation is first performed on the fixed original semantic vector of the document using a trainable nonlinear mapping structure to obtain a low-dimensional document representation that fits the topic vector space. Then, distance calculation is performed based on the transformed document representation and the topic semantic vector to quantify the semantic transmission loss of a single document corresponding to a single topic, thereby obtaining the first transmission cost information that constitutes a single row of the first transmission cost matrix.

[0068] The formula for calculating the first transmission cost information can be expressed as: ; in, The element in the i-th row and k-th column of the first transmission cost matrix represents the transmission cost from the i-th document to topic k in the first transmission cost information corresponding to the i-th document. This represents the latent vector information of the i-th document. This represents the topic embedding vector of the k-th topic.

[0069] In this embodiment, the frozen vector information is nonlinearly compressed and mapped by a preset nonlinear projection head to obtain potential vector information. Then, the first transmission cost information is calculated based on the potential vector information and the preset topic embedding vector. This can unify the vector space representation scale of the document and the topic, eliminate the semantic bias caused by the redundant dimensions of the original long text embedding vector, and improve the accuracy of the first transmission cost information in quantifying the semantic differences between the document and the topic.

[0070] In one implementation, determining a second transmission cost matrix based on each preset topic embedding vector and each preset word embedding vector includes: for each topic and each word, determining second transmission cost information from topic to word based on the preset topic embedding vector corresponding to the topic and the word embedding vector corresponding to the word; wherein the second transmission cost information is a row in the second transmission cost matrix, and the second transmission cost information represents the transmission cost from topic to word; and determining the second transmission cost matrix based on the second transmission cost information from each topic to each word.

[0071] Specifically, the second transmission cost information is a set of transmission cost values ​​obtained by matching all words for each single topic. This second transmission cost information corresponds to an independent single row of data within the second transmission cost matrix, and can centrally carry the semantic transmission loss for all words corresponding to a single topic. It provides a single row of cost input for subsequent optimal transmission solution of the topic-to-word transmission plan matrix through entropy regularization. The second transmission cost matrix is ​​a two-dimensional matrix formed by concatenating the second transmission cost information corresponding to all topics row by row. The second transmission cost matrix completely contains the transmission loss values ​​generated by pairwise pairing of all topics and all words, and serves as the complete computational input for the topic-to-word branch embedded in the transmission plan loss, supporting the subsequent optimal transmission solution process.

[0072] By traversing each topic category and each word to perform vector distance calculations, a complete set of transmission cost data corresponding to a single topic category is obtained. Then, the transmission cost data corresponding to all topics are neatly concatenated in rows to form a complete second transmission cost matrix, thus completing the quantitative collection of all pairwise paired semantic losses on the topic side and the word side.

[0073] The formula for calculating the second transmission cost information can be expressed as: ; in, The element in the k-th row and j-th column of the second transmission cost matrix represents the transmission cost from topic j to word j in the second transmission cost information corresponding to topic k. This represents the word embedding vector of the j-th word.

[0074] In this embodiment, the corresponding second transmission cost information is calculated for each topic and each word, and then all the second transmission cost information is integrated to generate the second transmission cost matrix. This can completely collect the semantic transmission loss of all topic and word pairings, and provide regular and hierarchical cost input data for solving the optimal transmission of topic to word branches.

[0075] In this embodiment, the document to be trained is input into the topic recognition model, and feature extraction is performed on the document to obtain frozen vector information. Multiple topics and words are pre-set. Based on the frozen vector information of each document to be trained, the topic embedding vector of each topic, and the word embedding vector of each word, a first transmission cost matrix and a second transmission cost matrix are determined. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word, thus introducing the embedding transmission plan algorithm and facilitating the calculation of the embedding transmission plan loss. Furthermore, the dual semantic relation reconstruction loss and structure-aware distribution alignment loss can be calculated to enable multi-dimensional training of the topic recognition model, improving the model's training accuracy and consequently, the topic recognition accuracy.

[0076] Figure 4 A flowchart illustrating the training method for a topic recognition model, as shown below. Figure 4 As shown, a method for training a topic recognition model according to an embodiment of this application may include: Step 410: Input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Step 420: Determine the first transmission cost matrix and the second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. Step 430: Based on the first transmission cost matrix and a preset entropy regularization optimal transmission algorithm, determine the first transmission plan matrix; and based on the second transmission cost matrix and a preset entropy regularization optimal transmission algorithm, determine the second transmission plan matrix; wherein, the first transmission plan matrix represents the transmission plan from document to topic, and the second transmission plan matrix represents the transmission plan from topic to vocabulary. Step 440: Determine the embedded transmission plan loss based on the first transmission cost matrix, the second transmission cost matrix, the first transmission plan matrix, and the second transmission plan matrix; Step 450: Determine the topic distribution matrix according to the first transmission plan matrix, and determine the word distribution matrix according to the second transmission plan matrix; wherein, the rows in the topic distribution matrix represent topic distribution vectors, and the topic distribution vectors represent the matching probability of a document with each topic; the rows in the word distribution matrix represent word distribution vectors, and the word distribution vectors represent the matching probability of a topic with each word. Step 460: Determine the dual semantic relation reconstruction loss based on the topic distribution matrix, word distribution matrix, and the preset bag-of-words vectors for each document to be trained; wherein, the preset bag-of-words vectors represent the actual word distribution in the document; Step 470: Determine the structure-aware distribution alignment loss based on the frozen vector information and topic distribution matrix of each document to be trained; Step 480: Train the topic recognition model based on the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0077] Steps 410, 420, and 480 have been described in detail above and will not be repeated here. Steps 430-470 will be described in detail below.

[0078] In step 430, the first transmission cost matrix is ​​a two-dimensional matrix generated by pairwise operations on document-side and topic-side vectors, representing the semantic loss values ​​from all documents to topics. This matrix provides complete loss data for document-to-topic matching for the entropy-regularized optimal transmission operation. The second transmission cost matrix is ​​a two-dimensional matrix generated by pairwise operations on topic-side and word-side vectors, representing the semantic loss values ​​from all topics to words. This matrix provides complete loss data for topic-to-word matching for the entropy-regularized optimal transmission operation.

[0079] The Embedding Transport Plan (ETP) is introduced, which solves for the optimal transport plan matrix using entropy-regularized optimal transport (Sinkhorn algorithm). In other words, an entropy-regularized optimal transmission algorithm is pre-set, which can output a transmission plan matrix based on the transmission cost matrix. In this embodiment, the process of outputting the transmission plan matrix from the transmission cost matrix is ​​not specifically limited.

[0080] A first transmission plan matrix and a second transmission plan matrix can be determined separately. The first transmission plan matrix is ​​a two-dimensional weight matrix obtained by solving the first transmission cost matrix using a preset entropy-regularized optimal transmission algorithm, and it participates in the calculation of the document-to-topic branch loss in the embedded transmission plan loss. The second transmission plan matrix is ​​a two-dimensional weight matrix obtained by solving the second transmission cost matrix using a preset entropy-regularized optimal transmission algorithm, and it participates in the calculation of the topic-to-vocabulary branch loss in the embedded transmission plan loss.

[0081] In step 440, the embedding transport plan loss is a comprehensive loss term obtained by superimposing the losses of the two transport branches from document to topic and from topic to vocabulary. It can simultaneously constrain the semantic representations of document, topic, and vocabulary to tend to be unified and aligned, providing a basic semantic space optimization target for the topic recognition model.

[0082] The formula for calculating the embedded transmission plan loss can be characterized as: ; Among them, L ETP Characterizing the embedded transmission plan loss, L OT Optimal TransportLoss is intermediate data used to calculate the loss of the embedded transport plan. This is the document-to-topic transmission cost matrix, also known as the first transmission cost matrix. The topic-to-vocabulary transfer cost matrix is ​​also known as the second transfer cost matrix. The document-to-topic transport plan is represented by the first transport plan matrix. The transport plan representing the topic to the vocabulary is the second transport plan matrix.

[0083] The formula for calculating the optimal transmission loss can be: ; Where C is the transmission cost matrix, which can be either the first transmission cost matrix or the second transmission cost matrix. This is a transmission plan matrix, for example, it could be a first transmission plan matrix or a second transmission plan matrix. x can represent a document number or a topic number, and y can represent a topic number or a vocabulary number. When x is a document number, y is a topic number; when x is a topic number, y is a vocabulary number. is the entropy regularization coefficient.

[0084] In step 450, the topic distribution matrix is ​​a two-dimensional matrix formed by concatenating the topic distribution vectors corresponding to all documents to be trained row-wise. Each row of the topic distribution matrix represents the probability of a single document matching various topics, i.e., the document-topic distribution. The word distribution matrix is ​​a two-dimensional matrix formed by concatenating the word distribution vectors corresponding to all topics row-wise. Each row of the word distribution matrix represents the probability of a single topic matching various words, i.e., the topic-word distribution. The word distribution matrix and the topic distribution matrix work together to reconstruct the original word distribution of the document, serving as the core input for the dual semantic relation reconstruction loss.

[0085] The topic distribution matrix can be determined based on the first transmission plan matrix. For example, for each row in the first transmission plan matrix, the topic distribution vector of the corresponding document is determined according to the transmission plan of that row. Based on the topic distribution vectors of each document, the complete topic distribution matrix is ​​obtained.

[0086] The formula for calculating the topic distribution vector can be: ; in, Let D be the topic distribution vector representing the i-th document, and let D represent the number of documents to be trained. This represents the transmission plan vector in the i-th row of the first transmission plan matrix. Each row in the transmission plan matrix is ​​a transmission plan vector and corresponds to a document to be trained.

[0087] The word distribution matrix can be determined based on the second transmission plan matrix.

[0088] The formula for calculating the word distribution matrix can be: ; in, The word distribution matrix represents the number of topics.

[0089] In step 460, the preset bag-of-words vector represents the original document's bag of words, which is vector data generated from the pre-statistical occurrence frequency of words within a single training document. The preset bag-of-words vector objectively records the document's true vocabulary composition and serves as a true benchmark for the dual semantic relation reconstruction loss, used to measure the deviation between the model's reconstructed vocabulary distribution and the actual text vocabulary distribution.

[0090] Dual Semantic-relation Reconstruction (DSR) ensures that the extracted low-dimensional topics remain consistent with the underlying objective lexical facts. The goal of DSR is to minimize the cross-entropy between the original document's bag-of-words and the reconstructed distribution. The dual semantic-relation loss is a cross-entropy loss term calculated using the topic distribution matrix, word distribution matrix, and predefined bag-of-words vectors as input. This ensures that the vocabulary distribution derived by the model closely matches the original textual information of the document, preventing topic semantics from deviating from the document's objective lexical facts.

[0091] The formula for calculating the loss of dual semantic relation reconstruction can be represented as: ; in, The predefined bag-of-words vector represents document i. This optimal transfer process not only solves the relation bias problem in traditional methods, but also enables the topic distribution θ to inherit the semantic structure information contained in the pre-trained model to a certain extent.

[0092] In step 470, the frozen vector information can be used to construct the initial nearest neighbor relationship of the document to be trained, and together with the topic distribution matrix, the neighborhood refinement, the solution of the reference distribution and the learning distribution are completed, and finally the structure-aware distribution alignment loss is generated.

[0093] Structure-aware distribution alignment loss is a regularized loss term obtained by constructing two sets of conditional distributions based on topic-consistent neighborhoods after refinement and filtering, and calculating the distribution deviation. It can correct noisy neighborhood associations with semantic mismatch in the original embedding space and optimize the local topological structure of the document's latent representation.

[0094] For example, a nearest neighbor graph can be constructed based on frozen vector information, and topic similarity and topic confidence can be calculated by combining the topic distribution matrix to complete neighborhood filtering. Reference distribution and learning distribution can be constructed based on the filtered reliable neighborhood, and structure-aware distribution alignment loss can be obtained by calculating the distribution bias.

[0095] In this embodiment, the transmission plan matrix is ​​solved by dividing the transmission cost matrix into two layers. The embedded transmission plan loss, dual semantic relationship reconstruction loss, and structure-aware distribution alignment loss are calculated step by step in sequence. This can quantify the three types of constraint objectives of document topic vocabulary semantic alignment, original text information restoration, and local topic neighborhood structure calibration in a hierarchical manner, providing a complete, hierarchical, and traceable multi-dimensional optimization loss basis for the iterative training of the topic recognition model.

[0096] In one implementation, the structure-aware distribution alignment loss is determined based on the frozen vector information and topic distribution matrix of each training document, including: determining the learning distribution and initial neighborhood set of each training document based on the frozen vector information of each training document; wherein the initial neighborhood set includes all topic neighborhood documents of the initially determined training document, and the topic neighborhood documents represent other training documents that are semantically related to the topic of the training document; the learning distribution represents the probability predicted by the model that other training documents belong to the same topic as each training document for each training document; for each training document, a reference distribution is determined based on the topic distribution vectors corresponding to each topic neighborhood document in the initial neighborhood set of the training document and the topic distribution vector corresponding to the training document; wherein the reference distribution represents the standard probability that topic neighborhood documents and the training document belong to the same topic; and the structure-aware distribution alignment loss is determined based on the learning distribution and the reference distribution.

[0097] Specifically, for each training document, the learning distribution can be determined by combining the frozen vector information of each document. The learning distribution is the conditional probability distribution learned by the model, representing the probability that the model determines other documents belong to the same topic category as the anchor document based on the current vector representation. It serves as the fitting object that the structure-aware distribution alignment loss needs to approximate to the standard distribution. For each training document, the learning distribution between that document and any other training document can be calculated. That is, the degree of topic matching between that document and other training documents can be determined.

[0098] For each document to be trained, the frozen vector information of each document can be used to determine the initial neighborhood set of that document. The initial neighborhood set is a set of documents selected based on the vector space distance of the frozen vector information. The initial neighborhood set includes other documents to be trained that are geometrically close in vector space, forming an original candidate sample pool that has not been filtered for topic consistency, providing a basis for subsequent selection of reliable topic neighborhood documents and solving the reference distribution.

[0099] The topic neighborhood document is another training document within the initial neighborhood set that has similar topic semantics to the anchor training document.

[0100] For each document to be trained, a reference distribution can be determined based on the topic distribution vectors of each topic's neighboring documents in the initial neighborhood set of the document and the topic distribution vector corresponding to the document itself. The reference distribution is a conditional probability distribution unaffected by the model's trainable parameters, representing the standard probability of topic matching between documents, and serving as a supervisory benchmark for fitting the learning distribution to the alignment. The structure-aware distribution alignment loss measures the numerical deviation between the learning distribution and the reference distribution, reducing the difference between them, correcting the mismatch between geometric distance and topic semantics in the original frozen vector space, optimizing the local topological structure of the document's latent representation, and working in conjunction with the embedding transport plan loss and the dual semantic relationship reconstruction loss to jointly optimize the topic recognition model.

[0101] The formula for calculating the structure-aware distribution alignment loss can be: ; Among them, L SADA For structure-aware distribution alignment loss, The character j represents a document other than i. Let i and j be the reference distribution. Let be the learning distribution between i and j.

[0102] In this embodiment, an initial neighborhood set and a learning distribution are constructed by freezing vector information. Then, a reference distribution is generated by filtering the topic distribution vectors. Finally, the deviation between the two distributions is measured to obtain the structure-aware distribution alignment loss. This can eliminate topic-irrelevant noise samples in the original embedding neighborhood, guide the model to learn the document latent representation that fits the real topic clustering structure, and improve the discriminative stability of document topic distribution.

[0103] In one implementation, determining the learning distribution of each training document based on its frozen vector information includes: compressing the frozen vector information of the training document using a preset nonlinear projection head to obtain the latent vector information of the training document; wherein the latent vector information represents a low-dimensional vector mapped from the frozen vector information; and determining the learning distribution of each training document based on its latent vector information.

[0104] Specifically, the preset nonlinear projection head is a trainable mapping network structure composed of a multilayer sensing mechanism. The preset nonlinear projection head can perform nonlinear dimensionality reduction and compression transformation on the frozen vector information, eliminate redundant semantic dimensions in the original frozen vector information, and output latent vector information.

[0105] Based on the latent vector information between any two documents, the learning distribution of those two documents can be calculated.

[0106] The formula for calculating the learning distribution can be: ; in, Let i be the learning distribution of document i and document j. denoted as the preset temperature coefficient, and z represents the potential vector information.

[0107] The learning distribution is a conditional probability distribution calculated based on latent vector information. It is indirectly determined by the trainable parameters of the model and can represent the probability that the model determines that the other training documents and the anchor training documents belong to the same topic based on the currently learned vector representation. It serves as the distribution to be optimized in the structure-aware distribution alignment loss that needs to be fitted and aligned with the reference distribution.

[0108] In this embodiment, the frozen vector information is nonlinearly compressed and mapped by a preset nonlinear projection head to obtain the latent vector information. Then, the learning distribution corresponding to each document to be trained is calculated based on all the latent vector information. This can unify the metric standard for document vector representation, accurately quantify the probability of document similarity to the same topic currently determined by the model, and provide an iteratively optimized fitting object for the structure-aware distribution alignment loss.

[0109] In one implementation, determining the reference distribution of the document to be trained based on the topic distribution vectors corresponding to each topic neighboring document in the initial neighborhood set of the document to be trained and the topic distribution vector corresponding to the document to be trained includes: selecting target neighboring documents from each topic neighboring document based on the topic distribution vectors corresponding to each topic neighboring document in the initial neighborhood set of the document to be trained; and determining the reference distribution of the document to be trained based on the topic distribution vectors corresponding to the target neighboring documents and the topic distribution vector corresponding to the document to be trained.

[0110] Specifically, the initial neighborhood set is generated based on the spatial distance of the frozen vector information, including other training documents that are adjacent to the vector space of a single training document. The initial neighborhood set only provides raw candidate document samples that have not undergone topic verification, providing a basic candidate range for subsequent selection of target neighborhood documents.

[0111] The topic distribution vector is taken from a single row of the topic distribution matrix and represents the probability weight of a single training document matching all topics. The topic distribution vector can quantify the topic semantic similarity between two documents, and can also calculate the topic confidence of a single document, serving as the core quantitative basis for selecting target neighboring documents and calculating reference distribution weights.

[0112] The topical semantic similarity between two documents can be represented as: ; in, The semantic similarity of topics between i and j in question-and-answer representation. Let i be the topic distribution vector. Let be the topic distribution vector of document j.

[0113] The target neighborhood documents are document samples selected from all topic-related neighborhood documents within the initial neighborhood set through a dual filtering mechanism of topic similarity and topic confidence. That is, noisy documents with close vector distances but irrelevant topic semantics can be eliminated, retaining only samples with reliable topic semantic matching to form an effective sample set for the reference distribution. For example, the basic structural weights of two documents can be calculated based on their similarity, where the basic structural weights represent the initially determined semantic matching degree between the two documents.

[0114] The formula for calculating the weight of the infrastructure can be: ; in, P represents the basic structural weights between documents i and j, where P is either 1 or 0. If document j belongs to the topic neighborhood of document i, then P... ij P is 1 if it is not 0 otherwise im Similarly.

[0115] To mitigate the noisy neighborhood relationships caused by ambiguous topic assignments, a topic confidence filtering mechanism is further introduced. Each document corresponds to its own topic confidence score c. i The topic distribution vector includes the probability values ​​of a document belonging to each topic, and the topic confidence can be the maximum probability value in the topic distribution vector of a document.

[0116] A binary mask M is constructed based on topic confidence, with a pre-set confidence threshold. The difference between the topic confidence of document i and document j is calculated. If this difference is greater than the pre-set confidence threshold, M is set to 1; otherwise, it is set to 0. For each document i to be trained, the target structure weights of document i and j can be determined based on the basic structure weights of document i and j and the corresponding binary mask M.

[0117] The formula for calculating the target structure weights can be: ; in, For the target structure weights, This is a binary mask between documents i and j.

[0118] For each document i to be trained, the target structure weights between document i and each document j are sorted, and the top H documents j are taken as target neighborhood documents. Target neighborhood documents are documents in the topic neighborhood documents.

[0119] For each training document, there is a corresponding reference distribution between it and any other training document. The reference distribution between this training document and any other training document (excluding target neighborhood documents) can be directly set to 0. Therefore, it is necessary to determine the reference distribution between this training document and each target neighborhood document. The reference distribution is the conditional probability distribution obtained by normalizing the target structure weights of samples within the target neighborhood documents, and serves as the standard supervision benchmark for learning distribution alignment and fitting. The calculation process for the reference distribution between the training document and the target neighborhood documents can be achieved by dividing the target structure weight of the target neighborhood document by the sum of the target structure weights of all corresponding target neighborhood documents.

[0120] In this embodiment, by filtering the topic neighborhood documents in the initial neighborhood set based on the topic distribution vector, the target neighborhood documents are obtained. Then, the reference distribution is calculated based on the topic matching weight of the target neighborhood documents. This can eliminate topic-irrelevant noise samples in the original embedded neighborhood and generate a standard probability benchmark that fits the real risk topic clustering relationship, providing a reliable alignment supervision signal for the structure-aware distribution alignment loss.

[0121] In this embodiment, the document to be trained is input into the topic recognition model, and feature extraction is performed on the document to obtain frozen vector information. Multiple topics and words are pre-set. Based on the frozen vector information of each document to be trained, the topic embedding vector of each topic, and the word embedding vector of each word, a first transmission cost matrix and a second transmission cost matrix are determined. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word, thus introducing the embedding transmission plan algorithm and facilitating the calculation of the embedding transmission plan loss. Furthermore, the dual semantic relation reconstruction loss and structure-aware distribution alignment loss can be calculated to enable multi-dimensional training of the topic recognition model, improving the model's training accuracy and consequently, the topic recognition accuracy.

[0122] Figure 5 A flowchart illustrating the training method for a topic recognition model, as shown below. Figure 5 As shown, a method for training a topic recognition model according to an embodiment of this application may include: Step 510: Input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Step 520: Determine the first transmission cost matrix and the second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. Step 530: Determine the embedded transmission plan loss and the dual semantic relation reconstruction loss based on the first transmission cost matrix and the second transmission cost matrix; Step 540: Add the embedded transmission plan loss and the dual semantic relation reconstruction loss together to obtain the basic modeling loss; Step 550: Train the topic recognition model using the basic modeling loss to obtain the initial training model; Step 560: Input the document to be trained into the initial training model to obtain the new frozen vector information of the document to be trained; Step 570: Determine the new first transmission cost matrix and the new second transmission cost matrix based on the new frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector. Step 580: Based on the new first transmission cost matrix and the new second transmission cost matrix, determine the new embedded transmission plan loss, the new dual semantic relation reconstruction loss, and the new structure-aware distribution alignment loss; Step 590: Perform a weighted summation of the new embedding transport plan loss, the new dual semantic relation reconstruction loss, and the new structure-aware distribution alignment loss to train the initial training model.

[0123] Steps 510-530 have been described in detail above and will not be repeated here. Steps 540-590 will be described in detail below.

[0124] In step 540, since the self-guided reference distribution depends on the document-topic distribution learned by the model, i.e., the topic distribution matrix, directly introducing structure-aware distribution alignment loss in the early stages of training could easily introduce unstable neighborhood signals. Therefore, this embodiment adopts a two-stage training strategy. In the first stage, the model only uses the basic loss function for optimization. The first stage is a warm-up stage, which enables the model to learn preliminary document-topic distribution and topic-word distribution, providing a foundation for subsequently building reliable topic-guided neighborhoods.

[0125] The basic loss function can refer to the embedding transport plan loss and the dual semantic relation reconstruction loss. The addition process directly superimposes these two types of basic loss terms, integrating document, topic, and lexical semantic alignment constraints with the original text restoration constraints to generate a unified single-stage warm-up optimization objective. The basic modeling loss is the comprehensive loss term obtained after adding the embedding transport plan loss and the dual semantic relation reconstruction loss. It can drive the model to prioritize learning stable and reliable document topic distributions and topic lexical distributions, providing reliable intermediate representation data for the subsequent introduction of distribution alignment constraints.

[0126] ; Among them, L TM Characterize the basic modeling loss.

[0127] In step 550, the initial training model is a pre-trained model obtained after multiple rounds of iterative updates based on the basic modeling loss. The topic distribution matrix and word distribution matrix inside the initial training model have stable topic semantic discrimination effects and can output reliable document topic representations, supporting the complete construction of subsequent neighborhood refinement and reference distribution.

[0128] ; Among them, L stage1 The loss function represents the first stage. The initial topic recognition model is trained using the basic modeling loss; this first stage of training yields the initial trained model.

[0129] In step 560, after completing the first stage of pre-training, the model enters the second stage, performing self-guided neighborhood refinement and structure-aware distribution alignment. This embodiment utilizes a pre-built KNN (K-Nearest Neighbors) graph, topic similarity, and confidence masks to calculate the refined target structure weights and construct a reference distribution on the calibrated target neighborhood documents. Subsequently, the model is jointly optimized using the basic modeling loss and the SADA regularization term.

[0130] In other words, the initial training model is used as the new model to be trained and retrained. Features are extracted from the training document using the initial training model to obtain new frozen vector information for the training document. The processing procedure in step 560 is the same as that in step 210.

[0131] In step 570, the new first transmission cost matrix and the new second transmission cost matrix correspond to the document-to-topic and topic-to-vocabulary semantic loss matrices recalculated after the initial training model iterations, respectively. The processing procedure of step 570 is the same as that of step 220.

[0132] In step 580, based on the two new types of transmission cost matrices and the initial training model, three types of losses can be obtained: embedding transmission plan loss, dual semantic relation reconstruction loss, and structure-aware distribution alignment loss. The new embedding transmission plan loss and the new dual semantic relation reconstruction loss are basic modeling-related loss terms obtained by resolving the two new sets of transmission cost matrices and transmission plan matrices, which can adapt to the updated semantic representation state of the initial training model. The new structure-aware distribution alignment loss is a regularization constraint loss term obtained by reconstructing the neighborhood and two sets of conditional distributions based on the stable topic distribution matrix output by the initial training model and frozen vector information, used to correct the neighborhood semantic mismatch problem in the potential vector space of the initial training model. The processing procedure in step 580 is the same as in step 230.

[0133] In step 590, the three new loss terms are weighted and summed to balance the influence of the three constraints of semantic alignment, text restoration, and neighborhood structure calibration on the parameter update of the initial training model, thereby achieving multi-objective synchronous joint optimization.

[0134] ; Among them, L stage2 The loss function for the second stage, These are preset weights.

[0135] A two-stage, hierarchical training logic is employed to train the topic recognition model. In the first stage, only the embedding transport plan loss and the dual semantic relationship reconstruction loss are added to obtain the basic modeling loss. Based on this basic modeling loss, iterative training generates an initial training model with stable topic representation capabilities. In the second stage, the original document semantic vectors are reused to regenerate the complete cost matrix and three new loss terms. Weighted summation is then used to integrate these three new loss terms, simultaneously optimizing the initial training model and completing the fine-tuning of all model parameters. First, provide the model with a coarse-grained topic structure, and Then, the local geometry of the document representation is further optimized based on the neighborhood relationship with consistent themes.

[0136] In this embodiment, by first training an initial training model based on the basic modeling loss in stages, and then recalculating the entire set of loss terms based on the initial training model and performing weighted summation and joint optimization, a stable and reliable semantic foundation for document topics can be established first, and then the neighborhood semantic noise brought by the original embedding space can be corrected, taking into account both the spatial distinction effect of topic coherence and the potential representation of documents.

[0137] Figure 6 This is a diagram illustrating the overall framework for model training. Figure 6 In the first stage, square boxes represent the latent vector information of the document, triangular boxes represent the topic embedding vector of the topic, and circular boxes represent the word embedding vector of the vocabulary.

[0138] In this embodiment, the document to be trained is input into the topic recognition model, and feature extraction is performed on the document to obtain frozen vector information. Multiple topics and words are pre-set. Based on the frozen vector information of each document to be trained, the topic embedding vector of each topic, and the word embedding vector of each word, a first transmission cost matrix and a second transmission cost matrix are determined. The first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word, thus introducing the embedding transmission plan algorithm and facilitating the calculation of the embedding transmission plan loss. Furthermore, the dual semantic relation reconstruction loss and structure-aware distribution alignment loss can be calculated to enable multi-dimensional training of the topic recognition model, improving the model's training accuracy and consequently, the topic recognition accuracy.

[0139] Figure 7 A flowchart illustrating the document topic identification method, such as... Figure 7 As shown, a document topic identification method according to one embodiment of this application may include: Step 710: Obtain the document to be recognized; Step 720: Input the document to be identified into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; Step 730: Determine the topic corresponding to the document to be identified based on the topic distribution vector of the document to be identified.

[0140] Steps 710-730 are described in detail below.

[0141] In step 710, the document to be identified can be a customs inspection record text that has undergone de-identification and cleaning processing and needs to be determined to belong to a risk theme. The document to be identified is the original input object of the theme identification process, and the complete text content will be sent to the theme identification model to complete encoding and theme inference calculations.

[0142] In step 720, the topic recognition model in this embodiment can be a SADA Topic model. This model integrates a long text embedding encoding module, a nonlinear projection mapping structure, and an optimal transport semantic alignment module. It outputs stable and interpretable topic distribution vectors and serves as the core computational carrier for document topic inference. The long text embedding encoding module generates frozen vector information; the nonlinear projection mapping structure is the nonlinear projection head of a multilayer perceptron, used to generate latent vector information; and the optimal transport semantic alignment module generates topic distribution vectors and word distribution vectors. Inputting the document to be recognized into the topic recognition model yields the topic distribution vector and word distribution vector of the document.

[0143] The topic distribution vector represents the probability weight values ​​of a single document to be identified matching all preset topics. It can quantify the degree of association between a document and each potential risk topic. On the one hand, it is used to directly determine the core topic to which the document belongs; on the other hand, it can be output for downstream business processes such as risk label-assisted identification, similar case retrieval, and topic evidence interpretation. The word distribution vector can include the probability weight values ​​between each topic and each word. Alternatively, it can first determine the topic with the highest relevance to the document to be identified based on the topic distribution vector, and then generate the word distribution vector for that topic, i.e., only the probability weight values ​​between that topic and each word are generated. In the process of model application, the words used to generate the word distribution vector can be keywords extracted from the document to be identified. This embodiment does not specifically limit the method of extracting keywords.

[0144] In step 730, the topic distribution vector and word distribution vector of the document to be identified can be obtained, as well as information such as risk records similar to the document to be identified. Combining this information, the topic corresponding to the document is determined. For example, the topic corresponding to the document can be a target risk topic selected based on the probability weights in the topic distribution vector. The topic corresponding to the document represents the core violation risk category in the document to be identified, providing standardized classification results for customs risk assessment, manual review, and labeling.

[0145] In this embodiment, the topic distribution vector is obtained by inputting the document to be identified into the topic recognition model obtained through two-stage hierarchical training. Then, the topic to which the document belongs is determined based on the topic distribution vector. This can stably output document classification results that fit the semantics of real regulatory risks, and provide interpretable intermediate topic representations for risk assessment, manual review, and risk labeling of customs inspection records.

[0146] This embodiment employs Long Context LLM (Large Language Model) embedding to encode long customs inspection texts holistically, mitigating the semantic fragmentation issues caused by truncation or segmentation in long record processing using traditional sentence vector models. By optimizing transmission, it simultaneously models the semantic relationships between document-topic and topic-word relationships, placing inspection records, potential risk topics, and topic keywords within a unified interpretable space. A self-guided neighborhood refinement strategy is proposed, utilizing frozen LLM embeddings to construct an initial KNN graph, then combining model-learned topic consistency and topic confidence filtering to select more reliable topic-consistent local structures from the original retrieval neighborhood. A structure-aware distribution alignment objective is proposed, transforming the refined topic-consistent neighborhood into a reference distribution and constraining the learned neighborhood conditional distribution to match it, thereby mitigating the inconsistency between retrieval-guided neighborhoods and topic neighborhoods. A two-stage training strategy is adopted, introducing structure-aware alignment after the basic topic structure stabilizes, reducing the impact of unstable topic distributions in the early training stages on neighborhood construction. In the scenario of risk identification of customs inspection records, this embodiment can serve as the intermediate layer of the subject structure of the risk label identification system, providing support for the differentiation of similar labels, comparison of historical cases, explanation of subject keywords, and manual review.

[0147] This embodiment's topic recognition model was tested on four real-world datasets to evaluate the structure-aware distribution alignment method. These datasets are 20NG, NYT, ACL, and NeurIPS. Different models were selected as baseline methods for comparison, including BERTopic (BERT-based Topic Model), ECRTM (Embedding Convex Regularized Topic Model), FASTopic (Fast Adaptive Transferable Topic Model), and CGTM (Correlated Graph Topic Model). Specific statistical information for each dataset is shown in Table 1. The experiments primarily validated the effectiveness of the topic recognition model on publicly available topic modeling datasets, demonstrating its improvements in topic coherence, topic diversity, and document-topic distribution differentiation. Table 1. Datasets and their basic characteristics

[0148] The model is evaluated using Topic Quality (TQ), Topic Coherence (TC), and Topic Diversity (TD). TC measures the semantic consistency of the first 15 words for each topic; TD calculates the proportion of unique words among the first 15 words for each topic; TD is defined as the product of TC and TD, capturing the coherence and diversity of the learned topics. Tables 2 and 3 illustrate the performance and characteristics of the model algorithm in this embodiment. For document-topic distribution, a classification-based evaluation method is used, employing NMI (Normalized Mutual Information) and Purity to measure the effectiveness of the model in clustering documents under dominant topics.

[0149] In the evaluation of four benchmark datasets, the topic recognition model SADA Topic in this embodiment showed good performance on multiple metrics, as shown in Table 2, with 200 topics.

[0150] Table 2. Indicator information under different modeling methods

[0151] While BERTopic and CGTM occasionally achieve higher topic consistency scores on specific datasets, their topic diversity is significantly reduced, leading to a decline in topic quality. Compared to existing baseline models, the SADA Topic model effectively addresses this issue, demonstrating the effectiveness of structure-aware distribution alignment mechanisms. Optimizing topic-consistent neighborhoods and aligning local structural distributions can mitigate the interference of retrieval biases on document relationships to some extent, thereby improving topic diversity and discriminability.

[0152] Table 3 Clustering evaluation of the dataset under different models

[0153] The SADA Topic model achieves good results on most clustering metrics, including Purity and NMI, as shown in Table 3. This indicates that the SADA Topic model can improve the discriminative power of document-topic distributions to a certain extent. Existing neural topic models generally ignore the fine-grained relationships between adjacent documents, making the inference process susceptible to noise interference. In contrast, the SADA Topic model extracts more reliable local semantic topologies by fusing KNN graphs with latent topic consistency filtering.

[0154] Table 4 Ablation Experiment

[0155] Table 4 shows that the SADA module's impact varies across different datasets. On the NeurIPS, ACL, and NYT datasets, the full model achieves higher topic quality compared to the version without SADA, indicating that the structure-aware distribution alignment mechanism can improve topic representation and enhance the overall performance of topic coherence and diversity in most scenarios. Meanwhile, on the 20NG dataset, the TQ after removing SADA is slightly higher than the full model, suggesting that the module's performance may be affected by dataset size, text length, category structure, and topic distribution characteristics. Further observation reveals that the version without SADA on 20NG primarily exhibits higher topic diversity, while the full model, after introducing structural constraints, emphasizes the preservation of consistent local relationships within the topic. Therefore, the role of SADA is not simply to improve a single metric, but rather to guide the model to learn more stable and topic-consistent underlying structures while mitigating noise in the original embedding neighborhood.

[0156] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0157] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.

[0158] Description of apparatus and devices in embodiments of this application Reference Figure 8 , Figure 8This is a schematic diagram of the structure of a training device 800 for a topic recognition model provided in an embodiment of this application. The training device 800 for the topic recognition model includes: The document input module 801 is used to input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; The matrix determination module 802 is used to determine a first transmission cost matrix and a second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. The loss determination module 803 is used to determine the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss based on the first transmission cost matrix and the second transmission cost matrix; wherein, the embedding transmission plan loss represents the transmission loss from document to topic and the transmission loss from topic to word, the dual semantic relation reconstruction loss represents the loss between the actual word distribution in the document and the predicted word distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution. Model training module 804 is used to train the topic recognition model based on the embedding transport plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0159] In one example, the matrix determination module 802 includes: The first determining unit is used to determine the first transmission cost matrix based on the frozen vector information of each document to be trained and each preset topic embedding vector. The second determining unit is used to determine the second transmission cost matrix based on each preset topic embedding vector and each preset word embedding vector.

[0160] In one example, the first determining unit is specifically used for: For each document to be trained and each topic, the first transmission cost information from the document to the topic is determined based on the frozen vector information of the document to be trained and the preset topic embedding vector corresponding to the topic; wherein, the first transmission cost information is a row in the first transmission cost matrix, and the first transmission cost information represents the transmission cost from the document to the topic. The first transmission cost matrix is ​​determined based on the first transmission cost information from each document to each topic.

[0161] In one example, the first determining unit is specifically used for: The frozen vector information of the document to be trained is compressed using a pre-defined nonlinear projection head to obtain the latent vector information of the document to be trained; wherein, the latent vector information represents the low-dimensional vector mapped from the frozen vector information. Based on the latent vector information of the document to be trained and the preset topic embedding vector corresponding to the topic, the first transmission cost information from the document to be trained to the topic is determined.

[0162] In one example, the second determining unit is specifically used for: For each topic and each word, the second transmission cost information from topic to word is determined based on the preset topic embedding vector corresponding to the topic and the word embedding vector corresponding to the word; wherein, the second transmission cost information is a row in the second transmission cost matrix, and the second transmission cost information represents the transmission cost from topic to word; The second transmission cost matrix is ​​determined based on the second transmission cost information from each topic to each word.

[0163] In one example, the loss determination module 803 includes: The plan matrix determination unit is used to determine a first transmission plan matrix based on a first transmission cost matrix and a preset entropy regularization optimal transmission algorithm, and to determine a second transmission plan matrix based on a second transmission cost matrix and a preset entropy regularization optimal transmission algorithm; wherein, the first transmission plan matrix represents the document-to-topic transmission plan, and the second transmission plan matrix represents the topic-to-vocabulary transmission plan. The first loss determination unit is used to determine the embedded transmission plan loss based on the first transmission cost matrix, the second transmission cost matrix, the first transmission plan matrix, and the second transmission plan matrix. The distribution matrix determination unit is used to determine the topic distribution matrix according to the first transmission plan matrix and the word distribution matrix according to the second transmission plan matrix; wherein, the rows in the topic distribution matrix represent topic distribution vectors, the topic distribution vectors represent the matching probability of documents with each topic, and the rows in the word distribution matrix represent word distribution vectors, the word distribution vectors represent the matching probability of topics with each word. The second loss determination unit is used to determine the dual semantic relation reconstruction loss based on the topic distribution matrix, the word distribution matrix, and the preset bag-of-words vectors of each document to be trained; wherein the preset bag-of-words vectors represent the actual word distribution in the document; The third loss determination unit is used to determine the structure-aware distribution alignment loss based on the frozen vector information and topic distribution matrix of each document to be trained.

[0164] In one example, the third loss determination unit is specifically used for: Based on the frozen vector information of each training document, the learning distribution and initial neighborhood set of each training document are determined. The initial neighborhood set includes all topic neighborhood documents of the initially determined training document. The topic neighborhood documents represent other training documents that are semantically related to the topic of the training document. The learning distribution represents the probability predicted by the model that other training documents belong to the same topic as each training document for each training document. For each document to be trained, the reference distribution of the document to be trained is determined based on the topic distribution vectors of each topic neighborhood document in the initial neighborhood set of the document to be trained and the topic distribution vector of the document to be trained; wherein, the reference distribution represents the standard probability that the topic neighborhood documents and the document to be trained belong to the same topic. The structure-aware distribution alignment loss is determined based on the learning distribution and the reference distribution.

[0165] In one example, the third loss determination unit is specifically used for: The frozen vector information of the document to be trained is compressed using a pre-defined nonlinear projection head to obtain the latent vector information of the document to be trained; wherein, the latent vector information represents the low-dimensional vector mapped from the frozen vector information. The learning distribution of each training document is determined based on its latent vector information.

[0166] In one example, the third loss determination unit is specifically used for: Based on the topic distribution vectors corresponding to each topic neighborhood document in the initial neighborhood set of the document to be trained, the target neighborhood documents are selected from each topic neighborhood document; The reference distribution of the document to be trained is determined based on the topic distribution vectors corresponding to the target neighborhood documents and the topic distribution vectors corresponding to the documents to be trained.

[0167] In one example, model training module 804 is specifically used for: The topic recognition model is trained by weighting and summing the embedded transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

[0168] In one example, it also includes: The basic determination module is used to add the embedded transmission plan loss and the dual semantic relation reconstruction loss together after obtaining them to obtain the basic modeling loss. The first training module is used to train the topic recognition model using the basic modeling loss to obtain the initial training model; The vector freezing module is used to input the document to be trained into the initial training model to obtain new frozen vector information of the document to be trained. The vector calculation module is used to determine the new first transmission cost matrix and the new second transmission cost matrix based on the new frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector. The loss update module is used to determine the new embedded transmission plan loss, the new dual semantic relation reconstruction loss, and the new structure-aware distribution alignment loss based on the new first transmission cost matrix and the new second transmission cost matrix. The second training module is used to perform weighted summation of the new embedding transport plan loss, the new dual semantic relation reconstruction loss, and the new structure-aware distribution alignment loss to train the initial training model.

[0169] Reference Figure 9 , Figure 9 This is a schematic diagram of the structure of a document topic identification device 900 provided in an embodiment of this application. The document topic identification device 900 includes: Document acquisition module 901 is used to acquire the document to be recognized; The vector output module 902 is used to input the document to be identified into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; The topic determination module 903 is used to determine the topic corresponding to the document to be identified based on the topic distribution vector of the document to be identified.

[0170] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0171] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0172] Figure 10This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 1000 of this embodiment includes: at least one processor 1001 ( Figure 10 The above describes the implementation of the training method for any of the above-described topic recognition models and the steps in the document topic recognition method embodiments. (Only one is shown in the image.)

[0173] Electronic device 1000 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. This electronic device may include, but is not limited to, a processor 1001 and a memory 1002. Those skilled in the art will understand that... Figure 10 This is merely an example of electronic device 1000 and does not constitute a limitation on electronic device 1000. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0174] The processor 1001 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0175] In some embodiments, the memory 1002 may be an internal storage unit of the electronic device 1000, such as a hard disk or memory of the electronic device 1000. In other embodiments, the memory 1002 may be an external storage device of the electronic device 1000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 1000. Further, the memory 1002 may include both internal and external storage units of the electronic device 1000. The memory 1002 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of computer programs. The memory 1002 can also be used to temporarily store data that has been output or will be output.

[0176] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0178] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0180] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0181] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for training a topic recognition model, characterized in that, include: The document to be trained is input into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; Based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector, a first transmission cost matrix and a second transmission cost matrix are determined; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. Based on the first transmission cost matrix and the second transmission cost matrix, the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss are determined; wherein, the embedding transmission plan loss represents the document-to-topic transmission loss and the topic-to-vocabulary transmission loss, the dual semantic relation reconstruction loss represents the loss between the actual vocabulary distribution in the document and the predicted vocabulary distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution. The topic recognition model is trained based on the embedded transport plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

2. The method according to claim 1, characterized in that, The step of determining the first transmission cost matrix and the second transmission cost matrix based on the frozen vector information of each of the documents to be trained, each preset topic embedding vector, and each preset word embedding vector includes: The first transmission cost matrix is ​​determined based on the frozen vector information of each document to be trained and each preset topic embedding vector; The second transmission cost matrix is ​​determined based on each preset topic embedding vector and each preset word embedding vector.

3. The method according to claim 2, characterized in that, The step of determining the first transmission cost matrix based on the frozen vector information of each of the documents to be trained and each preset topic embedding vector includes: For each training document and each topic, the first transmission cost information from the training document to the topic is determined based on the frozen vector information of the training document and the preset topic embedding vector corresponding to the topic; wherein, the first transmission cost information is a row in the first transmission cost matrix, and the first transmission cost information represents the transmission cost from the document to the topic. The first transmission cost matrix is ​​determined based on the first transmission cost information from each of the training documents to each of the topics.

4. The method according to claim 3, characterized in that, The step of determining the first transmission cost information from the document to the topic based on the frozen vector information of the document to be trained and the preset topic embedding vector corresponding to the topic includes: The frozen vector information of the document to be trained is compressed using a preset nonlinear projection head to obtain the latent vector information of the document to be trained; wherein the latent vector information represents a low-dimensional vector mapped from the frozen vector information. Based on the latent vector information of the document to be trained and the preset topic embedding vector corresponding to the topic, the first transmission cost information from the document to be trained to the topic is determined.

5. The method according to claim 2, characterized in that, The step of determining the second transmission cost matrix based on each preset topic embedding vector and each preset word embedding vector includes: For each topic and each word, based on the preset topic embedding vector corresponding to the topic and the word embedding vector corresponding to the word, the second transmission cost information from the topic to the word is determined; wherein, the second transmission cost information is a row in the second transmission cost matrix, and the second transmission cost information represents the transmission cost from the topic to the word; The second transmission cost matrix is ​​determined based on the second transmission cost information from each of the topics to each of the words.

6. The method according to claim 1, characterized in that, The step of determining the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss based on the first transmission cost matrix and the second transmission cost matrix includes: Based on the first transmission cost matrix and a preset entropy regularization optimal transmission algorithm, a first transmission plan matrix is ​​determined, and based on the second transmission cost matrix and a preset entropy regularization optimal transmission algorithm, a second transmission plan matrix is ​​determined; wherein, the first transmission plan matrix represents the document-to-topic transmission plan, and the second transmission plan matrix represents the topic-to-vocabulary transmission plan. The embedded transmission plan loss is determined based on the first transmission cost matrix, the second transmission cost matrix, the first transmission plan matrix, and the second transmission plan matrix. Based on the first transmission plan matrix, a topic distribution matrix is ​​determined, and based on the second transmission plan matrix, a word distribution matrix is ​​determined; wherein, the rows in the topic distribution matrix represent topic distribution vectors, and the topic distribution vectors represent the matching probability of a document with each topic; the rows in the word distribution matrix represent word distribution vectors, and the word distribution vectors represent the matching probability of a topic with each word. The dual semantic relation reconstruction loss is determined based on the topic distribution matrix, the word distribution matrix, and the preset bag-of-words vectors of each of the documents to be trained; wherein the preset bag-of-words vectors represent the actual word distribution in the document. The structure-aware distribution alignment loss is determined based on the frozen vector information of each document to be trained and the topic distribution matrix.

7. The method according to claim 6, characterized in that, The step of determining the structure-aware distribution alignment loss based on the frozen vector information of each of the training documents and the topic distribution matrix includes: Based on the frozen vector information of each training document, the learning distribution and initial neighborhood set of each training document are determined; wherein, the initial neighborhood set includes all topic neighborhood documents of the initially determined training document, and the topic neighborhood documents represent other training documents that are semantically related to the topic of the training document; the learning distribution represents the probability predicted by the model that other training documents belong to the same topic as each training document for each training document; For each document to be trained, a reference distribution is determined based on the topic distribution vectors corresponding to each topic neighboring document in the initial neighborhood set of the document to be trained and the topic distribution vector corresponding to the document to be trained; wherein, the reference distribution represents the standard probability that the topic neighboring documents and the document to be trained belong to the same topic. The structure-aware distribution alignment loss is determined based on the learning distribution and the reference distribution.

8. The method according to claim 7, characterized in that, Determining the learning distribution of each training document based on its frozen vector information includes: The frozen vector information of the document to be trained is compressed using a preset nonlinear projection head to obtain the latent vector information of the document to be trained; wherein the latent vector information represents a low-dimensional vector mapped from the frozen vector information. The learning distribution of each training document is determined based on the latent vector information of each training document.

9. The method according to claim 7, characterized in that, The step of determining the reference distribution of the document to be trained based on the topic distribution vectors corresponding to each topic's neighboring documents in the initial neighborhood set of the document to be trained and the topic distribution vector corresponding to the document to be trained includes: Based on the topic distribution vectors corresponding to each topic neighborhood document in the initial neighborhood set of the document to be trained, target neighborhood documents are selected from each topic neighborhood document; The reference distribution of the document to be trained is determined based on the topic distribution vectors corresponding to the target neighborhood documents and the topic distribution vectors corresponding to the document to be trained.

10. The method according to claim 1, characterized in that, The step of training the topic recognition model based on the embedded transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss includes: The topic recognition model is trained by weighting and summing the embedded transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: After obtaining the embedded transmission plan loss and the dual semantic relation reconstruction loss, the embedded transmission plan loss and the dual semantic relation reconstruction loss are added together to obtain the basic modeling loss; The topic recognition model is trained using the basic modeling loss to obtain an initial training model; The document to be trained is input into the initial training model to obtain new frozen vector information of the document to be trained; Based on the new frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector, a new first transmission cost matrix and a new second transmission cost matrix are determined. Based on the new first transmission cost matrix and the new second transmission cost matrix, a new embedded transmission plan loss, a new dual semantic relation reconstruction loss, and a new structure-aware distribution alignment loss are determined. The new embedding transport plan loss, the new dual semantic relation reconstruction loss, and the new structure-aware distribution alignment loss are weighted and summed to train the initial training model.

12. A method for identifying document topics, characterized in that, include: Obtain the document to be recognized; The document to be identified is input into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; The topic corresponding to the document to be identified is determined based on the topic distribution vector of the document to be identified; wherein the topic identification model is the topic identification model according to any one of claims 1-11.

13. A training device for a topic recognition model, characterized in that, include: The document input module is used to input the document to be trained into the topic recognition model to obtain the frozen vector information of the document to be trained; wherein, the frozen vector information is a vector representing the overall semantics of the document; The matrix determination module is used to determine a first transmission cost matrix and a second transmission cost matrix based on the frozen vector information of each document to be trained, each preset topic embedding vector, and each preset word embedding vector; wherein, the first transmission cost matrix represents the transmission cost from each document to each topic, and the second transmission cost matrix represents the transmission cost from each topic to each word; the preset topic embedding vector represents the semantic features of the topic, and the preset word embedding vector represents the semantic features of the word. The loss determination module is used to determine the embedding transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss based on the first transmission cost matrix and the second transmission cost matrix; wherein, the embedding transmission plan loss represents the document-to-topic transmission loss and the topic-to-vocabulary transmission loss, the dual semantic relation reconstruction loss represents the loss between the actual vocabulary distribution in the document and the predicted vocabulary distribution, and the structure-aware distribution alignment loss represents the alignment loss between the document reference distribution and the learned distribution; The model training module is used to train the topic recognition model based on the embedded transmission plan loss, the dual semantic relation reconstruction loss, and the structure-aware distribution alignment loss.

14. A document topic identification device, characterized in that, include: The document acquisition module is used to acquire the document to be recognized. The vector output module is used to input the document to be identified into the topic recognition model to obtain the topic distribution vector of the document to be identified; wherein, the topic distribution vector represents the matching probability between the document and each topic; The topic determination module is used to determine the topic corresponding to the document to be identified based on the topic distribution vector of the document to be identified; wherein the topic identification model is the topic identification model according to any one of claims 1-11.

15. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 12.

16. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 12.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 12.

Images