Low-dimensional vocabulary generation method and evaluation system combining reverse decoding and GAN (Generic Area Network)

Abstract

The invention discloses a low-dimensional vocabulary generation method and evaluation system combining reverse decoding and a GAN network, relates to the technical field of deep learning, and solves the technical problems of information loss, redundant feature retention and lack of flexibility of a dimensionality reduction result in a dimensionality reduction process. The GAN network can capture a complex nonlinear relationship through a deep neural network structure, the embedding expression ability after dimension reduction is significantly improved, and the limitation of directly using a full connection layer dimension reduction method is solved; the discriminator guides the generator to remove redundant features in the data through a feedback mechanism, so that the generated low-dimensional embedded information is more compact, unnecessary redundant information is effectively reduced, and the efficiency of the model is improved; and the reverse mapping network is used for embedding and re-mapping low-dimensional embedding back to high-dimensional embedding, so that the integrity of information in the dimension reduction process is ensured, and meanwhile, the interpretability of the model is enhanced, and the data after dimension reduction can be effectively reconstructed.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a low-dimensional vocabulary generation method and evaluation system combining reverse decoding and GAN network. Background Art

[0002] In the current field of deep learning and big data processing, dimensionality reduction technology is widely used to reduce data dimensions and retain important features. With the rapid growth of data scale, especially in the fields of images, voice, text and time series, the data dimension is getting higher and higher, which brings huge challenges to the computational cost and training complexity of the model. Dimensionality reduction can not only reduce data storage and computing overhead, but also improve the training efficiency of the model and prevent the occurrence of dimensionality disaster. At the same time, dimensionality reduction can also remove redundant information and improve the generalization ability and prediction performance of the model. Therefore, how to effectively reduce data dimensionality has become one of the current research hotspots.

[0003] Other existing dimensionality reduction methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA), can effectively process linear data or classification tasks, but perform poorly when processing nonlinear data. Although autoencoders can capture nonlinear features, their training is complex and prone to overfitting. These traditional methods have deficiencies in the generation and reconstruction of high-dimensional data, especially in terms of information retention and dimensionality reduction quality. In addition, directly using fully connected layers to achieve data dimensionality compression is also a common method. It compresses data dimensions through linear transformation and is widely used in neural networks. However, there are obvious limitations to directly using fully connected layers for dimensionality reduction, especially when processing high-dimensional and complex nonlinear data. It is difficult to capture deep dependencies and nonlinear relationships in the data. In addition, directly using fully connected layers for dimensionality reduction usually retains redundant information and cannot dynamically adjust the dimensionality reduction results, which affects the efficiency and performance of the model. Summary of the invention

[0004] The present application provides a low-dimensional vocabulary generation method and evaluation system that combines inverse decoding with a GAN network. The technical purpose is to enhance the ability to capture nonlinear features, reduce redundant information, and enhance the interpretability of the model so that the reduced-dimensional data can be effectively reconstructed.

[0005] The above technical objectives of this application are achieved through the following technical solutions: A low-dimensional vocabulary generation method combining reverse decoding and GAN network, comprising: The original high-dimensional vocabulary is input into the trained generator, and the generator is used to reduce the dimension of the original high-dimensional vocabulary to obtain a low-dimensional vocabulary; Among them, the training process of the generator includes: Input the original high-dimensional vocabulary into the generator, and the generator outputs a low-dimensional vocabulary; The low-dimensional vocabulary is input into the reverse decoding network, and the reverse decoding network outputs a high-dimensional vocabulary; The original high-dimensional vocabulary and the generated high-dimensional vocabulary are input into the discriminator respectively, and the discriminator outputs the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary; In each iteration, the loss of the discriminator is calculated according to the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary until the loss of the discriminator is minimized and the parameters are updated; In each iteration, the loss of the reverse decoding network is calculated according to the parameters updated by the discriminator until the loss of the reverse decoding network is minimized and the parameters are updated; In each iteration, the loss of the generator is calculated according to the parameters updated by the inverse decoding network until the generator loss is minimized, the parameters are updated, and the training of the generator is completed.

[0006] Furthermore, the generator includes a first input layer, a first self-attention layer, a first multi-scale convolution layer and a dimensionality reduction layer; the first input layer receives an original high-dimensional vocabulary, the first self-attention layer enhances the correlation between different positions in the original high-dimensional vocabulary, the multi-scale convolution layer extracts local features and global features of the original high-dimensional vocabulary after the correlation is enhanced, and the dimensionality reduction layer maps the local features and global features to a low-dimensional space to generate a low-dimensional vocabulary.

[0007] Furthermore, the inverse decoding network includes a second input layer, a linear transformation layer, a second self-attention layer and a first multilayer perceptron; the second input layer receives a low-dimensional vocabulary; the linear transformation layer performs a linear transformation on the low-dimensional vocabulary based on the self-attention mechanism of the second self-attention layer to restore the data dimension; the multilayer perceptron gradually reconstructs the dimensionally restored data to generate a high-dimensional vocabulary.

[0008] Furthermore, the discriminator includes a third input layer, a second multi-scale convolution layer, a third self-attention layer and a second multi-layer perceptron; the third input layer receives the original high-dimensional vocabulary and the generated high-dimensional vocabulary; the third self-attention layer enhances the correlation between different positions in the original high-dimensional vocabulary and the generated high-dimensional vocabulary respectively; the second multi-scale convolution layer extracts the local features and global features of the original high-dimensional vocabulary and the generated high-dimensional vocabulary after the correlation is enhanced respectively; the second multi-layer perceptron judges and outputs the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary according to the local features and the global features.

[0009] Furthermore, the loss function of the inverse decoding network is expressed as:

[0010] in, Represents the original high-dimensional vocabulary; Indicates the generation of a high-dimensional vocabulary; represents the square of the Euclidean distance between the original high-dimensional vocabulary and the generated high-dimensional vocabulary output by the inverse decoding network, m Indicates the sample size.

[0011] Among them, the reasoning process of the generator includes: Input the new high-dimensional vocabulary into the trained generator; The trained generator will output a low-dimensional vocabulary.

[0012] An evaluation system for a low-dimensional vocabulary generation method combining reverse decoding and a GAN network, the evaluation system being used for the low-dimensional vocabulary generation method, comprising: The generator and discriminator performance evaluation model generation module is used to establish a performance evaluation model for vocabulary generation tasks. It helps users evaluate the generation effect of the generator and the discrimination effect of the discriminator by defining different evaluation criteria. Data preprocessing and dataset construction module, used to preprocess the original text data and build training sets and test sets; The test execution module is used to provide an interface to automatically execute tests on the generator and discriminator; Comprehensive evaluation module, which comprehensively analyzes the performance of the generator and discriminator through multi-dimensional evaluation criteria; Data visualization module, used for visual display of vocabulary generation and evaluation results.

[0013] Furthermore, the data preprocessing and data set construction module organizes the data set by means of vocabulary library data processing and task scenario vocabulary generation.

[0014] Furthermore, the comprehensive evaluation module is used to: Establish an evaluation model. According to the characteristics of vocabulary generation, an evaluation model including generation quality and term diversity is constructed. Setting weights: Setting weights for each evaluation criterion through quantitative analysis and expert opinions; Evaluation result generation: Comprehensively consider the performance of each evaluation criterion, generate the overall score of the generator and discriminator, and output the evaluation results.

[0015] The beneficial effects of this application are: (1) Enhanced ability to capture nonlinear features: The GAN network can capture complex nonlinear relationships through a deep neural network structure, significantly improving the embedding expression capability after dimensionality reduction and solving the limitations of the dimensionality reduction method using a fully connected layer.

[0016] (2) Reducing redundant information: The discriminator guides the generator to remove redundant features in the data through a feedback mechanism, so that the generated low-dimensional embedding information is more compact, effectively reducing unnecessary redundant information and improving the efficiency of the model.

[0017] (3) Enhanced model interpretability: The inverse mapping network ensures the integrity of information during the dimensionality reduction process by remapping the low-dimensional embedding back to high dimensions, while enhancing the interpretability of the model so that the reduced-dimensional data can be effectively reconstructed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a framework diagram of a low-dimensional vocabulary generation method combining reverse decoding and a GAN network in an embodiment of the present application; DETAILED DESCRIPTION

[0019] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0020] The low-dimensional vocabulary generation method combining inverse decoding and GAN network described in the present application inputs the original high-dimensional vocabulary into a trained generator, and the generator reduces the dimensionality of the original high-dimensional vocabulary to obtain a low-dimensional vocabulary.

[0021] The training process of the generator specifically includes: (1) The original high-dimensional vocabulary is input into the generator, and the generator outputs a low-dimensional vocabulary.

[0022] Specifically, the generator receives the original high-dimensional vocabulary , and processes it through multiple neural network layers to generate a low-dimensional embedding representation.

[0023] The forward propagation process of the generator consists of the following layers: Input layer: receives the high-dimensional representation of the complete vocabulary; Multi-scale convolution layer: The local and global features of the input original high-dimensional vocabulary are extracted through convolution kernels of different scales, which helps to capture features at different levels; Self-attention layer: Use the self-attention mechanism to enhance the correlation between different positions in the original high-dimensional vocabulary, ensuring that the generator can focus on important relationships and dependencies in the vocabulary; Dimensionality reduction layer: Map high-dimensional features to low-dimensional space through linear transformation to generate a low-dimensional vocabulary .

[0024] In summary, the generator formula is expressed as:

[0025] (2) The low-dimensional vocabulary is input into the reverse decoding network, and the reverse decoding network outputs a high-dimensional vocabulary.

[0026] Specifically, the task of the reverse decoding network is to remap the low-dimensional vocabulary output by the generator back to the high-dimensional space to ensure the integrity of the information during the dimensionality reduction process. The forward propagation process of the reverse decoding network includes: Input layer: receives the low-dimensional vocabulary generated by the generator ; Linear transformation layer: linearly transform the low-dimensional vocabulary and gradually restore the dimension of the data; Self-attention layer: Through the self-attention mechanism, it ensures that the dependencies between different positions can be maintained during the recovery of high-dimensional vocabulary; Multilayer Perceptron: for step-by-step reconstruction of high-dimensional vocabulary , ensuring that it is as close as possible to the real high-dimensional vocabulary.

[0027] In summary, the formula of the reverse decoding network is expressed as:

[0028] (3) The original high-dimensional vocabulary and the generated high-dimensional vocabulary are input into the discriminator respectively, and the discriminator outputs the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary.

[0029] Specifically, the discriminator is used to determine whether the vocabulary generated by the generator is similar to the real vocabulary, and optimize the performance of the generator through adversarial training.

[0030] The discriminator receives two inputs: Real vocabulary ( ): original high-dimensional vocabulary; Generate vocabulary ( ): Generate a high-dimensional vocabulary output by the reverse decoding network.

[0031] The discriminator completes the judgment through the following layers: Input layer: receives two input vocabularies (real vocabularies and generated vocabularies); Multi-scale convolutional layer: extracts different features from the vocabulary, focusing on global and local information; Self-attention layer: captures the dependencies between different parts of the vocabulary and enhances the discrimination ability; Multilayer Perceptron: Through linear transformation and activation function, it outputs the judgment scores of the generated high-dimensional vocabulary and the original high-dimensional vocabulary.

[0032] The discriminator outputs two judgment scores and , respectively represent the discrimination results of the original high-dimensional vocabulary and the generated high-dimensional vocabulary.

[0033] (4) The loss of the discriminator is calculated based on the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary until the loss of the discriminator is minimized and the parameters are updated.

[0034] Specifically, the discriminator loss function is calculated based on the discriminator output. Used to train the discriminator so that it can effectively distinguish the real original high-dimensional vocabulary and the generated high-dimensional vocabulary reconstructed by the inverse decoding network The task of the discriminator is to determine whether the generated high-dimensional vocabulary output by the reverse decoding network is generated by real data, thereby providing feedback to the generator and reverse decoding network for optimization.

[0035] The loss function of the discriminator is expressed as:

[0036] in, It means that the discriminator hopes to maximize the true original high-dimensional vocabulary The more the discriminator can correctly identify the real data, the smaller the loss; It means that the discriminator hopes to minimize the generation of high-dimensional vocabulary The more the discriminator can correctly identify the generated false data, the smaller the loss. m Indicates the sample size.

[0037] (5) The loss of the reverse decoding network is calculated based on the updated parameters of the discriminator until the loss of the reverse decoding network is minimized and the parameters are updated.

[0038] Specifically, the inverse decoding network loss function is Generate high-dimensional vocabulary for measuring reconstruction With the real original high-dimensional vocabulary This loss ensures that the low-dimensional embedding information retains its original semantic information as much as possible when reconstructing it into a high-dimensional vocabulary by minimizing the reconstruction error.

[0039] The loss function of the inverse decoding network optimizes the output of the inverse decoding network by calculating the average error over multiple real samples, expressed as:

[0040] in, Represents the reverse decoding network I Low dimensional vocabulary The reconstructed vocabulary after mapping back to the high-dimensional space; It represents the square of the Euclidean distance between the true original high-dimensional vocabulary and the generated high-dimensional vocabulary reconstructed by the inverse decoding network output. The goal is to minimize the error, so as to ensure that the low-dimensional representation generated by the generator can successfully restore the high-dimensional vocabulary in the inverse decoding network. m Indicates the sample size.

[0041] (6) The generator loss is calculated based on the updated parameters of the reverse decoding network until the generator loss is minimized, the parameters are updated, and the generator training is completed.

[0042] Specifically, the generator loss function The purpose is to optimize the performance of the generator so that it can learn more accurate low-dimensional representations. The task of the generator is to generate low-dimensional word embeddings. , so that the reverse decoding network can reconstruct it into a high-dimensional vocabulary The purpose of this loss function is to improve the quality of the generated low-dimensional representation through the adversarial training mechanism, and try to make the reconstructed vocabulary output by the inverse decoding network as close to the real vocabulary as possible.

[0043] The loss function of the generator calculates the loss by averaging the output of the generator on multiple samples, thereby avoiding over-reliance on a single generation result. The loss function is expressed as:

[0044] in, Meaning: Although the discriminator does not directly process the generator output, the generator output still affects the discriminator through the reverse decoding network. The generator hopes to minimize this loss so that the vocabulary output by the reverse decoding network is as real as possible. It is evaluated based on the high-dimensional vocabulary output by the reverse decoding network. represents the probability of the discriminator judging the low-dimensional embedding generated by the generator, m Indicates the sample size.

[0045] In each iteration, back propagation calculates the gradient according to the loss function of each part and updates the parameters, including: Update the discriminator : Minimize the loss of the discriminator , making it better able to distinguish between real and generated data; Update the reverse decoding network I : Minimize the loss of the inverse decoding network , ensure that the generated high-dimensional vocabulary is as close as possible to the real vocabulary; Update Generator : Minimize the loss of the generator , allowing the generator to produce more realistic low-dimensional embeddings.

[0046] The reasoning process of the generator specifically includes: The new high-dimensional vocabulary Input to the trained generator middle; Trained Generator The output is a low-dimensional vocabulary .

[0047] The evaluation system for the low-dimensional vocabulary generation method combining inverse decoding and GAN network described in this application aims to solve the problems of information loss and low generation quality in existing vocabulary generation technology. The evaluation system includes a generator and discriminator performance evaluation model generation module, a data preprocessing and data set construction module, a test execution module, a comprehensive evaluation module and a data visualization module.

[0048] The generator and discriminator performance evaluation model generation module is used to establish a performance evaluation model for vocabulary generation tasks. It helps users evaluate the generation effect of the generator and the discrimination effect of the discriminator by defining different evaluation criteria (such as the quality of the generated vocabulary, the diversity of the vocabulary, and the discrimination ability of the discriminator). The evaluation model can be refined into multiple levels, and each level provides specific evaluation dimensions to ensure that users can fully understand the performance of the model.

[0049] Furthermore, the generator and discriminator performance evaluation model generation module guides users to establish a complete evaluation model by setting different evaluation dimensions of the vocabulary generation task. The evaluation dimensions may include the generation quality of the vocabulary, the diversity of generated terms, the sparsity of the vocabulary, etc.; the evaluation dimensions of the discriminator may include its accuracy, the ability to distinguish generated terms from real terms, etc. The construction of the evaluation model can be achieved by setting evaluation criteria at different levels. For example, the subdivision dimensions of generation quality include the fluency and semantic rationality of the vocabulary, and the generation diversity can evaluate the repetition rate and novelty of the vocabulary, etc. Users can complete the definition of the evaluation model through the wizard of this module.

[0050] The data preprocessing and dataset construction module is used to preprocess the original text data and build training and test sets. This module can automatically convert the original text data into entry datasets according to the requirements of the vocabulary generation task, and divide them according to the set rules for subsequent model training and evaluation.

[0051] Furthermore, the data preprocessing and dataset construction module is to ensure that the generator and discriminator have suitable data for training and testing. This module will first perform preprocessing operations such as word segmentation, deduplication, and stop word removal on the original text data to ensure that the data is clean and has good quality. This module can organize the dataset in the following ways: Vocabulary data processing: extract entries from the existing vocabulary and divide them into training set and test set in proportion so that the model can learn the vocabulary distribution during training; Task scenario vocabulary generation: According to specific tasks (such as vocabulary construction in natural language processing tasks), useful vocabulary is extracted from text data to construct suitable training data sets and test data sets.

[0052] The test execution module is used to provide an interface to automatically execute the test of the generator and the discriminator. The system analyzes the quality of the word list generated by the generator and the accuracy of the word list distinguished by the discriminator by calling different test data sets, and returns the corresponding performance indicators.

[0053] Furthermore, the test execution module provides interfaces for the generator and the discriminator, encapsulating the test execution process of the model. The system will automatically call the data set generated by the organization management module according to the evaluation model set by the user to test the word list generation effect of the generator and the discrimination ability of the discriminator. The test content of the generator includes evaluating the diversity, fluency and semantic rationality of the generated word list; the test content of the discriminator includes its accuracy in identifying real terms and generated terms. By executing these tests, the module returns the corresponding indicators so that users can further adjust and optimize the model.

[0054] The comprehensive evaluation module comprehensively analyzes the performance of the generator and the discriminator through multi-dimensional evaluation criteria. This module provides a comprehensive analysis of the overall performance of the model based on different evaluation dimensions in the vocabulary generation task, and can weigh the relative importance of different criteria.

[0055] Furthermore, the comprehensive evaluation module comprehensively analyzes the performance of the generator and the discriminator in the vocabulary generation task through multi-level criteria. This module balances the relative importance of different evaluation dimensions through weight setting to form the final evaluation result of the vocabulary generation task. The comprehensive evaluation module includes the following functions: Establish an evaluation model: Based on the characteristics of vocabulary generation, build an evaluation model that includes factors such as generation quality and entry diversity; Setting weights: Setting weights for each evaluation dimension through quantitative analysis and expert opinions; Evaluation result generation: The performance of each evaluation criterion is integrated to generate the overall score of the generator and discriminator, and the evaluation results are output.

[0056] The data visualization module is used to visualize the word list generation and evaluation results. Users can intuitively view the quality of the generated word list, the performance of the discriminator, the generation trend of the model, etc., which is convenient for users to further optimize the model.

[0057] Furthermore, the data visualization module provides a visual display of the generator and discriminator test results. Users can intuitively view the process of vocabulary generation, the semantic rationality of the generated entries, the judgment results of the discriminator, and the performance of the model under different evaluation dimensions. The data visualization module displays the characteristics of the generated vocabulary, the decision process of the discriminator, the convergence trend of the generator, etc., to help users conduct in-depth analysis and optimization of the model.

[0058] Taking vocabulary generation as an example, the following introduces a method for generating a low-dimensional vocabulary that combines reverse decoding with a GAN network. The purpose of this application is to provide an optimization method for vocabulary generation tasks through the collaborative work of a generator and a discriminator, so as to improve the quality of vocabulary generation, reduce information loss, and increase the diversity of generated entries. In order to achieve this goal, this application introduces a reverse decoding network, so that more original vocabulary information can be retained in the process of generating the vocabulary, and the performance of the model is improved through back propagation and adversarial training.

[0059] Step 1: Generator forward propagation In the vocabulary generation task, the role of the generator is to generate new vocabulary embeddings by learning the distribution of the original vocabulary data. The forward propagation process of the generator is to input the original high-dimensional vocabulary and map it to a low-dimensional space to generate a more compact vocabulary embedding. The generator is designed to learn the complex nonlinear relationships in the data. The generated low-dimensional embedding not only retains the key features of the data, but also can well represent the distribution of the original data. Through adversarial training, the output of the generator is continuously optimized to better deceive the discriminator in subsequent training.

[0060] In the specific implementation, assume that the original vocabulary contains 100,000 entries, and each entry is represented by a 300-dimensional vector. The input of the generator is a 300-dimensional high-dimensional vector. Through deep neural network structures such as multi-scale convolutional layers and self-attention layers, the vector of the input entry is reduced to a lower representation space (for example, 50 dimensions). Through adversarial training, the generator learns how to map high-dimensional entries to low-dimensional embeddings while trying to retain the semantic information of the original entry.

[0061] In the embodiment of the present application, the output of the generator is a low-dimensional vocabulary representation, denoted as The main task of the generator is to generate such a low-dimensional embedding that the inverse decoding network can reconstruct the high-dimensional vocabulary representation based on this low-dimensional embedding, thereby ensuring that the information loss in the vocabulary generation task is minimized.

[0062] Step 2: Reverse decoding network forward propagation In order to ensure that the low-dimensional vocabulary embedding generated by the generator retains the original information as much as possible, this application introduces a reverse decoding network. The main task of the reverse decoding network is to remap the low-dimensional vocabulary generated by the generator back to the high-dimensional space, so as to verify whether the low-dimensional embedding generated by the generator can reconstruct a high-dimensional representation close to the original vocabulary. The introduction of the reverse decoding network is not only to maintain the integrity of the information after dimensionality reduction, but also allows researchers to observe which features are retained and reconstructed during the dimensionality reduction process. This step reduces information loss through the reverse mapping operation and enhances the interpretability of the model.

[0063] The reverse decoding network receives the low-dimensional vocabulary generated by the generator As input, it is gradually mapped back to a 300-dimensional high-dimensional vocabulary representation using model structures such as linear transformation, self-attention mechanism, and multi-layer perception network. Through this reconstruction process, the inverse decoding network can ensure that the generated low-dimensional vocabulary retains as much original information as possible.

[0064] In the specific implementation, the reverse decoding network remaps each 50-dimensional low-dimensional embedding into a 300-dimensional high-dimensional vector. In order to ensure the accuracy of the reconstruction, the reverse decoding network will try to make the generated high-dimensional vocabulary consistent with the original high-dimensional vocabulary in terms of semantics and representation. In this way, through the verification of the reverse decoding network, the generator can not only generate a low-dimensional vocabulary that is semantically close to the real one, but also ensure its performance in high-dimensional reconstruction.

[0065] For example, suppose the original vocabulary "apple" is represented as a 300-dimensional vector in the high-dimensional space, a 50-dimensional low-dimensional embedding is generated by the generator, and then reconstructed back to a 300-dimensional high-dimensional representation through the inverse decoding network. This high-dimensional representation should be as close to the original "apple" representation as possible in semantics.

[0066] Step 3: Discriminator forward propagation The discriminator plays a crucial role in the generative adversarial network. Its task is to distinguish between the word list embedding generated by the generator and the real word list embedding. Through adversarial training, the discriminator continuously improves its ability to distinguish between generated terms and real terms, thereby helping the generator to continuously improve its generation strategy. The design of the discriminator helps to remove redundant features and improve the quality of the low-dimensional embedding representation of the generator, making the generated low-dimensional word list more compact and effective.

[0067] In this embodiment, the discriminator receives two inputs: one is the high-dimensional vocabulary vector from the real vocabulary dataset , the other is the word list representation reconstructed by the generator and the reverse decoding network The task of the discriminator is to determine whether the input vocabulary comes from the real dataset or a fake vocabulary embedding generated by the generator.

[0068] For example, the discriminator receives the embedding of the word "orange" generated by the generator and the reverse decoding network, and compares it with the high-dimensional embedding of "orange" in the real dataset. If the discriminator thinks that the generated embedding is close enough to the real embedding, it will output a high score, indicating that the generator has successfully fooled the discriminator. Otherwise, it will output a low score, indicating that the discriminator has successfully distinguished between real and fake terms.

[0069] Step 4: Loss function calculation In order to achieve adversarial training of the generator, the reverse decoding network and the discriminator, the loss function design is crucial. This application designs three loss functions, which are used for the optimization of the generator, the reverse decoding network and the discriminator respectively.

[0070] 1) Generator loss function The generator's loss function aims to maximize the probability of deceiving the discriminator. By minimizing the following loss function, the generator can continuously adjust its generation strategy so that the generated vocabulary embedding is closer and closer to the distribution of the real vocabulary:

[0071] in, represents the probability of the discriminator's judgment on the low-dimensional embedding generated by the generator. Through adversarial training, the generator will continuously adjust its parameters so that the discriminator cannot easily distinguish between the generated vocabulary and the real vocabulary. m Indicates the sample size.

[0072] 2) Inverse decoding network loss function The loss function of the inverse decoding network is used to measure the quality of the generated vocabulary reconstruction. Its goal is to minimize the mean square error between the generated high-dimensional vocabulary and the true high-dimensional vocabulary, expressed as:

[0073] This loss function ensures that the generated vocabulary embedding can not only deceive the discriminator, but also reconstruct a high-dimensional representation close to the original high-dimensional vocabulary.

[0074] 3) Discriminator loss function The loss function of the discriminator is used to distinguish the word list embedding generated by the generator from the real word list embedding. The goal is to maximize the discriminant probability of the real word list while minimizing the discriminant probability of the generated word list:

[0075] By minimizing the loss of the discriminator, the system can effectively improve the discriminator's ability to distinguish between real vocabularies and generated vocabularies.

[0076] Step 5: Back propagation and parameter update During the back-propagation process, the parameters of the generator, the inverse decoding network, and the discriminator are updated according to the loss function. In each round of adversarial training, the generator, the inverse decoding network, and the discriminator update their respective parameters through back-propagation. First, by minimizing the discriminator's loss, the discriminator's weights are updated so that it can better distinguish between real and generated samples. Next, the generator minimizes the generator's loss based on the discriminator's feedback and adjusts the generator's parameters to generate more realistic low-dimensional embeddings. Finally, the inverse decoding network optimizes its parameters by minimizing the reconstruction loss to ensure that the generated high-dimensional representation can restore the real data as much as possible. Each module is continuously adjusted during the training process, ultimately achieving efficient dimensionality reduction and reconstruction of the data.

[0077] 1) Generator parameter update The generator minimizes its loss function , using gradient descent to adjust its parameters so that more realistic low-dimensional word embeddings can be generated in the next iteration. The direction of the gradient update is to make the word embedding output by the generator better deceive the discriminator.

[0078] 2) Parameter update of the reverse decoding network The reverse decoding network minimizes the loss function To update its parameters, the generated high-dimensional vocabulary is as close to the real vocabulary as possible. Through continuous updating, the inverse decoding network can gradually improve the quality of the reconstructed vocabulary and reduce information loss.

[0079] 3) Discriminator parameter update The discriminator minimizes the loss function , adjusts its parameters based on feedback from the real and generated vocabulary, thereby improving its ability to distinguish between real and fake vocabulary. As the training process progresses, the discriminator is able to better detect fake vocabulary embeddings produced by the generator.

[0080] Step 6: Model Inference (1) New high-dimensional vocabulary Input to the trained generator middle.

[0081] (2) Trained Generator The output is a low-dimensional vocabulary .

[0082] In the embodiment of the present application, through the iterative training of the above steps, the generator and the reverse decoding network gradually improve their ability to generate and reconstruct the vocabulary, and the discriminator gradually enhances its ability to distinguish between real and generated vocabulary. After a large number of iterative training, the system can eventually generate high-quality vocabulary embeddings, and while maintaining information integrity, ensure that the generated vocabulary has high semantic consistency, and finally output a trained generator. The new high-dimensional vocabulary is input into the trained generator to obtain a low-dimensional vocabulary.

[0083] The above are exemplary embodiments of the present application, and the protection scope of the present application is defined by the claims and their equivalents.

Claims

1. A low-dimensional vocabulary generation method combining reverse decoding and GAN network, characterized in that: include: The original high-dimensional vocabulary is input into the trained generator, and the generator is used to reduce the dimension of the original high-dimensional vocabulary to obtain a low-dimensional vocabulary; Among them, the training process of the generator includes: Input the original high-dimensional vocabulary into the generator, and the generator outputs a low-dimensional vocabulary; The low-dimensional vocabulary is input into the reverse decoding network, and the reverse decoding network outputs a high-dimensional vocabulary; The original high-dimensional vocabulary and the generated high-dimensional vocabulary are input into the discriminator respectively, and the discriminator outputs the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary; In each iteration, the loss of the discriminator is calculated according to the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary until the loss of the discriminator is minimized and the parameters are updated; In each iteration, the loss of the reverse decoding network is calculated according to the parameters updated by the discriminator until the loss of the reverse decoding network is minimized and the parameters are updated; In each iteration, the loss of the generator is calculated according to the parameters updated by the inverse decoding network until the generator loss is minimized, the parameters are updated, and the training of the generator is completed.

2. The method for generating a low-dimensional vocabulary according to claim 1, wherein: The generator includes a first input layer, a first self-attention layer, a first multi-scale convolution layer and a dimensionality reduction layer; the first input layer receives an original high-dimensional vocabulary, the first self-attention layer enhances the correlation between different positions in the original high-dimensional vocabulary, the multi-scale convolution layer extracts local features and global features of the original high-dimensional vocabulary after the correlation is enhanced, and the dimensionality reduction layer maps local features and global features to a low-dimensional space to generate a low-dimensional vocabulary.

3. The method for generating a low-dimensional vocabulary according to claim 2, wherein: The reverse decoding network includes a second input layer, a linear transformation layer, a second self-attention layer and a first multilayer perceptron; the second input layer receives a low-dimensional vocabulary; the linear transformation layer performs a linear transformation on the low-dimensional vocabulary based on the self-attention mechanism of the second self-attention layer to restore the data dimension; the multilayer perceptron gradually reconstructs the dimensionally restored data to generate a high-dimensional vocabulary.

4. The method for generating a low-dimensional vocabulary according to claim 3, wherein: The discriminator includes a third input layer, a second multi-scale convolutional layer, a third self-attention layer, and a second multi-layer perceptron; the third input layer receives the original high-dimensional vocabulary and the generated high-dimensional vocabulary; the third self-attention layer enhances the correlation between different positions in the original high-dimensional vocabulary and the generated high-dimensional vocabulary respectively; The second multi-scale convolutional layer extracts local features and global features of the original high-dimensional vocabulary and the generated high-dimensional vocabulary after correlation enhancement respectively; the second multi-layer perceptron judges and outputs the scores of the original high-dimensional vocabulary and the generated high-dimensional vocabulary according to the local features and the global features.

5. The method for generating a low-dimensional vocabulary according to claim 4, wherein: The loss function of the inverse decoding network is expressed as: in, Represents the original high-dimensional vocabulary; Indicates the generation of a high-dimensional vocabulary; represents the square of the Euclidean distance between the original high-dimensional vocabulary and the generated high-dimensional vocabulary output by the inverse decoding network, m Indicates the sample size.

6. An evaluation system for a low-dimensional vocabulary generation method combining reverse decoding and a GAN network, the evaluation system being used for the low-dimensional vocabulary generation method described in any one of claims 1 to 5, characterized in that: include: The generator and discriminator performance evaluation model generation module is used to establish a performance evaluation model for vocabulary generation tasks. It helps users evaluate the generation effect of the generator and the discrimination effect of the discriminator by defining different evaluation criteria. Data preprocessing and dataset construction module, used to preprocess the original text data and build training and test sets; The test execution module is used to provide an interface to automatically execute tests on the generator and discriminator; Comprehensive evaluation module, which comprehensively analyzes the performance of the generator and discriminator through multi-dimensional evaluation criteria; Data visualization module, used for visual display of vocabulary generation and evaluation results.

7. The evaluation system according to claim 6, characterized in that The data preprocessing and data set construction module organizes the data set by means of vocabulary library data processing and task scenario vocabulary generation.

8. The evaluation system according to claim 6, wherein: The comprehensive assessment module is used to: Establish an evaluation model. According to the characteristics of vocabulary generation, an evaluation model including generation quality and term diversity is constructed. Setting weights: Setting weights for each evaluation criterion through quantitative analysis and expert opinions; Evaluation result generation: Comprehensively consider the performance of each evaluation criterion, generate the overall score of the generator and discriminator, and output the evaluation results.

Images