Text data augmentation method based on auto-encoder and domain knowledge graph

Abstract

The invention discloses a text data augmentation method based on an auto-encoder and a domain knowledge graph, which comprises the following steps of: acquiring text data of a specific domain, preprocessing the text data to obtain text related information, and combining a dynamic attention mechanism to obtain text related information; the text related information is dynamically fused to serve as an input prompt word to be input into the generative auto-encoder; training a generation type auto-encoder, sampling potential vectors from the potential space, and generating an initial augmented text; inputting the initial augmented text into a domain knowledge graph verification module, and verifying the consistency of the initial augmented text and the domain where the text data is located; and a verification result of the domain knowledge graph verification module is fed back to the generation type auto-encoder, an input prompt and an augmented text generation strategy are adjusted according to the verification result, and text data augmentation is performed in combination with multi-modal information. According to the method, the augmented text conforming to the domain specific knowledge system can be automatically generated, and the quality and diversity of text augmented data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text data augmentation method based on an autoencoder and a domain knowledge graph. Background Art

[0002] With the rapid development of natural language processing technology, text data augmentation has become one of the important methods to improve data quality and enhance model generalization ability. At the same time, text data augmentation technology is widely used, including text classification, sentiment analysis, machine translation and other scenarios. However, existing text data augmentation methods mainly rely on simple rules, random replacement or generative methods based on generative models, which have the following challenges:

[0003] Insufficient domain adaptability. Most traditional data augmentation methods are general techniques that cannot fully consider the expertise and context of a specific field. For example, in fields such as medicine, law, and finance, texts often contain highly specialized terminology and complex domain knowledge. Using general data augmentation methods may result in the generated text being inconsistent with domain knowledge, or even semantic errors.

[0004] Lack of domain knowledge constraints. Existing generative model-based generation methods usually rely on contextual autoregressive generation, but lack clear domain knowledge constraints. This causes the generated text to deviate from the domain logic. For example, in the scenario of medical text data augmentation, the generated drugs and symptoms may not match, violating domain rules. Although knowledge graphs can provide relevant information, their combination with generative models is still in its early stages, and it is difficult to adaptively adjust the data augmentation process.

[0005] It is difficult to balance text diversity and quality. Traditional text data augmentation methods such as synonym replacement, sentence rearrangement, and text expansion can easily lead to insufficient diversity or reduced quality of augmented text. Although generative models can generate rich and diverse texts, their excessive randomness may lead to a decline in text generation quality.

[0006] Lack of feedback optimization mechanism. Existing text data augmentation methods are generally static generation processes and lack dynamic feedback mechanisms. In practical applications, when the generated text does not match the domain knowledge, it cannot be corrected through automatic feedback. This makes the text data generated by the model likely to have domain errors, lack of adaptability, and difficult to continuously improve its augmentation effect through the optimization process. Summary of the invention

[0007] In order to solve the problems of insufficient domain adaptability, lack of domain knowledge constraints, difficulty in balancing text diversity and quality, and lack of feedback optimization mechanism in the text data augmentation methods in the prior art, the present invention proposes a text data augmentation method based on autoencoder and domain knowledge graph.

[0008] The specific technical solutions are as follows:

[0009] A text data augmentation method based on autoencoder and domain knowledge graph, comprising the following steps:

[0010] Obtain text data in a specific field, preprocess the text data to obtain text-related information, combine the dynamic attention mechanism, dynamically fuse the text-related information as input prompt words and input them into the generative autoencoder;

[0011] Train a generative autoencoder to sample latent vectors from the latent space to generate initial augmented text;

[0012] Input the initial augmented text into the domain knowledge graph verification module to verify the consistency between the initial augmented text and the domain where the text data is located;

[0013] The verification results of the domain knowledge graph verification module are fed back to the generative autoencoder, and the input prompts and augmented text generation strategies are adjusted according to the verification results, and text data is augmented in combination with multimodal information.

[0014] Furthermore, the text data preprocessing step includes:

[0015] A1: Clean the input data to remove useless noise, perform word segmentation on the cleaned data, and divide the text into words or phrases based on the domain word segmenter;

[0016] A2: Perform part-of-speech tagging and named entity recognition on the word segmentation results, extract key terms and entities, and represent the entity recognition results as NER (T i ), the expression is as follows:

[0017] NER(T i ) = {(entity1, type1), (entity2, type2), ...};

[0018] A3: Use the LDA topic modeling method to extract the potential topics of the text, determine the sentence structure through dependency syntactic parsing, and obtain dependency relationships.

[0019] Furthermore, the text-related information includes: entity information, subject keywords and dependency relationships.

[0020] Furthermore, the dynamic attention mechanism includes: dynamically fusing entity information, topic keywords and dependency relationships to generate input prompts p, which are expressed as follows:

[0021] p=α·NER(T i )+β·Topic(T i )+γ·Dep(T i ),

[0022] α=Attention(NER(T i ), x),

[0023] β=Attention(Topic(T i ), x),

[0024] γ=Attention(Dep(T i ), x),

[0025] Among them, p represents the prompt information fused by the dynamic attention mechanism, α, β, γ are the dynamically adjusted attention weight coefficients, representing the importance of each dimension information in prompt generation, and x represents the preprocessed text input.

[0026] Furthermore, the generative autoencoder includes an encoder and a decoder;

[0027] Encoder: responsible for mapping the vector embedding of the preprocessed text input x and the input prompt p into the latent space to generate the global latent vector z;

[0028] Decoder: Reconstruct the text from the latent vector z and generate augmented text data.

[0029] Furthermore, the training of the generative autoencoder includes:

[0030] By minimizing the reconstruction loss L recon Optimize the generative autoencoder and train the generative autoencoder to minimize the reconstruction loss L recon The expression is:

[0031]

[0032] Where T represents the sequence length, y t represents the tth word generated, z represents the conditional vector generated by the encoder, and y ＜t Represents all words generated before time t;

[0033] Concatenate the input prompt p and the preprocessed text x and adjust them to a specific input format. The encoder uses the BERT pre-trained language model to extract the deep semantic information of the input sequence and map the input sequence to a vector space.

[0034] Initialize the decoder, take the latent vector z output by the encoder as the conditional vector, and generate the text Y = {y1, y2, ..., y n}, in the autoregressive process, the decoder generates the output based on the previous time step as the input of the current time step t.

[0035] Furthermore, the verification of the domain knowledge graph includes:

[0036] The domain knowledge graph is formalized as KG = (V, E), where V is the professional concept in a specific field and E represents the relationship between concepts;

[0037] With the help of information extraction technology in natural language processing, the augmented text is converted into structured data represented by RDF triples, and the SPARQL language is used to check whether the generated text is consistent with the information in the knowledge graph;

[0038] The generated augmented text is mapped to the knowledge graph, and the similarity between the entities in the augmented text and the nodes in the knowledge graph is calculated. The expression of the similarity calculation function sim(y, KG) is as follows:

[0039]

[0040] Where V y and V KG They are the embedding vector representations of augmented text and knowledge graph respectively;

[0041] Design domain knowledge verification loss function L domain , when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold When , the domain consistency penalty is triggered:

[0042]

[0043] At the same time, the domain knowledge verification loss function L domain and the autoencoder generation loss L VAE For joint training, the corresponding mathematical expression is as follows:

[0044] L total =λ domain L domain +L VAE ,

[0045] Where L domain Represents the weight coefficient.

[0046] Furthermore, a feedback closed-loop optimization mechanism is adopted between the domain knowledge graph verification module and the generative autoencoder, and the feedback closed-loop optimization mechanism is specifically:

[0047] The domain knowledge graph verification module calculates the similarity between entities in the SPARQL query and augmented text and nodes in the knowledge graph. When the SPARQL query detects that the generated augmented text does not conform to the domain knowledge graph, the domain knowledge graph verification module feeds back the mismatch to the generative autoencoder to adjust and regenerate the prompt p′; when the verification loss L domainExceeding the threshold, that is, when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold , regenerate the input prompt p′ or adjust the generation parameters of the autoencoder, and regenerate the text. The corresponding expression of the process is as follows:

[0048] p′=feedback(y,KG),

[0049] y′=VAE(p′,x),

[0050] Where p′ represents the input prompt generated after feedback, y′ represents the new augmented text generated under the new input prompt p′, x represents the preprocessed text information, and VAE() represents the use of autoencoder to generate augmented text.

[0051] Furthermore, the mismatch situation includes:

[0052] Entity mismatch: the entities in the augmented text are inconsistent with those in the knowledge graph;

[0053] Relation mismatch: the relations in the augmented text are inconsistent with those in the knowledge graph;

[0054] Attribute mismatch: The entity attributes (such as age, location, etc.) in the augmented text do not match the known attributes of the entity in the knowledge graph.

[0055] Furthermore, the fusion and augmentation of the multimodal information includes:

[0056] Encode the multimodal information into a vector form and fuse it with the text input prompt p. The multimodal information includes image modality and table modality. The image modality uses the pre-trained image model ResNet to generate an embedded representation, and the table modality uses the pre-trained Transformer model to generate an embedded representation.

[0057] When generating augmented text, the autoencoder combines multimodal information to generate text that is highly correlated with multimodal information. The expression is as follows:

[0058] V image =ResNet(D image ),

[0059] V table = Transformer(D table ),

[0060] y multi =VAE(p,x,V image , V table ),

[0061] Where V image and V tableare the vector embedding representations of images and tables, respectively, multi It is an augmented text generated by combining multimodal information. VAE() means that the autoencoder is used to generate augmented text.

[0062] The above technical solution has the following advantages or technical effects:

[0063] 1. The present invention can generate high-quality augmented text through a context-based generative autoencoder and a feedback enhancement mechanism of a domain knowledge graph, ensuring that it meets domain-specific semantic and knowledge requirements.

[0064] 2. The present invention can automatically detect the domain consistency of the augmented text and make dynamic adjustments to improve the diversity and accuracy of the augmented text.

[0065] 3. The present invention can automatically generate augmented text that conforms to the domain-specific knowledge system, improve the quality and diversity of text augmentation data, and at the same time combine multimodal information to make the augmented text more detailed and professional, and adapt to more complex application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of the method of the present invention;

[0067] Figure 2 It is a model architecture diagram of the present invention. DETAILED DESCRIPTION

[0068] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] A text data augmentation method based on autoencoder and domain knowledge graph, comprising the following steps:

[0070] Text input and preprocessing: obtain text data in a specific field, preprocess the text data to obtain text-related information, and combine the dynamic attention mechanism to dynamically integrate the text-related information as input prompt words into the generative autoencoder;

[0071] Train a generative autoencoder to sample latent vectors from the latent space to generate initial augmented text;

[0072] Input the initial augmented text into the domain knowledge graph verification module to verify the consistency between the initial augmented text and the domain where the text data is located;

[0073] The verification results of the domain knowledge graph verification module are fed back to the generative autoencoder, and the input prompts and augmented text generation strategies are adjusted according to the verification results, and text data is augmented in combination with multimodal information.

[0074] Text data augmentation is of great significance in the field of natural language processing, especially when the amount of data is limited or the data is unbalanced. By augmenting the existing text data, the quality and quantity of the data can be improved, thereby improving the performance of the model. Figure 1 As shown in the figure, the present invention provides a text data augmentation method based on a generative autoencoder and a domain knowledge graph, which can generate high-quality augmented text through a context-based generative autoencoder and a feedback enhancement mechanism of a domain knowledge graph to ensure that it meets the domain-specific semantics and knowledge requirements. At the same time, combined with multimodal information, the augmented text is made more detailed and professional, and adapted to more complex application scenarios. The model architecture diagram of the present invention is shown in the figure. Figure 2 As shown, the specific steps include:

[0075] S1: Text input and preprocessing:

[0076] The input text of the present invention mainly comes from domain-specific data sets, such as medical reports, financial documents, legal contracts, etc. Considering the different structures and characteristics of texts in different fields, it is necessary to perform natural language standardization processing on these domain-specific text data, which involves the following steps:

[0077] S11: Text cleaning and word segmentation: Perform text cleaning on the input data to remove useless noise, including special symbols, HTML tags, etc.; perform word segmentation operations, using field-customized word segmenters according to different fields. Field word segmenters include medical word segmenters, legal word segmenters, etc., to divide the text into words or phrases.

[0078] In one embodiment of the present invention, the input text is d i After cleaning and word segmentation, we get the word segmentation result T i , and its corresponding mathematical expression is as follows:

[0079] T i =tokenize(clean(d i )),

[0080] Among them, T i Represents the preprocessed word sequence, tokenize() represents the word segmentation operation, and clean() represents the text cleaning operation.

[0081] S12: Part-of-speech tagging and named entity recognition: word segmentation results T i Perform part-of-speech tagging and named entity recognition to extract key terms and entities. The entity recognition results include disease names, legal terms, etc. The entity recognition results are expressed as NER (T i ), where the marked parts of speech help generate domain-related knowledge, and the corresponding mathematical expression is as follows:

[0082] NER(T i ) = {(entity 1, type 1), (entity 2, type 2), ...}.

[0083] S13: Topic Modeling and Dependency Parsing: Use LDA topic modeling method to extract the potential topic of the text (T i ), determine the sentence structure through dependency syntactic analysis, and obtain the dependency relationship Dep(T i ), potential topic Topic(T i ) and the dependency relationship Dep(T i ) can form domain knowledge constraints and guide the generative autoencoder to generate text augmentation data. The corresponding mathematical expression is as follows:

[0084] Topic(T i )=LDA(T i ),

[0085] Dep(T i ) = dependency_parse(T i ).

[0086] S2: Dynamic Attention Based Hint Fusion:

[0087] The fusion of entity information and topic keywords can ensure that the generated prompt information is highly coherent. The topic keywords usually reflect the macro information of the entire text. The dependency relationship provides fine-grained syntactic structure information. The fusion of topic keywords and dependency relationships can generate more hierarchical prompt information. The present invention dynamically fuses entity information, topic keywords and dependency relationships through a dynamic attention mechanism to generate an input prompt p, and provides the input prompt p to the generative autoencoder. The corresponding mathematical expression is as follows:

[0088] p=α·NER(T i )+β·Topic(T i )+γ·Dep(T i ),

[0089] α=Attetion(NER(T i ), x),

[0090] β=Attention(Topic(T i ), x),

[0091] γ=Attention(Dep(T i ), x),

[0092] Where p represents the prompt information fused by the dynamic attention mechanism, which serves as the input of the generative autoencoder to guide the model to generate augmented text content that conforms to the key information of the text. α, β, and γ are dynamically adjusted attention weight coefficients, representing the importance of each dimension of information in prompt generation. x represents the preprocessed text input.

[0093] S3: Autoencoder generation:

[0094] The generative autoencoder designed by the present invention consists of two parts: an encoder and a decoder. The encoder is responsible for embedding the preprocessed text input x and the vector of the input prompt p into the latent space to generate a global latent vector z for representing the feature distribution of the text, and the decoder reconstructs the text from the latent vector z to generate augmented text data.

[0095] During the training of the generative autoencoder, the reconstruction loss L is minimized. recon To optimize the autoencoder, its essence is the cross entropy loss function:

[0096]

[0097] Where T represents the sequence length, y t represents the tth word generated, z represents the conditional vector generated by the encoder, and y ＜t Represents all words generated before time t. By minimizing the reconstruction loss, it can be ensured that the autoencoder can restore the input data to the maximum extent, that is, the augmented text data with consistent output domain.

[0098] In one embodiment of the present invention, the input prompt p and the preprocessed text x are concatenated and adjusted to a specific input format:

[0099] Input=[CLS]p[SEP]x1x2x3...x n [SEP],

[0100] [CLS] represents the sequence classification identifier, and [SEP] is used to distinguish different fragments.

[0101] The encoder uses the BERT pre-trained language model to extract the deep semantic information of the input sequence and is responsible for mapping the input sequence to the vector space:

[0102] E = BERT (Input);

[0103] Specifically, BERT uses a multi-layer Transformer to encode the contextual information of the input sequence. For each layer of the Transformer architecture, the calculation formula for the lth layer is:

[0104] H l= Transformerlayer(H (l-1) ),

[0105] Among them, H 0 =E, the initial input embedding, Represents the output representation of the lth layer. After being encoded by multiple layers of Transformer, the potential representation vector Z of the output encoder is enc :

[0106]

[0107] Among them, L represents the total number of Transformer layers, d represents the dimension of vector embedding, and finally the [CLS] feature output by the encoder is extracted as the global potential vector z:

[0108] z=Z enc [CLS].

[0109] The decoder is initialized and uses the latent vector z output by the encoder as the conditional vector to generate augmented text. Specifically, the decoder generates text Y = {y1, y2, ..., y n}, in the autoregressive generation process, the decoder uses the output generated at the previous time step as the input of the current time step t.

[0110] In one embodiment of the present invention, the output generated in the previous time step is y t-1 , then e t is its embedding vector at the current time step t:

[0111] e t =Embedding(y t-1 ),

[0112] The Transformer decoder adopts a multi-layer structure design, which can gradually extract high-level features and integrate contextual information when generating sequences. At time step t, for each layer, the calculation expression of the lth layer is:

[0113]

[0114] in represents the input of the decoder in the previous layer, z represents the conditional vector generated by the encoder, and the decoder finally uses the linear layer and the Softmax function to map the conditional probability distribution P(y t ):

[0115] P(y t |y <t )=Softmax(W o H t+b o ),

[0116] Among them, H t is the output of the last layer of the decoder at the current time step t, W o and b o Represents the output layer parameters, selecting the word y with the highest probability t As output, it is also used as input for the next time step.

[0117] S4: Domain knowledge graph verification steps:

[0118] The present invention is based on the domain knowledge graph and verifies the domain consistency of the generated augmented text through logical reasoning, ensuring that the generated text conforms to the logic and common sense of the professional field.

[0119] S41: Formalize the domain knowledge graph as KG = (V, E), where V is the professional concept in a specific field and E represents the relationship between concepts.

[0120] S42: Considering that the augmented text generated by the autoencoder is unstructured, it needs to be converted into a structured representation before consistency and logic checking. With the help of information extraction technology in natural language processing, the augmented text is converted into RDF triple representation (entity, relationship, attribute). After the generated augmented text is converted into structured data, the next step is to build a query based on the knowledge graph. Use SPARQL (a standard query language for querying RDF data) to check whether the generated text is consistent with the information in the knowledge graph.

[0121] S43: Map the generated augmented text to the knowledge graph, calculate the similarity between the entities in the augmented text and the nodes in the knowledge graph, and the similarity function sim(y, KG) is calculated using cosine similarity, and its corresponding mathematical expression is as follows:

[0122]

[0123] Where V y and V KG They are the embedded vector representations of augmented text and knowledge graph respectively.

[0124] The present invention designs a domain knowledge verification loss function L domain , when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold When , the domain consistency penalty is triggered:

[0125]

[0126] At the same time, the domain knowledge verification loss function L domain and the autoencoder generation loss L VAEFor joint training, the corresponding mathematical expression is as follows:

[0127] L total =λ domain L domain +L VAE ,

[0128] where λ domain Represents the weight coefficient used to balance the loss of augmented text generation and domain verification.

[0129] S5: Feedback closed-loop optimization steps:

[0130] The core of the feedback closed-loop optimization mechanism is to feed back the verification results of the domain knowledge graph to the autoencoder, adjust the input prompts and the generation parameters of the autoencoder, and improve the domain consistency of the augmented text.

[0131] There are two verification processes in the domain knowledge graph verification module: SPARQL query and similarity calculation between entities in the augmented text and nodes in the knowledge graph. When the SPARQL query detects that the generated augmented text does not conform to the domain knowledge graph, there may be three situations: entity mismatch: the entity in the augmented text is inconsistent with the entity in the knowledge graph. Relationship mismatch: the relationship in the augmented text is inconsistent with the relationship in the knowledge graph. Attribute mismatch: the entity attributes (such as age, location, etc.) in the augmented text do not match the known attributes of the entity in the knowledge graph. Once an inconsistency is detected, the augmentation system will feed this information back to the model for adjustment and regeneration of the prompt p′; at the same time, when the inspection loss L domain Exceeding the threshold, that is, when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold , regenerate the input prompt p′ or adjust the generation parameters, and regenerate the text. The mathematical expression corresponding to the process is as follows:

[0132] p′=feedback(y,KG),

[0133] y′=VAE(p′,x),

[0134] Where p′ represents the input prompt generated after feedback, y′ represents the new augmented text generated under the new input prompt p′, x represents the preprocessed text information, and VAE() represents the use of autoencoder to generate augmented text.

[0135] One embodiment of the present invention takes medical text data augmentation as an example to illustrate how to use the feedback closed-loop optimization mechanism, combined with the autoencoder and the medical knowledge graph to generate case descriptions that meet domain standards. Specifically, the augmentation system hopes to generate augmented case descriptions about diabetes to enrich the model training data. When the autoencoder generates a case description based on the input prompt, the domain knowledge graph verifies the generated text to ensure that it meets the standards of common symptoms and treatment plans for diabetes.

[0136] For example, the case description generated by the autoencoder mentions that the patient has gained weight dramatically and is advised to stop exercising, but according to the medical knowledge graph, the typical symptom of diabetes is weight loss, and the treatment recommendation is usually to increase appropriate exercise. The domain knowledge graph feeds these deviations back to the model through a feedback mechanism and requires adjustments to the symptom description and treatment recommendations. After receiving the feedback, the model regenerates text that meets medical standards, such as recommendations for weight loss and increased light exercise. Ultimately, through this feedback closed-loop optimization, the generated case description meets medical logic and standards and can be used for subsequent model training.

[0137] S6: Multimodal information fusion and augmentation steps:

[0138] In order to further enhance the diversity and accuracy of augmented data, multimodal information (such as images, tables, etc.) is introduced in the augmentation process to be fused with text information. Multimodal information can provide richer context when generating text, thereby improving the generation ability of the model. At the same time, it should be noted that not all texts have additional multimodal data. The multimodal information fusion and augmentation module is only used as a supplement when additional modal data exists.

[0139] The present invention mainly considers images and tables, two types of multimodal information related to text. First, they are encoded into vector form and fused with the text input prompt p. For the image modality, the pre-trained image model ResNet is used to generate the embedded representation, and the pre-trained Transformer model is used to generate the embedded representation for the table modality. When the autoencoder generates augmented text, it fuses multimodal information to generate text that is highly associated with the multimodal information. The corresponding mathematical expression is as follows:

[0140] V image =ResNet(D image ),

[0141] V table = Transformer(D table ),

[0142] y multi =VAE(p,x,V image , V able ),

[0143] Where V image and V table are the vector embedding representations of images and tables, respectively, multi It is an augmented text generated by combining multimodal information. VAE() means that the autoencoder is used to generate augmented text.

[0144] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A text data augmentation method based on autoencoder and domain knowledge graph, characterized in that the steps include: Obtain text data in a specific field, preprocess the text data to obtain text-related information, combine the dynamic attention mechanism, dynamically fuse the text-related information as input prompt words and input them into the generative autoencoder; Train a generative autoencoder to sample latent vectors from the latent space to generate initial augmented text; Input the initial augmented text into the domain knowledge graph verification module to verify the consistency between the initial augmented text and the domain where the text data is located; The verification results of the domain knowledge graph verification module are fed back to the generative autoencoder, and the input prompts and augmented text generation strategies are adjusted according to the verification results, and text data is augmented in combination with multimodal information.

2. According to claim 1, a text data augmentation method based on autoencoder and domain knowledge graph is characterized in that: The preprocessing steps of the text data include: A1: Clean the input data to remove useless noise, perform word segmentation on the cleaned data, and divide the text into words or phrases based on the domain word segmenter; A2: Perform part-of-speech tagging and named entity recognition on the word segmentation results, extract key terms and entities, and represent the entity recognition results as NER (T i ), the expression is as follows: NER(T i ) = {([entity1, type1), (entity2, type2), ...}; A3: Use the LDA topic modeling method to extract the potential topics of the text, determine the sentence structure through dependency syntactic parsing, and obtain dependency relationships.

3. According to claim 1, a text data augmentation method based on autoencoder and domain knowledge graph is characterized in that: The text-related information includes: entity information, subject keywords and dependency relationships.

4. According to claim 3, a text data augmentation method based on autoencoder and domain knowledge graph is characterized in that: The dynamic attention mechanism includes: dynamically fusing entity information, topic keywords and dependency relationships to generate input prompts p, which are expressed as follows: p=α·NER(T i )+β·Topic(T i )+γ·Dep(T i ), α=Attention(NER(T i ),x), β=Attention(Topic(T i ),x), γ=Attention(Dep(T i ),x), Among them, p represents the prompt information fused by the dynamic attention mechanism, α, β, γ are the dynamically adjusted attention weight coefficients, representing the importance of each dimension information in prompt generation, and x represents the preprocessed text input.

5. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 1, characterized in that: The generative autoencoder comprises an encoder and a decoder; Encoder: responsible for mapping the vector embedding of the preprocessed text input x and the input prompt p into the latent space to generate the global latent vector z; Decoder: Reconstruct the text from the latent vector z and generate augmented text data.

6. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 5, characterized in that: The training of the generative autoencoder includes: By minimizing the reconstruction loss L recon Optimize the generative autoencoder and train the generative autoencoder to minimize the reconstruction loss L recon The expression is: Where T represents the sequence length, y t represents the tth word generated, z represents the conditional vector generated by the encoder, and y ＜t Represents all words generated before time t; Concatenate the input prompt p and the preprocessed text x and adjust them to a specific input format. The encoder uses the BERT pre-trained language model to extract the deep semantic information of the input sequence and map the input sequence to a vector space. Initialize the decoder, take the latent vector z output by the encoder as the conditional vector, and generate the text Y = {y1, y2, ..., y n }, in the autoregressive process, the decoder generates the output based on the previous time step as the input of the current time step t.

7. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 1, characterized in that: The verification of the domain knowledge graph includes: The domain knowledge graph is formalized as KG = (V, E), where V is the professional concept in a specific field and E represents the relationship between concepts; With the help of information extraction technology in natural language processing, the augmented text is converted into structured data represented by RDF triples, and the SPARQL language is used to check whether the generated text is consistent with the information in the knowledge graph; The generated augmented text is mapped to the knowledge graph, and the similarity between the entities in the augmented text and the nodes in the knowledge graph is calculated. The expression of the similarity calculation function sim(y, KG) is as follows: Where V y and V KG They are the embedding vector representations of augmented text and knowledge graph respectively; Design domain knowledge verification loss function L domain , when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold When , the domain consistency penalty is triggered: At the same time, the domain knowledge verification loss function L domain and the autoencoder generates loss L VAE For joint training, the corresponding mathematical expression is as follows: L total =λ domain L domain +L VAE , Where L domain Represents the weight coefficient.

8. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 7, characterized in that: The domain knowledge graph verification module and the generative autoencoder use a feedback closed-loop optimization mechanism, and the feedback closed-loop optimization mechanism is specifically: The domain knowledge graph verification module calculates the similarity between entities in the SPARQL query and augmented text and nodes in the knowledge graph. When the SPARQL query detects that the generated augmented text does not conform to the domain knowledge graph, the domain knowledge graph verification module feeds back the mismatch to the generative autoencoder to adjust and regenerate the prompt p′; when the verification loss L domain Exceeding the threshold, that is, when the similarity between the generated augmented text and the knowledge graph is lower than a certain threshold , regenerate the input prompt p′ or adjust the generation parameters of the autoencoder, and regenerate the text. The corresponding expression of the process is as follows: p′=feedback(y,KG), y′=VAE(p′,x), Where p′ represents the input prompt generated after feedback, y′ represents the new augmented text generated under the new input prompt p′, x represents the preprocessed text information, and VAE() represents the use of autoencoder to generate augmented text.

9. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 8, characterized in that: The mismatch situations include: Entity mismatch: the entities in the augmented text are inconsistent with those in the knowledge graph; Relation mismatch: the relations in the augmented text are inconsistent with those in the knowledge graph; Attribute mismatch: The entity attributes (such as age, location, etc.) in the augmented text do not match the known attributes of the entity in the knowledge graph.

10. The text data augmentation method based on autoencoder and domain knowledge graph according to claim 1, characterized in that: The fusion and augmentation of the multimodal information includes: Encode the multimodal information into a vector form and fuse it with the text input prompt p. The multimodal information includes image modality and table modality. The image modality uses the pre-trained image model ResNet to generate an embedded representation, and the table modality uses the pre-trained Transformer model to generate an embedded representation. When generating augmented text, the autoencoder combines multimodal information to generate text that is highly correlated with multimodal information. The expression is as follows: V image =ResNet(D image ), V table =Transformer(D table ), y multi =VAE(p, x, V image ,V table ), Where V image and V table are the vector embedding representations of images and tables, respectively, multi It is an augmented text generated by combining multimodal information. VAE() means that the autoencoder is used to generate augmented text.

Images