Deep learning-based professional noun synonym replacement assisting method and system

Abstract

The invention discloses a professional noun synonym replacement assisting method and system based on deep learning. The method comprises the steps of knowledge fusion pre-training, dynamic context sensing modeling, domain adaptation optimization, multi-dimensional output optimization and professional noun synonym replacement assisting. The invention relates to the technical field of natural language text generation, and adopts multi-modal learning and knowledge fusion technologies in a deep learning technology to construct a dual-channel model, cross-modal alignment operation and domain knowledge extraction in a knowledge fusion model pre-training process. The language modeling accuracy and the context information processing capability of the basic model are optimized through the combination of the three, and the targeted optimization of professional noun synonym replacement assistance and the availability of the whole scheme are realized; performing dynamic context sensing modeling by adopting a generative adversarial verification network combining improved attention and dynamic context coding; a lightweight model adaptation improvement method combined with domain classification is adopted, and a domain adaptation effect is optimized through adaptive adjustment and an elastic weight mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language text generation, and specifically refers to a method and system for assisting in synonym replacement of professional terms based on deep learning. Background Art

[0002] A method and system for assisting in synonym replacement of professional terms based on deep learning is an intelligent system that uses deep learning technology to automatically identify and replace synonyms or near-synonyms in a specific professional field within a text. This method constructs a knowledge graph and semantic model for the professional field, combines natural language processing (NLP) technology, analyzes the professional terms in the text, and automatically recommends or performs synonym replacement to ensure the fluency, accuracy, and professionalism of the text. This system is mainly applied to academic paper writing, technical document translation, optimization of professional texts in different fields, etc., aiming to improve the efficiency of document processing, reduce manual intervention, enhance the consistency and standardization of texts, and ensure that the expression of professional terms is more accurate and conforms to industry standards.

[0003] However, in the existing methods for assisting in synonym replacement of professional terms, there are technical problems in existing intelligent language assistance tools. The understanding ability of specific professional vocabulary in a professional field has relatively large limitations, and the auxiliary tools for replacing professional terms optimized for adaptation to a specific field mostly perform synonym replacement through dictionaries or simple word vectorization methods. This leads to poor semantic replacement accuracy of professional terms, and at the same time lacks awareness of the context content of professional terms. As a result, the replaced terms may not match the context content, making it difficult to support the precise use of professional vocabulary and the usability of the replacement function.

[0004] In the existing dynamic context-aware modeling process, there are technical problems in existing context information processing methods. They usually focus on the centralized processing of short-term dependencies and the understanding of content in a fixed context. This results in the fact that when traditional methods process professional terms, they can only capture basic syntactic and semantic information, and thus cannot fully consider the language changes and differences brought about by more refined and complex domain-specific professional terms, limiting the scope and modeling ability of context awareness, and thus affecting the replacement accuracy and optional range of professional terms.

[0005] In the process of domain adaptation optimization of the existing synonym replacement theory, there is a technical problem that existing methods are difficult to accurately adapt to the professional terms and their context differences in different fields, resulting in insufficient domain adaptability. Summary of the Invention

[0006] In view of the above situation, to overcome the defects of the existing technology, the present invention provides a method and system for assisting in synonymous replacement of professional terms based on deep learning. In the existing methods for assisting in synonymous replacement of professional terms, there are limitations in the understanding ability of specific professional vocabulary in the existing intelligent language assistance tools. And the auxiliary tools for replacing professional terms optimized specifically for the adaptation field mostly use dictionaries or simple word vectorization methods for synonymous replacement. This results in poor accuracy of semantic replacement of professional terms, and at the same time lacks awareness of the context content of professional terms. As a result, the replaced terms do not match the context content, making it difficult to support the precise use of professional vocabulary and the usability of the replacement function. The present solution creatively adopts multi-modal learning and knowledge fusion technologies in deep learning technology. Through the construction of a dual-channel model, cross-modal alignment operation, and domain knowledge extraction during the pre-training process of the knowledge fusion model, the accuracy of language modeling and the context information processing ability of the basic model are optimized through the combination of the three, realizing the targeted optimization of professional term synonymous replacement assistance and the usability of the overall solution. In the existing dynamic context awareness modeling process, there are existing context information processing methods that usually focus on the centralized processing of short-term dependencies and the content understanding of fixed contexts. This leads to the situation that when traditional methods process professional terms, they can only capture basic grammar and semantic information, and thus cannot fully consider the language changes and differences brought by more refined and complex domain professional terms, limiting the scope and modeling ability of context awareness, and thus affecting the replacement accuracy and optional range of professional terms. The present solution creatively adopts a generative adversarial verification network that combines improved attention and dynamic context encoding for dynamic context awareness modeling. Through a hierarchical attention mechanism at the semantic and syntactic levels, combined with feature optimization of the transformer encoding and temporal convolutional network, context information is jointly extracted, and finally a generative adversarial network is used to generate replacement words, improving the dynamics of context information modeling and the accuracy of semantic perception of professional terms. In the existing process of domain adaptation optimization for synonymous replacement, there are technical problems that existing methods are difficult to accurately adapt to professional terms and their context differences in different domains, resulting in insufficient domain adaptability. The present solution creatively adopts a lightweight model adaptation improvement method that combines domain classification. Through an adaptive adjustment and elastic weight mechanism, the domain adaptation effect is optimized, and the domain adaptability and accuracy of synonymous replacement are improved.

[0007] The technical solution adopted by the present invention is as follows: The method for assisting in synonymous replacement of professional terms based on deep learning provided by the present invention includes the following steps:

[0008] Step S1: Knowledge fusion pre-training;

[0009] Step S2: Dynamic context awareness modeling;

[0010] Step S3: Domain adaptation optimization;

[0011] Step S4: Multi-dimensional output optimization;

[0012] Step S5: Professional noun synonym replacement assistance.

[0013] Further, in Step S1, the knowledge fusion pre-training is used to construct a professional noun semantic network and perform pre-training. Specifically, by collecting professional noun data and performing pre-training on the knowledge fusion model, a professional noun knowledge group is obtained. The professional noun knowledge group includes a professional noun pre-training model and professional noun vector data;

[0014] The professional noun data collection includes text data collection and knowledge graph construction;

[0015] The text data collection obtains text data, including professional literature data, professional technical document data, and domain corpus data; the knowledge graph construction obtains knowledge graph data by constructing professional noun node parameters and semantic relationship edge parameters;

[0016] The knowledge fusion model pre-training includes the following steps:

[0017] Step S11: Construct a dual-channel pre-training architecture. Specifically, construct a dual-channel model, including a text processing channel and a graph information processing channel, and perform dual-channel pre-training by constructing a dual-channel pre-training objective function to obtain a knowledge fusion pre-training basic model;

[0018] Step S13: Domain knowledge extraction. Specifically, use a standard graph neural network to extract domain knowledge feature data from the domain corpus data in the text data, and use the domain knowledge feature data for domain adaptation optimization;

[0019] Step S12: Cross-modal alignment. Specifically, through a multi-modal learning method, align the information of the text data and the information of the knowledge graph data during pre-training, and optimize the consistency of the text features and the graph data features through a cross-modal alignment loss function to obtain an aligned and optimized knowledge fusion pre-training model;

[0020] Step S14: Knowledge fusion pre-training. Specifically, combine the text features, the graph data features, and the domain knowledge feature data, and perform knowledge fusion pre-training through a fusion target loss function to obtain a professional noun pre-training model Model PRE , and by using the professional noun pre-training model Model PRE , perform professional noun vector construction to obtain a professional noun knowledge group;

[0021] The professional noun vector data specifically refers to the professional noun semantic vector obtained by modeling the semantics of professional nouns through cosine similarity.

[0022] Further, in step S2, the dynamic context-aware modeling is used to dynamically model the context semantic information of professional nouns. Specifically, based on the professional noun knowledge group, a generative adversarial verification network combining improved attention and dynamic context encoding is adopted to perform dynamic context-aware modeling to obtain professional noun dynamic context-aware data, including the following steps:

[0023] Step S21: Construct a hierarchical attention mechanism. Specifically, by constructing semantic layer attention and syntactic layer attention, calculate the semantic hierarchical attention feature and the syntactic hierarchical attention feature, and through weighted synthesis, construct a hierarchical attention mechanism to obtain hierarchical context basic information data;

[0024] The calculation formula for the hierarchical context basic information data is:

[0025] ;

[0026] In the formula, is the hierarchical context basic information data of the i-th professional noun vector, is the syntactic layer attention weight, is the semantic hierarchical attention feature of the i-th professional noun vector, is the semantic layer attention weight, is the syntactic hierarchical attention feature of the i-th professional noun vector;

[0027] Step S22: Construct a transformer encoder. Specifically, by constructing a standard transformer model and combining variational inference methods, capture context information to obtain transformer encoded feature output data;

[0028] Step S23: Construct a temporal convolutional dynamic model. Specifically, construct standard temporal convolutional network data, and based on the transformer encoded feature output data, perform temporal convolutional modeling to obtain temporal convolutional feature data;

[0029] Step S24: Dynamic context encoding. Specifically, combine the transformer encoded feature output data and the temporal convolutional feature data to perform dynamic context encoding to obtain dynamic context encoding feature data. The calculation formula is:

[0030] ;

[0031] Where C is the dynamic context encoding feature data, Encoder(·) is the transformer encoder function, X is the professional noun knowledge group data input as the original data, and TCN(·) is the temporal convolutional dynamic model function;

[0032] Step S25: Construct a generative adversarial model for generating synonym replacement candidate words. Specifically, construct a generator model and a discriminator model, and combine the generative adversarial loss to generate synonym replacement candidate words based on the dynamic context encoding feature data to obtain synonym replacement candidate word data;

[0033] Step S26: Construct a semantic similarity verification network for evaluating the quality of replacement candidate words. Specifically, perform semantic similarity verification by constructing a pre-similarity to obtain semantic similarity evaluation reference data;

[0034] Step S27: Train the dynamic context-aware modeling model. Specifically, construct a comprehensive loss function through an end-to-end training strategy and train the dynamic context-aware modeling model to obtain a professional noun dynamic context-aware model, and obtain professional noun dynamic context-aware data by using the professional noun dynamic context-aware model;

[0035] The calculation formula of the comprehensive loss function is:

[0036] ;

[0037] Where L total Comprehensive loss function, L GAN is the generative adversarial loss function, is the cross-modal alignment loss weight, L align is the cross-modal alignment loss function, is the semantic similarity loss weight, L seman is the semantic similarity loss function.

[0038] Furthermore, in step S3, the domain adaptation optimization is used to optimize the domain adaptability of professional nouns. Specifically, based on the professional noun knowledge group, a lightweight model adaptation improvement method combined with domain classification is adopted to perform domain adaptation optimization to obtain domain adaptation model data, including the following steps:

[0039] Step S31: Construct a lightweight adaptation module. Specifically, introduce a parameter sharing and adaptive adjustment strategy, and replace the standard graph convolutional neural network with a lightweight adaptation module to generate an adaptive domain vector to obtain a lightweight adaptation semantic vector output;

[0040] Step S32: Construct a domain classifier for dynamically activating the adaptation module. Specifically, construct a domain classifier to classify the output of the lightweight adaptation semantic vector to obtain domain category data, and perform dynamic activation based on the domain category data to obtain dynamically activated output data;

[0041] Step S33: Elastic weight consolidation for optimizing synonym replacement forgetting. Specifically, construct an elastic weight loss function for domain adaptation synonym replacement optimization. The calculation formula is:

[0042] ;

[0043] In the formula, L EWC is the elastic weight loss function, L adapt is the domain adaptation loss function, is the elastic weight adjustment parameter, I is the total number of Fisher matrix elements, representing the total number of professional noun vectors, i is the Fisher matrix element index, representing the professional noun vector index, F i is the Fisher matrix element corresponding to the i-th professional noun vector, is the domain adaptation optimization model adaptation parameter corresponding to the i-th professional noun vector, is the optimal parameter of the domain adaptation optimization model corresponding to the i-th professional noun vector;

[0044] Step S34: Extract domain professional noun distribution features. Specifically, based on the dynamically activated output data, extract domain professional noun distribution features to obtain domain professional noun distribution feature data;

[0045] Step S35: Lightweight model adaptation improvement. Specifically, perform two-way improvement on the domain adaptation loss function for domain tasks and domain adaptability to obtain a two-way improvement loss function, and optimize model training by constructing a lightweight model adaptation improvement loss function to train the lightweight adaptation model Model AD ;

[0046] The calculation formula of the two-way improvement loss function is:

[0047] ;

[0048] In the formula, is the two-way improvement loss function, used to represent the improvement of the domain adaptation loss function by combining the domain task loss function and the domain adaptability loss function, L task is the domain task loss function, specifically referring to the cross-entropy loss function in the domain classifier process, is the adjustment parameter, L domain is the domain knowledge extraction loss function, used as the adaptability loss function;

[0049] The calculation formula for the improved loss function adapted to the lightweight model is as follows:

[0050] ;

[0051] In the formula, is the improved loss function adapted to the lightweight model, and L task is the domain task loss function, is the domain knowledge extraction loss weight, and L domain is the domain knowledge extraction loss function, which is used as the adaptation loss function, is the elastic weight loss weight, and L EWC is the elastic weight loss function;

[0052] Step S36: Domain adaptation optimization, specifically using the lightweight adaptation model Model AD , perform domain adaptation optimization to obtain domain adaptation model data.

[0053] Furthermore, in step S4, the multi-dimensional output optimization is used to enhance the diversity of the selection of replaceable professional terms. Specifically, based on the dynamic context-aware data of the professional terms, a controllable semantic converter is constructed and used to perform multi-dimensional expansion and output optimization of professional term replacement, and multi-dimensional output optimization professional term replacement reference data is obtained, including the following steps:

[0054] Step S41: Replacement strategy optimization, specifically introducing a replacement strategy optimization method based on the dynamic context-aware data of the professional terms. By optimizing the selection mechanism of replacement candidate words, the number of professional terms with similar semantics to be replaced is optimized. Specifically, the maximum sum of the product of semantic similarity and the occurrence probability of candidate words is used for replacement strategy optimization;

[0055] Step S42: Enhancement of replacement noun explanations, specifically during the replacement strategy optimization process, by adding the calculation of semantic similarity of noun explanation information, the interpretability of the replaced nouns is optimized;

[0056] Step S43: Multi-dimensional replacement post-processing, specifically by introducing syntactic consistency checking and semantic consistency checking, perform multi-dimensional replacement post-processing to optimize the semantic deviation processing ability;

[0057] Step S44: Multi-dimensional output optimization, specifically through the formaldehyde combination of syntactic features, semantic features, and context features, perform multi-dimensional output optimization to obtain multi-dimensional output optimization professional term replacement reference data.

[0058] Furthermore, in step S5, the professional term synonym replacement assistance is used to provide synonym replacement assistance. Specifically, by receiving the original input statement, through the dynamic context awareness modeling and the domain adaptation optimization, the domain-adapted professional term synonym replacement is generated, and through the multi-dimensional output optimization, the comprehensive reference assistance data for professional term synonym replacement is obtained.

[0059] The professional term synonym replacement assistance system based on deep learning provided by the present invention includes an interaction layer, a processing layer, a knowledge management layer, and an output optimization layer;

[0060] The interaction layer is used to construct a multi-modal input interface and provide visual interaction. Through the interaction layer, the original data input of the user is received and sent to the processing layer for professional term synonym replacement;

[0061] The processing layer is used for knowledge fusion pre-training and dynamic context awareness modeling. Through the processing layer, the professional term knowledge group and the professional term dynamic context awareness data are obtained, and the professional term knowledge group is sent to the knowledge management layer, and the professional term dynamic context awareness data is sent to the output optimization layer;

[0062] The knowledge management layer is used for domain adaptation optimization. Through the domain adaptation optimization, the domain adaptation model data is obtained and sent to the output optimization layer;

[0063] The output optimization layer is used for multi-dimensional output optimization. Through the multi-dimensional output optimization, the multi-dimensional output optimized professional term replacement reference data is obtained.

[0064] The beneficial effects achieved by the present invention using the above solution are as follows:

[0065] (1) In view of the technical problems existing in the existing professional term synonym replacement assistance methods, in the existing intelligent language assistance tools, the understanding ability of the specific professional terms in the professional field is greatly limited, and the assistance tools for replacing professional terms specifically adapted to the field optimization mostly use dictionaries or simple word vectorization methods for synonym replacement, which results in poor semantic replacement accuracy of professional terms and lack of awareness of the context content of professional terms. Furthermore, the replaced nouns do not match the context content, making it difficult to support the precise use of professional terms and the usability of the replacement function. This solution creatively uses the multi-modal learning and knowledge fusion technologies in deep learning. Through the construction of a dual-channel model, cross-modal alignment operation, and domain knowledge extraction in the pre-training process of the knowledge fusion model, the language modeling accuracy and context information processing ability of the basic model are optimized through the combination of the three, realizing the targeted optimization of professional term synonym replacement assistance and the usability of the overall solution;

[0066] (2)In the existing dynamic context awareness modeling process, there are existing context information processing methods that usually focus on the centralized processing of short-term dependencies and the content understanding of fixed contexts. This has led to the situation that in dealing with professional terms, traditional methods can only capture basic grammar and semantic information, and thus cannot fully consider the language changes and differences brought by more refined and complex domain-specific terms, restricting the scope of context awareness and the modeling ability, and thus affecting the replacement accuracy of professional terms and the optional range. In view of this technical problem, this solution creatively adopts a generative adversarial verification network that combines improved attention and dynamic context encoding for dynamic context awareness modeling. Through a hierarchical attention mechanism at the semantic and syntactic levels, combined with feature optimization of the transformer encoding and the temporal convolutional network, context information is jointly extracted. Finally, a generative adversarial network is used to generate replacement words, enhancing the dynamics of the context information modeling of professional terms and the accuracy of semantic perception.

[0067] (3)In the process of domain adaptation optimization of the existing synonym replacement, there is a technical problem that existing methods are difficult to accurately adapt to professional terms in different domains and their context differences, resulting in insufficient domain adaptability. In view of this, this solution creatively adopts a lightweight model adaptation improvement method that combines domain classification. Through an adaptive adjustment and elastic weight mechanism, the domain adaptation effect is optimized, and the domain adaptability and accuracy of synonym replacement are improved. Brief Description of the Drawings

[0068] Figure 1 It is a schematic flowchart of the method for assisting synonym replacement of professional terms based on deep learning provided by the present invention;

[0069] Figure 2 It is a schematic diagram of the system for assisting synonym replacement of professional terms based on deep learning provided by the present invention;

[0070] Figure 3 It is a schematic flowchart of the knowledge fusion pre-training in step S1;

[0071] Figure 4 It is a schematic flowchart of the dynamic context awareness modeling in step S2;

[0072] Figure 5 It is a schematic flowchart of the domain adaptation optimization in step S3;

[0073] Figure 6 It is a schematic flowchart of the multi-dimensional output optimization in step S4.

[0074] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. Detailed Description of the Embodiments

[0075] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0076] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0077] Embodiment 1, referring to Figure 1 , the method for assisting in synonym replacement of professional terms based on deep learning provided by the present invention includes the following steps:

[0078] Step S1: Knowledge fusion pre-training;

[0079] Step S2: Dynamic context-aware modeling;

[0080] Step S3: Domain adaptation optimization;

[0081] Step S4: Multi-dimensional output optimization;

[0082] Step S5: Assisting in synonym replacement of professional terms.

[0083] Embodiment 2, referring to Figure 1 , Figure 2 and Figure 3 , based on the above embodiment, in step S1, the knowledge fusion pre-training is used to construct a professional term semantic network and perform pre-training. Specifically, by collecting professional term data and performing pre-training on the knowledge fusion model, a professional term knowledge group is obtained. The professional term knowledge group includes a professional term pre-training model and professional term vector data;

[0084] The collection of professional term data includes text data collection and knowledge graph construction;

[0085] The text data collection obtains text data, including professional literature data, professional technical document data, and domain corpus data; the knowledge graph construction obtains knowledge graph data by constructing professional term node parameters and semantic relationship edge parameters;

[0086] The pre-training of the knowledge fusion model includes the following steps:

[0087] Step S11: Construct a dual-channel pre-training architecture, specifically, construct a dual-channel model, including a text processing channel and a graph information processing channel, and perform dual-channel pre-training by constructing a dual-channel pre-training objective function to obtain a knowledge fusion pre-training basic model;

[0088] The calculation formula of the dual-channel pre-training objective function is:

[0089] ;

[0090] In the formula, L P is the dual-channel pre-training objective function, is the text loss weight, L text (·) is the text processing channel loss function, X is the text data, is the graph loss weight, L graph (·) is the graph information processing channel loss function, G is the knowledge graph data;

[0091] Step S12: Cross-modal alignment, specifically, align the information of the text data and the information of the knowledge graph data during the pre-training process through a multi-modal learning method, and optimize the consistency of the text features and the graph data features through a cross-modal alignment loss function to obtain an aligned and optimized knowledge fusion pre-training model;

[0092] The calculation formula of the cross-modal alignment loss function is:

[0093] ;

[0094] In the formula, L align is the cross-modal alignment loss function, f text is the text feature, f graph is the graph data feature;

[0095] Step S13: Domain knowledge extraction, specifically, use a standard graph neural network to extract domain knowledge feature data from the domain corpus data in the text data, and use the domain knowledge feature data for domain adaptation optimization;

[0096] Step S14: Knowledge fusion pre-training, specifically, combine the text features, the graph data features and the domain knowledge feature data, and perform knowledge fusion pre-training through a fusion target loss function to obtain a professional term pre-training model Model PRE , and construct professional term vectors by using the professional term pre-training model Model PRE to obtain a professional term knowledge group;

[0097] The calculation formula of the fusion target loss function is:

[0098] ;

[0099] wherein, L fus is the fusion target loss function, L P is the dual-channel pre-training target function, is the cross-modal alignment loss weight, L align is the cross-modal alignment loss function, is the domain knowledge extraction loss weight, L domain is the domain knowledge extraction loss function;

[0100] The professional noun vector data specifically refers to the professional noun semantic vector obtained by modeling the semantics of professional nouns through cosine similarity.

[0101] By performing the above operations, in the existing professional noun synonym replacement assistance methods, there are limitations in the understanding ability of existing intelligent language assistance tools for specific professional vocabulary in professional fields. And the auxiliary tools for replacing professional nouns optimized specifically for the adaptation field mostly use dictionaries or simple word vectorization methods for synonym replacement, which leads to poor semantic replacement accuracy of professional nouns and lack of perception of the context content of professional nouns. As a result, the replaced nouns do not match the context content, making it difficult to support the precise use of professional vocabulary and the usability of the replacement function. This solution creatively adopts multi-modal learning and knowledge fusion technologies in deep learning. Through constructing a dual-channel model, cross-modal alignment operation, and domain knowledge extraction during the pre-training process of the knowledge fusion model, the language modeling accuracy and context information processing ability of the basic model are optimized through the combination of the three, achieving targeted optimization of professional noun synonym replacement assistance and the usability of the overall solution.

[0102] Embodiment 3, referring to Figure 1 , Figure 2 and Figure 4 , based on the above embodiment, in step S2, the dynamic context awareness modeling is used to dynamically model the context semantic information of professional nouns. Specifically, according to the professional noun knowledge group, a generative adversarial verification network combining improved attention and dynamic context encoding is used for dynamic context awareness modeling to obtain professional noun dynamic context awareness data, including the following steps:

[0103] Step S21: Construct a hierarchical attention mechanism, specifically by constructing semantic layer attention and syntactic layer attention, calculating the semantic hierarchical attention feature and syntactic hierarchical attention feature, and through weighted synthesis, constructing a hierarchical attention mechanism to obtain hierarchical context basic information data;

[0104] The calculation formula for the hierarchical context basic information data is as follows:

[0105] ;

[0106] In the formula, is the hierarchical context basic information data of the i-th professional noun vector, is the syntactic-level attention weight, is the semantic-level attention feature of the i-th professional noun vector, is the semantic-level attention weight, is the syntactic-level attention feature of the i-th professional noun vector;

[0107] Step S22: Construct a transformer encoder, specifically by constructing a standard transformer model and combining the variational inference method to capture context information and obtain the transformer encoding feature output data;

[0108] Step S23: Construct a temporal convolutional dynamic model, specifically by constructing a standard temporal convolutional network data and performing temporal convolutional modeling based on the transformer encoding feature output data to obtain the temporal convolutional feature data;

[0109] Step S24: Dynamic context encoding, specifically by combining the transformer encoding feature output data and the temporal convolutional feature data to perform dynamic context encoding and obtain the dynamic context encoding feature data. The calculation formula is as follows:

[0110] ;

[0111] In the formula, C is the dynamic context encoding feature data, Encoder(·) is the transformer encoder function, X is the professional noun knowledge group data input as the original data, and TCN(·) is the temporal convolutional dynamic model function;

[0112] Step S25: Construct a generative adversarial model for generating synonym replacement candidate words, specifically by constructing a generator model and a discriminator model, and combining the generative adversarial loss to generate synonym replacement candidate words based on the dynamic context encoding feature data to obtain the synonym replacement candidate word data;

[0113] Step S26: Construct a semantic similarity verification network for evaluating the quality of replacement candidate words, specifically by constructing a pre-similarity to perform semantic similarity verification and obtain the semantic similarity evaluation reference data;

[0114] Step S27: Training of the dynamic context awareness modeling model. Specifically, through an end-to-end training strategy, a comprehensive loss function is constructed, and the dynamic context awareness modeling model is trained to obtain a professional term dynamic context awareness model. By using the professional term dynamic context awareness model, professional term dynamic context awareness data is obtained;

[0115] The calculation formula of the comprehensive loss function is:

[0116] ;

[0117] In the formula, L total Comprehensive loss function, L GAN is the generative adversarial loss function, is the cross-modal alignment loss weight, L align is the cross-modal alignment loss function, is the semantic similarity loss weight, L seman is the semantic similarity loss function.

[0118] By performing the above operations, in the existing dynamic context awareness modeling process, there are existing context information processing methods that usually focus on the centralized processing of short-term dependencies and the content understanding of fixed contexts. This has led to the situation that traditional methods can only capture basic grammar and semantic information when dealing with professional terms, and thus cannot fully consider the language changes and differences brought by more refined and complex domain professional terms, restricting the scope and modeling ability of context awareness, and thus affecting the replacement accuracy and optional range of professional terms. To solve this technical problem, this solution creatively adopts a generative adversarial verification network that combines improved attention and dynamic context encoding for dynamic context awareness modeling. Through a two-layer hierarchical attention mechanism of semantics and grammar, combined with feature optimization of transform coding and temporal convolutional networks, context information is jointly extracted, and finally a generative adversarial network is used to generate replacement words, improving the dynamics of the context information modeling of professional terms and the accuracy of semantic perception.

[0119] Example 4, refer to Figure 1 、 Figure 2 and Figure 5 Based on the above embodiment, in step S3, the domain adaptation optimization is used to optimize the domain adaptability of professional terms. Specifically, according to the professional term knowledge group, a lightweight model adaptation improvement method combined with domain classification is adopted for domain adaptation optimization to obtain domain adaptation model data, including the following steps:

[0120] Step S31: Construct a lightweight adaptation module. Specifically, introduce parameter sharing and adaptive adjustment strategies, and replace the standard graph convolutional neural network with the lightweight adaptation module to generate adaptive domain vectors, obtaining a lightweight adaptation semantic vector output. The calculation formula is:

[0121] ;

[0122] In the formula, V adapted is the lightweight adaptation semantic vector output, M(·) is the lightweight adaptation module function, V input is the original data input of the lightweight adaptation module, which is used to represent the professional noun vector data in the professional noun knowledge group. is the adaptation parameter, f adapt (·) is the adaptation function of the adaptation module;

[0123] Step S32: Construct a domain classifier for dynamically activating the adaptation module. Specifically, construct a domain classifier to classify the lightweight adaptation semantic vector output to obtain domain category data, and perform dynamic activation based on the domain category data to obtain dynamic activation output data;

[0124] The calculation formula for the dynamic activation is:

[0125] ;

[0126] In the formula, is the dynamic activation output data, M(·) is the lightweight adaptation module function, V input is the original data input of the lightweight adaptation module, is the adaptation parameter, Softmax(·) is the domain classifier function, specifically using a softmax classifier, is the professional noun dynamic context awareness data;

[0127] Step S33: Elastic weight consolidation is used to optimize synonym replacement forgetting. Specifically, construct an elastic weight loss function to perform domain adaptation synonym replacement optimization. The calculation formula is:

[0128] ;

[0129] In the formula, L EWC is the elastic weight loss function, L adapt is the domain adaptation loss function, is the elastic weight adjustment parameter, I is the total number of Fisher matrix elements, which is used to represent the total number of professional noun vectors, i is the Fisher matrix element index, which is used to represent the professional noun vector index, F i is the Fisher matrix element corresponding to the i-th professional noun vector, is the adaptation parameter of the domain adaptation optimization model corresponding to the i-th professional noun vector, is the optimal parameter of the domain adaptation optimization model corresponding to the i-th professional noun vector;

[0130] Step S34: Extract the distribution characteristics of domain-specific nouns. Specifically, based on the dynamic activation output data, extract the distribution characteristics of domain-specific nouns to obtain the distribution characteristic data of domain-specific nouns;

[0131] Step S35: Improve the lightweight model adaptation. Specifically, make two-way improvements to the domain adaptation loss function for the domain task and domain adaptability to obtain the two-way improved loss function, and optimize the model training by constructing the lightweight model adaptation improvement loss function to train the lightweight adaptation model Model AD ;

[0132] The calculation formula of the two-way improved loss function is:

[0133] ;

[0134] In the formula, is the two-way improved loss function, which is used to represent the improvement of the domain adaptation loss function by combining the domain task loss function and the domain adaptability loss function, L task is the domain task loss function, specifically referring to the cross-entropy loss function in the domain classifier process, is the adjustment parameter, L domain is the domain knowledge extraction loss function, which is used as the adaptability loss function;

[0135] The calculation formula of the lightweight model adaptation improvement loss function is:

[0136] ;

[0137] In the formula, is the lightweight model adaptation improvement loss function, L task is the domain task loss function, is the domain knowledge extraction loss weight, L domain is the domain knowledge extraction loss function, which is used as the adaptability loss function, is the elastic weight loss weight, L EWC is the elastic weight loss function;

[0138] Step S36: Domain adaptation optimization. Specifically, use the lightweight adaptation model Model AD , perform domain adaptation optimization to obtain the domain adaptation model data.

[0139] By performing the above operations, in the process of adapting and optimizing the existing synonym replacement in the field, there is a technical problem that the existing methods are difficult to accurately adapt to the professional terms and their context differences in different fields, resulting in insufficient field adaptability. This solution creatively adopts a lightweight model adaptation improvement method combined with field classification. Through adaptive adjustment and elastic weight mechanism, the field adaptation effect is optimized, and the field adaptability and accuracy of synonym replacement are improved.

[0140] Example Five, refer to Figure 1 、 Figure 2 and Figure 6 , based on the above example, in step S4, the multi-dimensional output optimization is used to enhance the diversity of selectable professional noun replacements. Specifically, according to the dynamic context-aware data of the professional noun, a controllable semantic converter is constructed and used to perform multi-dimensional expansion and output optimization of professional noun replacement, and the multi-dimensional output optimized professional noun replacement reference data is obtained, including the following steps:

[0141] Step S41: Replacement strategy optimization, specifically introducing a replacement strategy optimization method based on the dynamic context-aware data of the professional noun. By optimizing the selection mechanism of replacement candidate words, the number of professional nouns with similar replacement semantics is optimized. Specifically, the maximum summation of the product of semantic similarity and the occurrence probability of candidate words is used for replacement strategy optimization;

[0142] Step S42: Enhancement of replacement noun explanation, specifically in the process of the above replacement strategy optimization, by adding the calculation of semantic similarity of noun explanation information, the interpretability of replacement nouns is optimized;

[0143] Step S43: Multi-dimensional replacement post-processing, specifically by introducing syntax consistency checking and semantic consistency checking for multi-dimensional replacement post-processing to optimize the semantic deviation processing ability;

[0144] Step S44: Multi-dimensional output optimization, specifically through the combination of syntactic features, semantic features and context features of formaldehyde, multi-dimensional output optimization is performed to obtain multi-dimensional output optimized professional noun replacement reference data.

[0145] Example Six, refer to Figure 1 and Figure 2 , based on the above example, in step S5, the professional noun synonym replacement assistance is used to provide synonym replacement assistance. Specifically, by receiving the original input sentence, through the dynamic context-aware modeling and the field adaptation optimization, domain-adapted professional noun synonym replacement generation is performed, and through multi-dimensional output optimization, professional noun synonym replacement comprehensive reference assistance data is obtained.

[0146] Example Seven, refer to Figure 1 andFigure 2 , based on the above embodiments, the system for assisting in synonym replacement of professional terms based on deep learning provided by the present invention includes an interaction layer, a processing layer, a knowledge management layer, and an output optimization layer;

[0147] The interaction layer is used to construct a multimodal input interface and provide visual interaction. Through the interaction layer, the original data input of the user is received and sent to the processing layer for synonym replacement of professional terms;

[0148] The processing layer is used for knowledge fusion pre-training and dynamic context-aware modeling. Through the processing layer, a professional term knowledge group and professional term dynamic context-aware data are obtained, and the professional term knowledge group is sent to the knowledge management layer, and the professional term dynamic context-aware data is sent to the output optimization layer;

[0149] The knowledge management layer is used for domain adaptation optimization. Through domain adaptation optimization, domain adaptation model data is obtained and sent to the output optimization layer;

[0150] The output optimization layer is used for multi-dimensional output optimization. Through multi-dimensional output optimization, multi-dimensional output optimized professional term replacement reference data is obtained.

[0151] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0152] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

[0153] The above describes the present invention and its implementation manners. Such description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the purpose of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A professional term synonym replacement auxiliary method based on deep learning, characterized by: The method comprises the following steps: Step S1: knowledge fusion pre-training, through professional noun data collection and knowledge fusion model pre-training, a professional noun knowledge group is obtained, the professional noun knowledge group includes a professional noun pre-training model and professional noun vector data, including the following steps: building a dual-channel pre-training architecture, cross-modal alignment, domain knowledge extraction and knowledge fusion pre-training; Step S2: Dynamic context-aware modeling, which is used to dynamically model the contextual semantic information of professional nouns. Specifically, based on the professional noun knowledge group, a generative adversarial verification network combining improved attention and dynamic context encoding is used to perform dynamic context-aware modeling to obtain dynamic context-aware data of professional nouns; Step S3: domain adaptation optimization, which is used to optimize the domain adaptability of professional terms. Specifically, based on the professional term knowledge group, a lightweight model adaptation improvement method combined with domain classification is used to perform domain adaptation optimization to obtain domain adaptation model data; Step S4: multi-dimensional output optimization, used to enhance the diversity of alternative choices for professional nouns, specifically, based on the dynamic context perception data of the professional nouns, construct and use a controllable semantic converter to perform multi-dimensional expansion and output optimization of professional noun replacement, and obtain multi-dimensional output optimized professional noun replacement reference data; Step S5: Assist in synonymous replacement of professional terms, and obtain comprehensive reference auxiliary data for synonymous replacement of professional terms.

2. The method for assisting synonymous replacement of professional terms based on deep learning according to claim 1 is characterized in that: In step S1, the professional term data collection includes text data collection and knowledge graph construction; The text data collection includes collecting text data, including professional literature data, professional technical document data and domain corpus data; the knowledge graph construction includes constructing professional noun node parameters and semantic relationship edge parameters to obtain knowledge graph data; The knowledge fusion model pre-training includes the following steps: Step S11: constructing a dual-channel pre-training architecture, specifically constructing a dual-channel model, including a text processing channel and a graph information processing channel, and performing dual-channel pre-training by constructing a dual-channel pre-training objective function to obtain a knowledge fusion pre-training basic model; Step S12: cross-modal alignment, specifically, aligning the information of text data and the information of knowledge graph data during the pre-training process through a multimodal learning method, and optimizing the consistency of text features and graph data features through a cross-modal alignment loss function to obtain an alignment optimization knowledge fusion pre-training model; Step S13: extracting domain knowledge, specifically using a standard graph neural network to extract domain knowledge feature data from the domain corpus data in the text data, and using the domain knowledge feature data for domain adaptation optimization; Step S14: Knowledge fusion pre-training, specifically combining text features, graph data features and domain knowledge feature data, and performing knowledge fusion pre-training through the fusion target loss function to obtain a professional term pre-training model Model PRE , and by using the professional term pre-training model Model PRE , construct professional noun vectors and obtain professional noun knowledge groups; The professional noun vector data specifically refers to the professional noun semantic vector obtained by modeling the semantics of the professional noun through cosine similarity.

3. The method for assisting synonymous replacement of professional terms based on deep learning according to claim 2 is characterized in that: In step S2, the dynamic context-aware modeling includes the following steps: Step S21: constructing a hierarchical attention mechanism, specifically, by constructing semantic layer attention and grammatical layer attention, calculating semantic layer attention features and grammatical layer attention features, and constructing a hierarchical attention mechanism through weighted synthesis to obtain hierarchical context basic information data; The calculation formula of the hierarchical context basic information data is: ； In the formula, is the hierarchical contextual basic information data of the i-th professional noun vector, is the grammatical level attention weight, is the semantic level attention feature of the i-th professional noun vector, is the semantic level attention weight, is the grammatical level attention feature of the i-th professional noun vector; Step S22: constructing a transformer encoder, specifically by constructing a standard transformer model and capturing context information by combining a variational inference method to obtain transformer encoding feature output data; Step S23: constructing a time convolution dynamic model, specifically constructing standard time convolution network data, and performing time convolution modeling according to the voltage-converting coding feature output data to obtain time series convolution feature data; Step S24: Dynamic context coding, specifically combining the voltage-converting coding feature output data and the temporal convolution feature data to perform dynamic context coding to obtain dynamic context coding feature data, and the calculation formula is: ； Where C is the dynamic context encoding feature data, Encoder(·) is the variable voltage encoder function, X is the professional term knowledge group data as the original data input, and TCN(·) is the time convolution dynamic model function; Step S25: constructing a generative adversarial model for generating synonym replacement candidate words, specifically constructing a generator model and a discriminator model, and combining the generative adversarial loss, generating synonym replacement candidate words according to the dynamic context encoding feature data, and obtaining synonym replacement candidate word data; Step S26: constructing a semantic similarity verification network to evaluate the quality of candidate replacement words, specifically by constructing a pre-similarity, performing semantic similarity verification, and obtaining semantic similarity evaluation reference data; Step S27: training a dynamic context-aware modeling model, specifically, constructing a comprehensive loss function through an end-to-end training strategy, and training a dynamic context-aware modeling model to obtain a dynamic context-aware model of professional nouns, and obtaining dynamic context-aware data of professional nouns by using the dynamic context-aware model of professional nouns; The calculation formula of the comprehensive loss function is: ； Where, L total Comprehensive loss function, L GAN is the generative adversarial loss function, is the cross-modal alignment loss weight, L align is the cross-modal alignment loss function, is the semantic similarity loss weight, L seman is the semantic similarity loss function.

4. The method for assisting synonym replacement of professional terms based on deep learning according to claim 3 is characterized in that: In step S3, the domain adaptation optimization comprises the following steps: Step S31: construct a lightweight adaptation module, specifically introduce parameter sharing and adaptive adjustment strategies, and replace the standard graph convolutional neural network with the lightweight adaptation module to generate adaptive domain vectors, and obtain lightweight adaptive semantic vector output; Step S32: constructing a domain classifier for dynamically activating the adaptation module, specifically constructing a domain classifier, classifying the lightweight adaptation semantic vector output to obtain domain category data, and dynamically activating according to the domain category data to obtain dynamically activated output data; Step S33: elastic weight consolidation, which is used to optimize synonym replacement forgetting, specifically constructing an elastic weight loss function to perform domain-adaptive synonym replacement optimization; Step S34: extracting the distribution features of domain professional terms, specifically extracting the distribution features of domain professional terms based on the dynamic activation output data to obtain the distribution feature data of domain professional terms; Step S35: Lightweight model adaptation improvement, specifically, bidirectional improvement of the domain adaptation loss function in terms of domain tasks and domain adaptability, obtaining a bidirectional improved loss function, and optimizing model training by constructing a lightweight model adaptation improved loss function to obtain a lightweight adaptation model Model AD ; The calculation formula of the two-way improved loss function is: ； In the formula, is a two-way improved loss function, which is used to indicate the improvement of the domain adaptation loss function by combining the domain task loss function and the domain adaptability loss function. task It is the domain task loss function, specifically the cross entropy loss function in the domain classifier process. is the adjustment parameter, L domain It is the domain knowledge extraction loss function, which is used as the adaptability loss function; The calculation formula of the lightweight model adaptation improvement loss function is: ； In the formula, is the lightweight model adaptation improvement loss function, L task is the domain task loss function, is the domain knowledge extraction loss weight, L domain is the domain knowledge extraction loss function, used as the adaptability loss function, is the elastic weight loss weight, L EWC is the elastic weight loss function; Step S36: Domain adaptation optimization, specifically using the lightweight adaptation model Model AD , perform domain adaptation optimization and obtain domain adaptation model data.

5. The method for assisting synonymous replacement of professional terms based on deep learning according to claim 4 is characterized in that: In step S4, the multi-dimensional output optimization includes the following steps: Step S41: Replacement strategy optimization, specifically introducing a replacement strategy optimization method based on the dynamic context perception data of the professional nouns, optimizing the selection mechanism of the replacement candidate words, optimizing the number of professional nouns with similar replacement semantics, and specifically optimizing the replacement strategy by maximizing the sum of the product of semantic similarity and the probability of occurrence of the candidate words; Step S42: enhancing the explanation of the replacement noun, specifically, in the process of optimizing the replacement strategy, by adding semantic similarity calculation of the noun explanation information to optimize the explainability of the replacement noun; Step S43: multi-dimensional replacement post-processing, specifically, by introducing syntax consistency check and semantic consistency check, multi-dimensional replacement post-processing is performed to optimize the semantic deviation processing capability; Step S44: multi-dimensional output optimization, specifically, multi-dimensional output optimization is performed by combining grammatical features, semantic features and context features to obtain multi-dimensional output optimized professional term replacement reference data.

6. The method for assisting synonym replacement of professional terms based on deep learning according to claim 5 is characterized by: In step S5, the professional term synonym replacement assistance is used to provide synonym replacement assistance, specifically by receiving the original input sentence, performing domain-adaptive professional term synonym replacement generation through the dynamic context-aware modeling and the domain adaptation optimization, and obtaining professional term synonym replacement comprehensive reference auxiliary data through multi-dimensional output optimization.

7. A professional noun synonym replacement auxiliary system based on deep learning, used to implement a professional noun synonym replacement auxiliary method based on deep learning as described in any one of claims 1 to 6, characterized in that: It includes interaction layer, processing layer, knowledge management layer and output optimization layer.

8. The deep learning-based synonym replacement auxiliary system for professional terms according to claim 7 is characterized by: The interaction layer is used to construct a multimodal input interface and provide visual interaction. Through the interaction layer, the user's original data input is accepted and sent to the processing layer for synonym replacement of professional terms. The processing layer is used for knowledge fusion pre-training and dynamic context-aware modeling. Through the processing layer, a professional noun knowledge group and professional noun dynamic context-aware data are obtained, and the professional noun knowledge group is sent to the knowledge management layer, and the professional noun dynamic context-aware data is sent to the output optimization layer; The knowledge management layer is used for domain adaptation optimization, and domain adaptation model data is obtained through domain adaptation optimization, and the domain adaptation model data is sent to the output optimization layer; The output optimization layer is used for multi-dimensional output optimization, and multi-dimensional output optimization professional terminology replacement reference data is obtained through multi-dimensional output optimization.

Images