A deep learning-based text data denoising method, electronic device and computer program product

Abstract

The application relates to a deep learning-based text data noise reduction method, an electronic device and a computer program product, and belongs to the technical field of natural language processing. The method steps comprise: converting an original text sequence into a semantic vector representation, performing similarity measurement calculation, constructing a corresponding semantic similarity matrix, applying a DBSCAN algorithm for density clustering processing, generating a hierarchical label, obtaining a semantic correlation weight vector and performing noise suppression, performing attention enhancement on the semantic correlation weight vector based on a Transformer architecture, and outputting a noise reduction text. According to the embodiment of the application, the word semantic correlation weight is used as an importance basis, a differential noise reduction is realized by using a decay function, and an enhanced attention mechanism is combined to further strengthen the modeling of key information. The text quality is optimized from the source by noise reduction, differential noise reduction is realized, and the method is more suitable for complex scenarios.

Description

Technical Field

[0001] This application relates to a deep learning-based text data noise reduction method, electronic devices, and computer program products, belonging to the technical field of natural language processing. Background Technology

[0002] Currently, the mainstream text data denoising methods in existing technologies mainly include rule-based methods, such as regular expression matching and stop word filtering; statistical methods, such as TF-IDF and frequency statistics; clustering methods, such as K-Means and DBSCAN; and neural network-based methods, such as BERT denoising and Transformer self-attention mechanism, etc.

[0003] A search revealed prior art closest to the embodiments of this application, including: "CN113420154A Method for Constructing a Hierarchical Multi-Label Text Classification Model Based on Hierarchical Attention" and "CN118193743A A Multi-Level Text Classification Model Based on a Pre-trained Model." Both patents involve "hierarchy," with the former also employing an "attention mechanism." However, their technical essence is "multi-level text classification," aiming to improve classification accuracy rather than cleaning or reconstructing the text content. The following is a further analysis of the main deficiencies in the aforementioned prior applications: Existing technology 1: CN113420154A, a method for constructing a hierarchical multi-label text classification model based on hierarchical attention. Its main drawbacks are as follows: 1. Noise-free recognition and suppression mechanism: The technical solution provided by the existing technology is completely focused on "classification prediction". It only integrates the semantic relationship between text and label through hierarchical attention and finally outputs classification label (such as "technology → artificial intelligence"). It does not involve any recognition or processing of noise in the original text (such as spelling errors, irrelevant words, and repetitive expressions), and cannot solve the core need of purifying text.

[0004] 2. Failure to optimize original text content: The existing technology explicitly states that it does not change the structure and content of the input text. Its attention mechanism is only used for feature weighting to assist classification decisions (such as strengthening text features related to labels), rather than judging whether terms are noise and retaining or deleting them. Therefore, even if the original text contains noise, this technology will include the noise in the classification features, which may lead to classification errors that cannot be optimized for text quality from the source through noise reduction.

[0005] 3. Inability to handle semantic redundancy: The hierarchical structure of the existing technology is based on "tag membership" (such as the inclusion relationship between first-level tags and second-level tags), rather than the semantic association of text content. The model cannot identify redundancy in the original text, such as repetitive expressions and semantically irrelevant information, nor can it suppress it through hierarchical clustering or weight adjustment. It also cannot divide clusters based on semantic similarity and specifically address redundant noise.

[0006] Existing technology 2: CN118193743A, a multi-level text classification model based on a pre-trained model. 1. Lack of text cleansing capabilities: The existing technology's solution revolves entirely around "classification label prediction" (such as encoding text through pre-trained models and dynamically adjusting the mapping matrix to optimize secondary classification), without designing any modules to handle noise in the original text. For spelling errors, irrelevant words, etc., in the input text, the model directly encodes the noisy text as features, which may lead to feature contamination, affecting classification accuracy. It cannot dynamically suppress noise through semantic weights, making it difficult to improve text quality from the source.

[0007] 2. "Dynamic mapping" does not involve semantic noise reduction: The core innovation of this patent is "dynamically adjusting the mapping matrix of the secondary classification based on the primary classification result." Its essence is to optimize the association logic between tags, rather than processing the text content itself. Even if the original text has semantic conflicts or redundancy, this solution cannot distinguish key information from noise through weight adjustment or attention mechanisms. It does not strengthen the weight of highly relevant terms (key information) or weaken noisy terms by introducing a semantic prior matrix.

[0008] 3. Inability to adapt to "unstructured text noise" scenarios: The preprocessing in this patent only includes basic operations such as "removing special characters, punctuation marks, and stop words," and does not involve semantic-based deep noise reduction (such as identifying context-independent words and repairing semantically conflicting expressions). Its solution cannot handle complex semantic noise in unstructured texts such as logs and conversations, nor can it combine with noise suppression functions to achieve differentiated noise reduction, making it difficult to adapt to complex scenarios.

[0009] In conclusion, existing technologies can no longer meet people's needs and urgently need to be improved. Summary of the Invention

[0010] This application addresses the deficiencies mentioned in the background technology by providing a text data noise reduction method, electronic device, and computer program product based on deep learning. It aims to solve the text analysis and understanding errors caused by inaccurate noise recognition, easy loss of key information, and lack of fine-grained control during the text analysis process.

[0011] The embodiments of this application are implemented using the following technical solutions: According to one aspect of the embodiments of this application, a deep learning-based text data denoising method is provided, comprising: acquiring an original text sequence; encoding the original text sequence using a pre-trained language model to convert the original text sequence into a semantic vectorized representation; calculating the similarity measure of the vectors in the semantic vectorized representation; constructing a corresponding semantic similarity matrix based on the result of the similarity measure calculation; applying the DBSCAN algorithm to perform density clustering processing on the semantic similarity matrix to generate hierarchical labels; calculating the global semantic importance score of each term in the semantic vectorized representation to obtain a semantic relevance weight vector; performing noise suppression on the hierarchical labels and the semantic relevance weight vector; and performing attention enhancement on the semantic relevance weight vector based on the Transformer architecture to output denoised text.

[0012] According to at least one specific embodiment of the present application, the step of obtaining the original text sequence, encoding the original text sequence through a pre-trained language model, and converting the original text sequence into a semantic vectorized representation further includes: mapping the original text sequence into a semantic vectorized representation in a low-dimensional continuous vector space through a pre-trained language model, wherein the semantic vectorized representation is used to characterize the overall semantics of the text, and the semantic vectorized representation includes sentence-level semantic vector representation and word-level word-level embedding vector sequence, wherein the word-level embedding vector sequence is used to characterize the context-related semantics of terms in the text.

[0013] According to at least one specific embodiment of the present application, the step of performing similarity measurement calculation on the vectors in the semantic vectorization representation and constructing a corresponding semantic similarity matrix based on the result of the similarity measurement calculation further includes: the similarity measurement calculation is a cosine similarity calculation, calculating the cosine similarity between any two vectors, and constructing a corresponding semantic similarity matrix based on the result of the cosine similarity calculation.

[0014] According to at least one specific embodiment of the present application, the step of applying the DBSCAN algorithm to perform density clustering processing on the semantic similarity matrix to generate hierarchical labels, calculating the global semantic importance score of each term in the semantic vectorized representation, and obtaining a semantic relevance weight vector, further includes: applying the DBSCAN algorithm to perform density clustering processing on the semantic similarity matrix, hierarchically dividing the semantic vectorized representation based on semantic similarity to form different semantic clusters, and generating corresponding hierarchical labels; sending the hierarchical labels to a noise suppression module, wherein the noise suppression module is used to nonlinearly attenuate the word frequency according to the semantic weight.

[0015] According to at least one specific implementation of the embodiments of this application, the global semantic importance score includes: semantic similarity to all other terms, and semantic relevance weight vector obtained after normalization.

[0016] According to at least one specific embodiment of the present application, the step of noise suppression of the hierarchical labels and the semantic relevance weight vector, and attention enhancement of the semantic relevance weight vector based on the Transformer architecture to output denoised text further includes: sending the hierarchical labels to a noise suppression module, the noise suppression module performing noise suppression through an exponential decay function, marking the current word frequency as noise and deleting it if the current word frequency is higher than a preset threshold, and sending the noise-attenuated word frequency or Boolean mask to an output module; introducing a semantic prior matrix based on the Transformer architecture, performing attention enhancement on the semantic relevance weight vector, and outputting attention-enhanced denoised text.

[0017] According to at least one specific embodiment of the present application, the noise suppression of the hierarchical labels and the semantic relevance weight vector further includes: the noise suppression module filters noise samples through hierarchical labels and filters noise words through semantic weights.

[0018] According to another aspect of the embodiments of this application, a deep learning-based text data denoising system is provided to implement the deep learning-based text data denoising method, comprising: a semantic vectorization representation module, which acquires the original text sequence, encodes the original text sequence through a pre-trained language model, and converts the original text sequence into a semantic vectorization representation; a semantic similarity matrix construction module, which performs similarity measurement calculation on the vectors in the semantic vectorization representation, and constructs a corresponding semantic similarity matrix based on the result of the similarity measurement calculation; a semantic relevance weight vector generation module, which applies the DBSCAN algorithm to perform density clustering processing on the semantic similarity matrix to generate hierarchical labels, calculates the global semantic importance score of each term in the semantic vectorization representation, and obtains a semantic relevance weight vector; and an attention-enhanced denoised text output module, which performs noise suppression on the hierarchical labels and the semantic relevance weight vector, and performs attention enhancement on the semantic relevance weight vector based on the Transformer architecture to output denoised text.

[0019] According to another aspect of the embodiments of this application, an electronic device is provided, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method.

[0020] According to another aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described herein.

[0021] The beneficial technical effects of the embodiments of this application are: This application proposes a collaborative mechanism of hierarchical clustering, semantic weighting, and graded noise reduction. It achieves text hierarchical division through semantic similarity clustering, uses the semantic relevance weight of terms as the basis for importance, utilizes a decay function to achieve differentiated noise reduction, and combines an enhanced attention mechanism to further strengthen the modeling of key information. This mechanism can identify or process noise in the original text, optimize text quality from the source through noise reduction, target redundant noise, strengthen the weight of highly relevant key information, weaken noisy terms, achieve differentiated noise reduction, and is more adaptable to complex scenarios.

[0022] This application focuses on text denoising, repair, and purification, filling the existing technological gap in "hierarchical semantic denoising." It specifically addresses the problems of inaccurate noise suppression and easy loss of key information in existing denoising methods. By dynamically adjusting the denoising intensity through semantic weights and noise suppression functions, it performs weak denoising on highly semantically relevant terms (key information) and strong denoising on low-relevance terms (noise), avoiding the problem of key information loss caused by a one-size-fits-all approach to denoising. Furthermore, it can capture semantic cluster associations through hierarchical clustering (DBSCAN) for non-continuous, multi-topic texts (such as multi-turn dialogues and mixed-topic logs), and enhance the attention of key information across clusters by combining an attention enhancement module (introducing a semantic prior matrix), greatly expanding the application scenarios of this application.

[0023] This application applies the DBSCAN clustering algorithm to semantic grouping before text denoising, performing hierarchical clustering instead of simply using it for classification or anomaly detection as in existing technologies. The DBSCAN clustering results directly influence the selection of subsequent denoising strategies, and can be globally set or dynamically adjusted hierarchically through parameter control to adapt to sparse or dense semantic distributions. While performing clustering through DBSCAN, this application also distinguishes key information from noise through weight adjustment and attention mechanisms, and introduces a semantic prior matrix to strengthen the weight of highly relevant terms (key information) while weakening noisy terms. That is, it first stratifies according to semantic features, and then performs hierarchical denoising based on the importance of the text, achieving high-fidelity preservation of key information and precise noise suppression. Attached Figure Description

[0024] Figure 1 This is a flowchart of a text data noise reduction method based on deep learning.

[0025] Figure 2This is a flowchart of the optimization technical solution provided in steps S31 to S32.

[0026] Figure 3 This is a flowchart of the optimization technical solutions provided in steps S41 to S43.

[0027] Figure 4 This is an architecture diagram of a text data denoising system based on deep learning.

[0028] Figure 5 This is a schematic diagram of the electronic device. Detailed Implementation

[0029] The present application / disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application / disclosure and are not intended to limit the scope of the present application / disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application / disclosure are shown in the accompanying drawings, not the entire structure.

[0030] The main idea of ​​this application's embodiments is based on a dual architecture of hierarchical structure and grouping, combined with quantitative modeling for noise suppression, to achieve quantitative control that "the higher the semantic importance, the lower the noise reduction level." During hierarchical division, DBSCAN clustering is used to achieve hierarchical division of text semantics, solving the problem of semantic fragmentation in unstructured text. During classification, the noise reduction intensity is dynamically adjusted based on semantic weights to achieve refined noise processing, overcoming the limitations of traditional rule filtering or single-threshold noise reduction. Semantic-driven attention enhancement is also implemented, introducing a semantic prior matrix into the Transformer, using semantic relevance as prior knowledge for attention calculation, strengthening the weights of high-value terms, and improving noise reduction accuracy. During quantitative modeling of noise suppression, an exponential function-based noise suppression function is used to correlate semantic weights with word frequency attenuation intensity, making the noise reduction process interpretable and controllable.

[0031] Definitions: Text encoding module: Converts raw unstructured text into semantic vector representations.

[0032] Hierarchical clustering module: Based on semantic similarity, text samples are hierarchically divided into different semantic clusters.

[0033] Semantic weight calculation module: Calculates the semantic importance weight of each term in the context.

[0034] Noise Suppression Module: This module performs non-linear decay of word frequencies based on semantic weights to achieve soft noise reduction. Soft noise reduction refers to a noise reduction method that does not directly delete terms using hard thresholds. Instead, it continuously and non-linearly decays word frequencies based on semantic weights, ensuring that terms with high semantic weights maintain or enhance their frequency contribution, while the frequencies of terms with low semantic weights are compressed rather than completely removed. Soft noise reduction preserves the potential semantic information of low-weight words while avoiding the information loss caused by hard truncation.

[0035] Attention Enhancement Module: Introduces a semantic prior matrix into the Transformer to increase the attention given to keyword items. Output module: Generates the final denoised text. The input to the output module comes from the mask from the noise suppression module and the context representation from the attention enhancement module.

[0036] like Figure 1 The deep learning-based text data denoising method shown includes: Step S1: Obtain the original text sequence, encode the original text sequence using a pre-trained language model, and convert the original text sequence into a semantic vectorized representation.

[0037] Step S2: Perform similarity measurement calculation on the vectors in the semantic vectorization representation, and construct the corresponding semantic similarity matrix based on the result of the similarity measurement calculation.

[0038] Step S3: Apply the DBSCAN algorithm to perform density clustering on the semantic similarity matrix to generate hierarchical labels L. k We calculate the global semantic importance score for each term in the semantic vectorized representation to obtain the semantic relevance weight vector β.

[0039] Step S4: Perform noise suppression on the hierarchical labels and the semantic relevance weight vector, and perform attention enhancement on the semantic relevance weight vector based on the Transformer architecture to output the denoised text X. clean .

[0040] The technical solutions provided in steps S1 to S4, through a unified representation framework of semantic vectorization, integrate density clustering and global semantic weight calculation to construct a processing flow from text encoding to noise suppression and attention enhancement, wherein: Step S1 realizes the semantic vectorization representation of the original text sequence. The semantic vectorization representation output by step S1 serves as the input for subsequent steps. Step S2 obtains the semantic vector representation of the text samples for calculating the similarity between texts. Step S3 obtains the term embedding vector sequence for calculating the global importance of terms. Steps S2 and S3 parse semantic information from the text-level and word-level dimensions, respectively. Step S2 outputs the hierarchical label L from the dimension of density clustering processing.k This paper utilizes DBSCAN's clustering results for semantic grouping before text denoising, rather than solely for classification or anomaly detection. The clustering results directly influence the selection of subsequent denoising strategies and support dynamic adjustment of eps and min_samples to adapt to sparse or dense semantic distributions. The weighting part outputs a semantic relevance weight vector β, proposing a method for explicitly calculating semantic relevance between terms and using it as a cross-module control signal. Steps S1, S2, and S3 work together to regulate multiple subsystems, such as denoising intensity and attention bias.

[0041] Step S2 constructs a semantic similarity matrix. After transforming the semantic similarity matrix, it is input into the DBSCAN clustering algorithm. Using the two parameters of neighborhood radius eps and minimum number of points min_samples, isolated points in the semantic space are identified and marked as noise samples. No need to preset the number of clusters or manually label them. Noise can be identified based solely on semantic distribution density. It can capture the isolation at the semantic level, rather than just counting anomalies.

[0042] Step S3 implements soft noise reduction based on semantic centrality. The semantic weight vector β output by step S3 reflects the degree of centrality of each term in the global semantic space. The strength of semantic centrality and marginality of a term can be determined based on the value of β. By continuously weighting and attenuating terms through semantic weights, low-weight noise words are suppressed, while high-weight core words are retained, resulting in a smooth noise reduction effect.

[0043] Step S4 is used to achieve cascaded noise reduction, while utilizing the hierarchical label L. k Two-stage noise suppression is performed using the semantic weight vector β: the first stage of noise suppression is based on the hierarchical label L. k The first level filters the entire document, while the second level filters low-importance terms within the document based on beta filtering. Through the two-level collaboration of hierarchical tags and semantic weights, a layer-by-layer purification process is formed, which first filters the document and then cleans the words. This avoids wasting computational resources on noisy documents while ensuring that the key information within the document is preserved.

[0044] Step S4 also incorporates the semantic weight vector β into the attention calculation of the Transformer architecture. When calculating attention, it automatically focuses on terms with strong semantic centrality, providing prior knowledge of which terms the attention mechanism should pay attention to. This makes up for the shortcomings of the traditional Transformer, which is entirely data-driven, and improves the semantic rationality of attention allocation.

[0045] In summary, steps S1 to S4, through a unified framework of semantic vectorization representation, achieve overall and local multi-granularity parsing of text semantics. On this basis, a cascaded noise reduction mechanism at the sample level and word level is constructed, and attention enhancement is guided by semantic priors, forming a complete text noise reduction processing flow from encoding, parsing, noise reduction to attention enhancement.

[0046] Preferably, in step S1, the original text sequence is encoded using a pre-trained language model to convert the original text sequence into a semantic vectorized representation, further including: The original text sequence is mapped to a semantic vectorized representation in a low-dimensional continuous vector space by a pre-trained language model. The semantic vectorized representation is used to characterize the overall semantics of the text. The semantic vectorized representation includes sentence-level semantic vector representation and word-level word-level embedding vector sequence. The word-level embedding vector sequence is used to characterize the context-related semantics of terms in the text.

[0047] In the optimization scheme of step S1, the original unstructured text is converted into a semantic vector representation, and its input is: the original text sequence. For example, "This is a test text containing noise." The original text sequence is processed by using a pre-trained language model (such as BERT or RoBERTa) to encode the text, outputting a sequence of word-level embedding vectors. The vector sequence V is then transmitted to the hierarchical clustering module and the semantic weight calculation module.

[0048] The hierarchical clustering module performs density clustering on text samples based on semantic similarity, grouping semantically similar texts into the same cluster and labeling semantically isolated texts as noise. The hierarchical clustering module outputs the hierarchical label L. k It can identify the cluster affiliation or noise state of each text, providing sample-level semantic grouping information for subsequent steps and supporting whole-document filtering in noise suppression.

[0049] The semantic weight calculation module calculates the semantic importance of each term in the text. By aggregating and normalizing the similarity between the term and all other terms, it obtains a semantic relevance weight vector β. A larger semantic relevance weight vector β indicates a stronger semantic centrality of the term and its closer proximity to the core theme of the text. The semantic weight calculation module outputs a word-level semantic importance distribution, providing corresponding control signals for subsequent noise suppression and attention enhancement steps.

[0050] The hierarchical clustering module performs density clustering on text samples based on semantic similarity, grouping semantically similar texts into the same cluster and labeling semantically isolated texts as noise. The hierarchical clustering module outputs the hierarchical label L. kIt can identify the cluster affiliation or noise status of each text, providing sample-level semantic grouping information for subsequent steps, and performing noise suppression and filtering on the entire document.

[0051] Preferably, in step S2, the step of performing a similarity measurement calculation on the vectors in the semantic vectorized representation, and constructing a corresponding semantic similarity matrix based on the result of the similarity measurement calculation, further includes: The similarity metric is calculated as cosine similarity, which calculates the cosine similarity between any two vectors, and constructs the corresponding semantic similarity matrix based on the result of the cosine similarity calculation.

[0052] Step S2 aims to hierarchically divide text samples based on semantic similarity, forming different semantic clusters. In the optimized version of step S2, the input is a sequence of word-level embedding vectors from the text encoding module. Word-level embedding vector sequence It can be at the sentence level or the document level.

[0053] The specific processing steps of the optimization scheme in step S2 are as follows: Calculate the cosine similarity between any two vectors:

[0054] Construct a semantic similarity matrix S∈ (m is the number of texts); Density clustering using the DBSCAN algorithm: Parameters: neighborhood radius eps, minimum number of points min_samples Clustering results: Each text was assigned a hierarchical label. ∈{ , ,...}; Signal output: Hierarchical label The similarity matrix S is optionally sent to the semantic weight calculation module for initialization of the semantic weight calculation module.

[0055] In the optimization scheme of step S2, DBSCAN clustering is used for semantic grouping before text denoising for the first time, rather than just for classification or anomaly detection. The clustering results after step S2 will directly affect the selection of subsequent denoising strategies (such as using different γ parameters at different levels). It supports dynamic adjustment of eps and min_samples to adapt to sparse or dense semantic distribution.

[0056] As a further improvement to the optimization scheme of step S2, if an "automatic parameter tuning module" is added to the system to optimize parameters such as eps / min_samples, then the method flow needs to add a step: after clustering, silhouette coefficient evaluation is introduced to provide feedback and adjust parameters, forming a closed loop. The specific processing flow is as follows: DBSCAN clustering is performed on the semantic similarity matrix using a preset combination of initial parameters to obtain initial clustering results and corresponding initial hierarchical labels. Based on the initial clustering results, the silhouette coefficient of each sample is calculated and the average silhouette coefficient of all samples is calculated. The average silhouette coefficient is used as a quantitative evaluation index of clustering quality.

[0057] The grid search mechanism iterates through the candidate combinations of parameters eps and min_samples in the preset parameter space. For each group of candidate parameters, clustering and silhouette coefficient calculation are repeatedly performed to obtain the average silhouette coefficient corresponding to each group of candidate parameters. The parameter combination corresponding to the one with the largest average silhouette coefficient is then determined as the optimal parameter. DBSCAN clustering is then re-executed with the optimal parameter to generate optimized hierarchical labels. The number of optimized clusters and the number of noise points are recorded as auxiliary output information.

[0058] Based on the clustering results under the optimal parameter combination, the silhouette coefficient is used to identify the cluster boundary samples. Specifically, the cluster boundary samples are the samples whose silhouette coefficient is lower than a preset threshold. The index of the boundary samples is transmitted to the noise suppression module as a second-level optional noise filtering basis, in addition to the noise samples with a level label of -1, so as to realize the adaptive closed-loop optimization of the clustering parameters.

[0059] like Figure 2 As shown, preferably, in step S3, the application of the DBSCAN algorithm to perform density clustering on the semantic similarity matrix to generate hierarchical labels, calculate the global semantic importance score of each term in the semantic vectorized representation, and obtain the semantic relevance weight vector β, further includes: Step S31: Apply the DBSCAN algorithm to perform density clustering on the semantic similarity matrix, and perform hierarchical division of the semantic vectorized representation based on semantic similarity to form different semantic clusters and generate corresponding hierarchical labels.

[0060] Step S32: The hierarchical label is sent to a noise suppression module, which is used to perform non-linear attenuation of word frequency according to semantic weight to achieve soft noise reduction.

[0061] The global semantic importance score includes: semantic similarity to all other terms and semantic relevance weight vector obtained after normalization.

[0062] The optimization solution in step S3 mainly involves semantic weight calculation, and the corresponding module is the semantic weight calculation module. The function of the semantic weight calculation module is to calculate the semantic importance weight of each term in the context. The input to the semantic weight calculation module is: a sequence of term vectors V = [ , ..., ].

[0063] The specific processing method of the optimization solution in step S3 is as follows: For each term w k Calculate the sum of its semantic similarity to all other terms:

[0064] Normalization yields semantic relevance weights:

[0065] The module's signal output is: weight vector β = [ , ..., Simultaneously transmitted to the noise suppression module and the attention enhancement module.

[0066] The optimization scheme in step S3 proposes a method for explicitly calculating the semantic relevance between terms, and uses it as a cross-module control signal, with semantic relevance weights. It does not rely on internal model attention and has external interpretability. Semantic relevance weights. It can be used for the regulation of multiple subsequent subsystems (such as noise reduction intensity and attention bias) - the optimization technology provided in step S3 has a clear implementation method and clear formula, and can be directly implemented by programming.

[0067] As a further improvement to step S3, if a "weight visualization module" is added to the system, the method flow can be extended: output semantic relevance weights. The front-end interface supports manual intervention or annotation feedback.

[0068] like Figure 3 As shown, preferably, in step S4, noise suppression is performed on the hierarchical labels and the semantic relevance weight vector, and attention enhancement is performed on the semantic relevance weight vector based on the Transformer architecture to output the denoised text X. clean Further including: Step S41: The hierarchical tag is sent to a noise suppression module, which performs noise suppression using an exponential decay function. If the current word frequency... If the frequency exceeds a preset threshold, it is marked as noise and deleted. The attenuated word frequency or Boolean mask is then sent to an output module. The noise suppression module filters noise samples through hierarchical labels and filters noise words through semantic weights.

[0069] Step S42: Based on the Transformer architecture, a semantic prior matrix B is introduced to enhance the attention of the semantic relevance weight vector, and the attention-enhanced denoised text is output.

[0070] The optimization solutions provided in steps S41 to S42 work together with the noise suppression module and the attention enhancement module to perform exponentially decaying word frequency threshold filtering on the noise samples identified by the hierarchical labels, and introduce a semantic prior matrix into the semantic relevance weight vector to guide the attention distribution of the Transformer architecture, thereby achieving dual optimization of text denoising and semantic enhancement.

[0071] Step S41 uses an exponential decay function to process word frequencies, allowing high-frequency noise words to decay more significantly while retaining core words. Combined with a dynamic threshold judgment mechanism, this more smoothly distinguishes between noise words and effective words, avoiding information loss or noise residue caused by a single threshold. Step S42 constructs a semantic prior matrix B, integrating the inter-word association information in the semantic relevance weight vector into the Transformer's attention calculation process. This enables the model to prioritize other words with strong semantic relevance when processing each word, compensating for the shortcomings of the original attention mechanism, which relies entirely on data-driven processing.

[0072] It can be seen that through the coordinated action of steps S41 and S42, the filtered word frequency information and the semantic prior matrix complement each other. First, non-semantic noise interference is eliminated from the statistical level, and then the effective association between words is strengthened from the semantic level. This enables the attention enhancement module to learn a more accurate attention distribution in a specific semantic environment and avoids the interference of noise words on attention weights.

[0073] The functional modules involved in steps S1 to S4 are described below: Noise suppression module: This module performs non-linear attenuation of word frequencies based on semantic weights to achieve soft noise reduction. The input to the noise suppression module includes: the original word frequencies. Semantic weight Control parameter γ>0 (can be set globally or dynamically adjusted by level).

[0074] The noise suppression module's processing methods include: Applying the exponential decay function:

[0075] like If 'k < τ (threshold), then it is marked as noise and ready for deletion; Signal output of the noise suppression module: attenuated word frequency or Boolean mask m k ∈{0,1} is passed to the output module.

[0076] The improvement of the noise suppression module in the technical solution lies in: A novel dynamic noise suppression function is proposed to achieve "preservation of high-importance words and decay of low-importance words". Unlike traditional hard threshold deletion, the exponential decay function is continuous, differentiable, and trainable. The parameter γ of the exponential decay function can be adaptively adjusted according to the clustering level (e.g., γ=1.2 for advertising class and γ=0.8 for technology class).

[0077] It can be seen that the technical solution of the noise suppression module is complete and can be directly reproduced by those skilled in the art.

[0078] As a further improvement, if a "parameter learning module" is added to the system, the parameter γ of the exponential decay function can automatically learn the optimal value through a small amount of labeled data. The method process would require the addition of a training phase.

[0079] Attention Enhancement Module: Introduces a semantic prior matrix into the Transformer to increase the attention given to keyword items.

[0080] The input to the attention enhancement module is a matrix of Query, Key, and Value, Q, K, and V. Semantic weight vector ; Handling method: Constructing a semantic prior matrix :

[0081] Modify the attention calculation method:

[0082] Where λ controls the strength of prior influence; Signal output of the attention enhancement module: The enhanced attention output is transmitted to the output module.

[0083] The improvement of the attention enhancement module in this application embodiment is that: for the first time, external semantic weights are injected into the Transformer attention mechanism, which solves the problem that traditional attention may ignore low-frequency but high-semantic words; the B matrix can be frozen or fine-tuned, and it is compatible with a variety of deployment scenarios.

[0084] The formula for the attention enhancement module is well-defined and can be directly integrated into frameworks such as HuggingFace.

[0085] As a further improvement to the attention enhancement module, if a "multi-head prior attention" structure is added to the system, it needs to be expanded to introduce B independently for each head. h The method flow needs to support parallel computing.

[0086] Output module: Generates the final denoised text. The input signal of the output module comes from the mask of the noise suppression module and the context representation of the attention enhancement module.

[0087] The output module processes text as follows: Noisy words are removed based on a mask; high-weight words are retained, and the sentence is reconstructed. Supported output formats include: plain text, weighted text, and JSON. The final output is the denoised text X. clean .

[0088] The connection relationships and signal types of the internal components of the system in this application embodiment

[0089] Analysis of the impact of system structure changes on methodology:

[0090] In summary, this application's embodiments, by constructing a closed-loop control system, realize a closed-loop system from hierarchical clustering, semantic weight adjustment, graded noise reduction to attention enhancement, achieving refined control of text noise. It is the first to use semantic weights as cross-module control signals and proposes a dynamic noise suppression function: f' k = f k e -γ(1-\βk) Furthermore, a semantic prior matrix is ​​introduced into the Transformer architecture, realizing the linkage design of structure and method, which can support the flexible expansion of the system in various contexts.

[0091] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of this application.

[0092] like Figure 4 The deep learning-based text data denoising system 100 shown is used to implement the deep learning-based text data denoising method described in any specific embodiment of this application, including: The semantic vectorization representation module 101 acquires the original text sequence, encodes the original text sequence using a pre-trained language model, and converts the original text sequence into a semantic vectorization representation.

[0093] The semantic similarity matrix construction module 102 performs similarity measurement calculations on the vectors in the semantic vectorization representation, and constructs the corresponding semantic similarity matrix based on the results of the similarity measurement calculation.

[0094] The semantic relevance weight vector generation module 103 applies the DBSCAN algorithm to perform density clustering on the semantic similarity matrix, generates hierarchical labels, calculates the global semantic importance score of each term in the semantic vectorized representation, and obtains the semantic relevance weight vector.

[0095] The attention-enhanced denoising text output module 104 performs noise suppression on the hierarchical labels and the semantic relevance weight vector, and enhances the attention of the semantic relevance weight vector based on the Transformer architecture to output denoised text X. clean .

[0096] The implementation methods of the system described above are merely illustrative. For example, the various functional modules, units, or subsystems in the system may or may not be physically separate, or they may or may not be physical units. That is, they may be located in the same place or distributed across multiple different systems and their subsystems or modules. Those skilled in the art can select some or all of the functional modules, units, or subsystems to achieve the purpose of the embodiments of this application according to actual needs. Those skilled in the art can understand and implement the above-described situations without creative effort.

[0097] like Figure 5 As shown, this application embodiment, in addition to providing a text data denoising method and system based on deep learning, also provides corresponding electronic devices and computer program products: An electronic device includes: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory stores a computer program, which, when executed by the processor, causes the processor to perform steps of a deep learning-based text data denoising method.

[0098] A computer program product includes a computer program that, when executed by a processor, implements a deep learning-based text data noise reduction method.

[0099] Explanation of reference numerals in the attached drawings: Electronic device 500, External device 514, Processor 516, Bus 518, Network adapter 520, I / O interface 522, Display device 524, Memory 528, RAM 530, Cache 532, Storage system 534, Program / Utility 540, Program module 542.

[0100] Figure 5The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Typically, this electronic device could be a device within an electronic product that operates based on the multimodal data fusion method incorporating generative AI as described in the above embodiments. The electronic device 500 is represented in the form of a general-purpose computing device. Components of the electronic device 500 may include, but are not limited to: one or more processing units or processors 516, a memory 528, and a bus 518 connecting different system components (including the memory 528 and the processor 516). The bus 518 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus. The electronic device 500 typically includes various computer system readable media. These media can be any available media accessible to the electronic device 500, including volatile and non-volatile media, removable and non-removable media. Memory 528 may include computer system readable media in the form of volatile memory, such as RAM 530 and / or cache 532. Electronic device 500 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 534 may be used to read and write non-removable, non-volatile magnetic media (not shown in the figure, commonly referred to as "hard disk drives"). Although not shown in the figure, storage system 534 may provide disk drives for reading and writing to removable non-volatile disks (e.g., floppy disks, portable hard disks, hot-swappable storage media) and optical disk drives for reading and writing to removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media). In these cases, each drive may be connected to bus 518 via one or more data media interfaces. Memory 528 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various specific embodiments of the present application. A program / utility 540 having a set (at least one) of program modules 542 may be stored, for example, in memory 528. Such program modules 542 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 542 typically perform the functions and / or methods described in the embodiments of this application.Electronic device 500 can also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display device 524, etc.), and with one or more devices that enable a user to interact with electronic device 500, and / or with any device that enables electronic device 500 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through I / O interface 522. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 520. Network adapter 520 communicates with other modules of electronic device 500 via bus 518. It should be understood that, although not shown in the figures, those skilled in the art can use other hardware and / or software modules in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems. Processor 516 executes various functional applications and data processing by running programs stored in memory 528, such as implementing the methods provided in any one or more embodiments of this application.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the specification of the embodiments of this application.

[0102] In the description of the embodiments of this application, the reference to terms such as "an embodiment," "example," "specific example," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] All features disclosed in the embodiments of this application, or all steps in the disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps. Any feature disclosed in the specification of the embodiments of this application, unless specifically stated otherwise, may be replaced by other equivalent or similar alternative features. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features. Throughout the specification, the same reference numerals indicate the same elements.

[0104] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification of embodiments (including the corresponding claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification of embodiments (including the corresponding claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of this application, and are not intended to limit them. Although the embodiments of this application have been described in detail with reference to the foregoing specific embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing specific embodiments, or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions provided by the specific embodiments of this application.

Claims

1. A text data denoising method based on deep learning, characterized in that, include: Obtain the original text sequence, encode the original text sequence using a pre-trained language model, and convert the original text sequence into a semantic vectorized representation; A similarity metric is calculated for the vectors in the semantic vectorized representation, and a corresponding semantic similarity matrix is ​​constructed based on the result of the similarity metric calculation. The DBSCAN algorithm is applied to perform density clustering on the semantic similarity matrix to generate hierarchical labels. The global semantic importance score of each term in the semantic vectorized representation is calculated to obtain the semantic relevance weight vector. Noise suppression is applied to the hierarchical labels and the semantic relevance weight vector, and attention enhancement is performed on the semantic relevance weight vector based on the Transformer architecture to output denoised text.

2. The text data denoising method based on deep learning according to claim 1, characterized in that, The step of obtaining the original text sequence, encoding the original text sequence using a pre-trained language model, and converting the original text sequence into a semantic vectorized representation further includes: The original text sequence is mapped to a semantic vectorized representation in a low-dimensional continuous vector space by a pre-trained language model. The semantic vectorized representation is used to characterize the overall semantics of the text. The semantic vectorized representation includes sentence-level semantic vector representation and word-level word-level embedding vector sequence. The word-level embedding vector sequence is used to characterize the context-related semantics of terms in the text.

3. The text data denoising method based on deep learning according to claim 1, characterized in that, The step of performing a similarity measurement calculation on the vectors in the semantic vectorized representation, and constructing a corresponding semantic similarity matrix based on the result of the similarity measurement calculation, further includes: The similarity metric is calculated as cosine similarity, which calculates the cosine similarity between any two vectors, and constructs the corresponding semantic similarity matrix based on the result of the cosine similarity calculation.

4. The text data denoising method based on deep learning according to claim 1, characterized in that, The application of the DBSCAN algorithm to perform density clustering on the semantic similarity matrix generates hierarchical labels, calculates the global semantic importance score for each term in the semantic vectorized representation, and obtains a semantic relevance weight vector, further including: The DBSCAN algorithm is applied to perform density clustering on the semantic similarity matrix, and the semantic vectorized representation is hierarchically divided based on semantic similarity to form different semantic clusters and generate corresponding hierarchical labels. The hierarchical tags are sent to a noise suppression module, which is used to non-linearly attenuate the word frequency according to the semantic weight.

5. The text data denoising method based on deep learning according to claim 4, characterized in that, The global semantic importance score includes: semantic similarity to all other terms and semantic relevance weight vector obtained after normalization.

6. The text data denoising method based on deep learning according to claim 1, characterized in that, The step of performing noise suppression on the hierarchical labels and the semantic relevance weight vector, and performing attention enhancement on the semantic relevance weight vector based on the Transformer architecture to output denoised text, further includes: The hierarchical label is sent to a noise suppression module, which performs noise suppression using an exponential decay function. If the current word frequency is higher than a preset threshold, it is marked as noise and deleted. The noise-attenuated word frequency or Boolean mask is then sent to an output module. Based on the Transformer architecture, a semantic prior matrix is ​​introduced to enhance the attention of the semantic relevance weight vector, and the denoised text after attention enhancement is output.

7. The text data denoising method based on deep learning according to claim 6, characterized in that, The noise suppression of the hierarchical labels and the semantic relevance weight vector further includes: the noise suppression module filters noise samples through hierarchical labels and filters noise words through semantic weights.

8. A deep learning-based text data denoising system, used to implement the deep learning-based text data denoising method according to any one of claims 1 to 7, characterized in that, include: The semantic vectorization representation module acquires the original text sequence, encodes the original text sequence using a pre-trained language model, and converts the original text sequence into a semantic vectorization representation. The semantic similarity matrix construction module performs similarity measurement calculations on the vectors in the semantic vectorization representation, and constructs the corresponding semantic similarity matrix based on the results of the similarity measurement calculation. The semantic relevance weight vector generation module applies the DBSCAN algorithm to perform density clustering on the semantic similarity matrix, generates hierarchical labels, calculates the global semantic importance score of each term in the semantic vectorized representation, and obtains the semantic relevance weight vector. The attention-enhanced denoising text output module suppresses noise on the hierarchical labels and the semantic relevance weight vector, and enhances the attention of the semantic relevance weight vector based on the Transformer architecture to output denoised text.

9. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; the memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images