Intelligent writing system and method based on multilayer feature fusion

Abstract

The invention relates to the technical field of text processing, and discloses an intelligent writing system and method based on multilayer feature fusion, and the system comprises a data collection module which is configured to collect text data and clean the text data; performing preprocessing operations of word segmentation, part-of-speech tagging and named entity recognition on the text data; the multi-layer feature extraction module is configured to judge the category of the text data and perform feature extraction on the text data according to the category, and the category of the text data comprises a word level and a sentence level; the feature fusion module is configured to fuse the features of the word-level text data and the sentence-level text data based on a weighted summation mode; and the compiling module is configured to compile the generated text based on the fused features. According to the method, multi-level feature extraction is carried out on the word-level and sentence-level text data, and effective feature fusion is carried out based on a weighted summation mode, so that the quality of the generated text is remarkably improved, and more accurate and smooth intelligent compiling service is provided for a user.

Description

Technical Field

[0001] The present invention relates to the field of text processing technology, and in particular to an intelligent writing system and method based on multi-layer feature fusion. Background Art

[0002] In the field of natural language processing, intelligent writing systems have broad application prospects, such as automatic news writing, copywriting, intelligent question answering, etc. However, the current intelligent writing technology faces the significant problem of poor quality of generated text, which largely limits its further promotion and application.

[0003] At present, most intelligent writing systems tend to focus only on a single level of information when extracting text features. Some systems focus on word-level features. For example, simply analyzing the semantics and frequency of words can grasp the basic semantics of the text to a certain extent, but it is difficult to take into account the overall structure and logical coherence of the sentence; while other systems focus on sentence-level features, obtaining the semantic representation of sentences through methods such as recurrent neural networks, but may ignore detailed information at the word level. This single-level feature extraction method makes it impossible for the system to fully and deeply understand the connotation of the text, which in turn leads to obvious defects in the generated text in terms of semantic accuracy, logical rationality, and fluency of expression, making it difficult to meet users' needs for high-quality text generation. Summary of the invention

[0004] In view of this, the present invention proposes an intelligent writing system and method based on multi-layer feature fusion, aiming to solve the problem of text generation quality caused by single-level feature extraction in current technology.

[0005] On the one hand, the present invention proposes an intelligent writing system based on multi-layer feature fusion, including: a data acquisition module, configured to collect text data and clean the text data; and perform preprocessing operations such as word segmentation, part-of-speech tagging, and named entity recognition on the text.

[0006] The multi-layer feature extraction module is configured to determine the category of text data and perform feature extraction on the text data according to the category. The categories of text data include: word level and sentence level.

[0007] The feature fusion module is configured to fuse the features of word-level and sentence-level text data based on a weighted summation method.

[0008] The writing module is configured to write and generate text based on the fused features.

[0009] Furthermore, the cleaning includes: detecting the encoding format of the text data and converting it into a unified encoding format, then using Python's chardet library to detect the encoding, calculating the encoding confidence of the text data, and deleting all text data with the encoding confidence lower than 0.2.

[0010] Furthermore, when determining the category of text data, it includes: constructing a grammatical connective word library, traversing the text data, obtaining the text length of the text data, and presetting a length threshold.

[0011] When the text length is less than or equal to the length threshold, the text data is determined to be at the word level; when the text length is greater than the length threshold, the text data is determined to be at the sentence level.

[0012] Furthermore, when feature extraction is performed according to the text data category, it includes: when the text data is determined to be word level, each text data is mapped into a low-dimensional vector using a word vector model, the semantic information of the word is captured, and the word frequency and inverse document frequency are extracted.

[0013] When the text data is determined to be at the sentence level, the text data is encoded based on a recurrent neural network to obtain the semantic representation of the text data, and to extract the length and syntactic structure of the text data.

[0014] Furthermore, it also includes: before the features of the text data are fused based on the weighted summation method, the features of the word-level and sentence-level text data are unified.

[0015] When the text data is at the word level, count the maximum and minimum values ​​of all low-dimensional vectors in each dimension, and then use the normalization formula to scale the value of each dimension to the range of 0 to 1; count the maximum and minimum values ​​of word frequency and inverse document frequency respectively, and then use the normalization formula to process.

[0016] Furthermore, it also includes: when the text data is at the sentence level, calculating the mean and standard deviation of the vector of semantic representation in each dimension, using the normalization formula to convert the value of each dimension into a standard normal distribution with a mean of 0 and a standard deviation of 1, and scaling the maximum and minimum values ​​of the vector based on the semantic representation to the interval of 0 to 1 through the normalization formula; counting the length of the text data at the sentence level, calculating the mean and standard deviation of all lengths, and processing through the normalization formula; counting the ratio of the total number of similar syntactic structures to the total number of all syntactic structures in each sentence-level text data, recorded as a syntactic structure quantitative index, calculating the maximum and minimum values ​​of each category in all sentence-level text data, and scaling to the interval of 0 to 1 through the normalization formula.

[0017] Furthermore, when the features of the text data of each category are fused based on the weighted summation method, it includes: the low-dimensional vectors corresponding to all word-level text data form a matrix W, where each row represents the low-dimensional vector of a word, each column represents a dimension, and for the i-th dimension, the standard deviation Var of all values ​​on the i-th dimension is calculated i .

[0018] Get the standard deviation of all word frequencies Var t The standard deviation of all inverse document frequencies Var f .

[0019] The weight of the word-level text data is calculated based on the standard deviation of all values ​​in the i-th dimension, the standard deviation of all word frequencies, and the standard deviation of all inverse document frequencies.

[0020] Furthermore, when the features of the text data of each category are fused based on the weighted summation method, it includes: the semantic representation vectors of all sentence-level text data form a matrix M, where each row represents the semantic representation vector of a word, each column represents a dimension, and for the jth dimension, the standard deviation Var of all values ​​on the jth dimension is calculated j .

[0021] Get the standard deviation of the length of sentence-level text data Var s ; Calculate the standard deviation Var of the syntactic structure of the sentence-level text data e ; The standard deviation of the syntactic structure of sentence-level text data satisfies the following relationship: ; Among them, m is the number of sentence-level text data, k represents the syntactic structure type, is the syntactic structure quantitative index of the k-th syntactic structure in the n-th text data, It is the syntactic structure quantitative index of the kth syntactic structure.

[0022] Furthermore, when the features of the word-level and sentence-level text data are fused based on a weighted summation method, the weight of the word-level text data is: ; The weight of sentence-level text data is: ; The fused feature vector is: ; in, is the weight of word-level text data, is the weight of sentence-level text data, a, b, c are the low-dimensional vector weight coefficient, word frequency weight coefficient and inverse document frequency weight coefficient respectively, a+b+c=1, A, B, C are the syntactic structure weight coefficient, semantic weight coefficient and length weight coefficient respectively, T is the word-level feature vector, G is the sentence-level feature vector, F is the fused feature vector, I is the number of dimensions of word-level text data, and J is the number of dimensions of sentence-level text data.

[0023] Compared with the prior art, the beneficial effect of the present invention is that by detecting the encoding format of text data and converting it into a unified format, using Python's chardet library to detect encoding confidence, and deleting low-confidence data, the quality and consistency of the data are effectively improved. This makes the subsequent feature extraction and model training based on a more reliable data foundation, reduces errors and deviations caused by data format problems and noisy data, and improves the stability and accuracy of the entire system.

[0024] Preprocessing operations such as word segmentation, part-of-speech tagging, and named entity recognition on text lay the foundation for a deep understanding of text semantics and structure. These preprocessing steps allow text data to enter the subsequent process in a more structured and easier to process form, helping to improve the efficiency and accuracy of feature extraction.

[0025] The system distinguishes between word-level and sentence-level text data categories, and performs targeted feature extraction for each. Word-level feature extraction includes word vectors, word frequency, and inverse document frequency, which can capture the semantic information of words, their importance in the text, and their uniqueness in the document collection; sentence-level feature extraction obtains semantic representation based on recurrent neural networks, while extracting text length and syntactic structure to fully reflect the semantics, structure, and length information of sentences. This multi-level feature extraction method can provide a more comprehensive and in-depth understanding of text, providing a rich source of information for generating high-quality text.

[0026] The word-level and sentence-level features are fused based on the weighted summation method, which fully considers the importance of features at different levels. The weights are determined by calculating the standard deviation of each feature, making the weight distribution more scientific and reasonable, and being able to dynamically adjust the contribution of word-level and sentence-level features in the fusion according to the characteristics of the data. This fusion method can give full play to the synergy of multiple layers of features, making the generated text perform better in terms of semantic coherence, logical rationality, and vocabulary richness.

[0027] The word-level features are normalized to unify the numerical ranges of low-dimensional vectors, word frequencies, and inverse document frequencies to the interval of 0 to 1, eliminating the differences in numerical ranges between different features, making them equally important in subsequent fusion calculations and improving the effect of feature fusion.

[0028] Unification of sentence-level features: A combination of standardization and normalization is used for sentence-level features, converting the semantic representation vector into a standard normal distribution and scaling it to the range of 0 to 1. Similar processing is also performed on the quantitative indicators of text length and syntactic structure. This unified processing method makes sentence-level features consistent in numerical distribution and range, facilitating effective fusion with word-level features, and improving the accuracy and stability of feature fusion.

[0029] The writing module writes and generates text based on the fused features. Since the fused features fully contain the key information at the word level and sentence level, and the weight distribution is reasonable, the generated text can better balance the semantic accuracy, logical coherence and fluency of expression. Whether in vocabulary selection, sentence construction or overall chapter structure, it can better conform to human language expression habits and meet users' needs for high-quality text.

[0030] On the other hand, the present invention proposes an intelligent writing method based on multi-layer feature fusion, which is applied to the above-mentioned intelligent writing system based on multi-layer feature fusion, including: S1: collecting text data and cleaning the text data; performing preprocessing operations of word segmentation, part-of-speech tagging, and named entity recognition on the text.

[0031] S2: Determine the category of the text data, and extract features from the text data according to the category. The categories of the text data include: word level and sentence level.

[0032] S3: The features of word-level and sentence-level text data are fused based on weighted summation.

[0033] S4: Write the generated text based on the fused features.

[0034] It is understandable that the above-mentioned intelligent writing system and method based on multi-layer feature fusion have the same beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0036] Figure 1 A functional framework diagram of an intelligent writing system based on multi-layer feature fusion provided in an embodiment of the present invention.

[0037] Figure 2 A flowchart of an intelligent writing method based on multi-layer feature fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The exemplary embodiments disclosed in the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] See also Figure 1 As shown, an embodiment of the present invention provides an intelligent writing system based on multi-layer feature fusion, including: a data acquisition module, configured to collect text data and clean the text data; and perform preprocessing operations such as word segmentation, part-of-speech tagging, and named entity recognition on the text.

[0040] The multi-layer feature extraction module is configured to determine the category of text data and extract features from the text data according to the category. The categories of text data include: word level and sentence level.

[0041] The feature fusion module is configured to fuse the features of word-level and sentence-level text data based on a weighted summation method.

[0042] The writing module is configured to write and generate text based on the fused features.

[0043] It should be noted that text data are collected from a variety of data sources, such as Internet articles, academic papers, books, etc.

[0044] The data collection module cleans the collected text data, detects the encoding format of the text data, and converts it to a unified encoding format to ensure data consistency. At the same time, it uses relevant tools (Python's chardet library) to detect encoding confidence and deletes all text data with encoding confidence below a certain threshold, thereby removing low-quality noise data and improving data quality.

[0045] Preprocessing operations include word segmentation, which is to divide continuous text into independent words or word blocks; part-of-speech tagging, which is to mark each word with its part of speech (such as noun, verb, adjective, etc.); named entity recognition, which is to identify entities with specific meanings in the text (such as names of people, places, and organizations). These preprocessing operations can convert the original text into a structured data form that is easier to process and analyze later.

[0046] Multi-layer feature extraction module: responsible for determining the category of text data and dividing text data into two types: word level and sentence level. Classification is achieved by building a lexicon of grammatical connectives, traversing the text data, obtaining the text length, and comparing it with a pre-set length threshold. Targeted feature extraction is performed according to the category of the text data. For word-level text data, the word vector model is used to map each text data into a low-dimensional vector to capture the semantic information of the word, and the word frequency and inverse document frequency are extracted to measure the importance and uniqueness of the word in the text. For sentence-level text data, the text data is encoded based on a recurrent neural network to obtain the semantic representation of the text data, and the length and syntactic structure of the text data are extracted, so as to fully understand the semantics, structure and length information of the sentence.

[0047] Feature fusion module: Fusion of word-level and sentence-level text data features. Using a weighted summation approach, the importance of word-level and sentence-level features is comprehensively considered, and by reasonably allocating weights, features at different levels are effectively combined to form a more comprehensive and representative feature set. In actual operation, weights are determined based on indicators such as the standard deviation of features, making the weight allocation more scientific and reasonable, and giving full play to the advantages of features at different levels.

[0048] As the final output module of the system, the writing module generates text through the natural language model based on the fused features output by the feature fusion module. Since the fused features contain rich information at the word level and sentence level, and the weight distribution is reasonable, the writing module can use these features to generate high-quality text. The generated text has good performance in semantic accuracy, logical coherence and fluency of expression, and can meet the user's needs for text generation in different scenarios.

[0049] In some embodiments of the present application, cleaning includes: detecting the encoding format of the text data and converting it into a unified encoding format, then using Python's chardet library to detect the encoding, calculating the encoding confidence of the text data, and deleting all text data with an encoding confidence lower than 0.2.

[0050] It should be noted that detecting the encoding format of text data and converting it to a unified encoding format can eliminate data confusion caused by different encodings. Since text data comes from a wide range of sources, there may be multiple encoding formats, such as UTF-8, GBK, etc. If unified conversion is not performed, garbled characters and character parsing errors may occur in the subsequent data processing process, affecting the system's understanding and processing of the text content. A unified encoding format can ensure the consistency and compatibility of data in the system, so that subsequent operations can proceed smoothly.

[0051] Using Python's chardet library to detect encoding and calculate encoding confidence is a further screening of data quality. The chardet library can detect and judge the encoding of text data and give the corresponding confidence. By setting the encoding confidence threshold (such as 0.2) and deleting all text data with encoding confidence lower than the threshold, data with inaccurate encoding detection, possible errors or noise can be effectively removed. These low-quality data may contain garbled characters, incorrect character representations, etc., which will interfere with subsequent feature extraction and model training. Deleting them will help improve the quality of the entire data set and provide a more reliable data foundation for subsequent processing.

[0052] During data processing and analysis, low-quality data may cause system anomalies or errors, affecting the stability and reliability of the system. Through the above cleaning operations, these potential problem data can be eliminated in advance, reducing the probability of errors in the system during operation and ensuring the normal operation of the system. At the same time, high-quality data also helps to improve the processing efficiency of the system, because the system does not need to spend additional resources to process erroneous or invalid data.

[0053] Accurate, high-quality data is the basis for effective feature extraction and model training. After cleaning, the text content of the data is more accurate and standardized, which enables the multi-layer feature extraction module to more accurately extract word-level and sentence-level features, improving the quality and reliability of the features. For the subsequent feature fusion and writing modules, based on high-quality data and features, it can generate texts that are more in line with expectations and of higher quality, improving the performance and effect of the entire intelligent writing system.

[0054] In some embodiments of the present application, when determining the category of text data, it includes: Construct a grammatical connective word library, traverse the text data, obtain the text length of the text data, and pre-set a length threshold; when the text length is less than or equal to the length threshold, judge the text data as word level; when the text length is greater than the length threshold, judge the text data as sentence level.

[0055] It should be noted that we need to build a vocabulary of grammatical connectives: grammatical connectives (such as "because", "so", "and", "but", etc.) are vocabulary that connects different parts of a sentence and expresses logical relationships. This provides a basis for subsequent analysis of the structure and logic of the text, and also provides a reference for determining the category of text data.

[0056] The length threshold is a key criterion, which is a value determined in advance based on actual needs and experience. This threshold is used to distinguish whether the text data is at the word level or sentence level. Different application scenarios may set different length thresholds. For example, it may be set to 10 characters in some scenarios, but it may be different in other scenarios.

[0057] Traversing text data means processing all text data collected by the system one by one. During the traversal process, the text length of each text data is obtained, that is, the number of characters contained in the text is counted. This step is the basis for determining the category of text data. Only by obtaining the text length can it be compared with the pre-set length threshold.

[0058] Word-level judgment: When the length of the acquired text is less than or equal to the preset length threshold, the system will judge the text data as word-level. This means that the text data may be just a single word, a combination of a small number of words, or some short text fragments that are not enough to form a complete sentence. At this time, the system will perform subsequent feature extraction and other operations according to the word-level processing method, such as using the word vector model to map it into a low-dimensional vector, extracting word frequency and inverse document frequency, etc.

[0059] When the text length is greater than the preset length threshold, the system will determine that the text data is at the sentence level. This indicates that the text data has a certain length and is more likely to be a complete sentence or a text paragraph containing multiple sentences. For sentence-level text data, the system will encode it using methods such as recurrent neural networks to obtain semantic representations and extract features such as the length and syntactic structure of the text data.

[0060] In some embodiments of the present application, when feature extraction is performed according to the text data category, it includes: when the text data is determined to be at the word level, each text data is mapped into a low-dimensional vector using a word vector model, the semantic information of the word is captured, and the word frequency and inverse document frequency are extracted; when the text data is determined to be at the sentence level, the text data is encoded based on a recurrent neural network to obtain the semantic representation of the text data, and the length and syntactic structure of the text data are extracted.

[0061] It should be noted that the word-level text data feature extraction is as follows: select a pre-trained word vector model, such as Word2Vec (Skip-gram or CBOW architecture), GloVe or FastText, and input the word-level text data into the model to obtain a low-dimensional vector.

[0062] Capturing semantic information: Using the representation of word vectors in the vector space, the semantic relationship is measured by calculating the cosine similarity between vectors.

[0063] Extract word frequency: Traverse the text data set and count the number of times each word appears as the word frequency. You can choose to normalize the word frequency by dividing it by the total number of words in the document.

[0064] Extract inverse document frequency: Count the total number of documents in the document collection. For each word, count the number of documents containing the word, and use the formula to calculate the inverse document frequency.

[0065] Sentence-level text data feature extraction: Based on recurrent neural network encoding, a recurrent neural network structure such as LSTM or GRU is selected. After the sentence-level text data is segmented, it is converted into a word vector sequence as input. A pre-trained model or a model trained on a large-scale corpus can be used to obtain the semantic representation vector of the sentence.

[0066] Obtain semantic representation and use the fixed-dimensional vector output by the recurrent neural network as the semantic representation of the sentence.

[0067] Extract the text length and count the number of words or characters in the sentence-level text data as the text length.

[0068] Extract syntactic structure, use natural language processing tool NLTK to perform syntactic analysis on sentences, generate syntactic structure tree, and extract syntactic features such as part of speech tagging and phrase structure from it.

[0069] In some embodiments of the present application, it also includes: before the features of the text data are fused based on the weighted summation method, the features of the word-level and sentence-level text data are unified; when the text data is at the word level, the maximum and minimum values ​​of all low-dimensional vectors in each dimension are counted, and then the normalization formula is used to scale the value of each dimension to the interval of 0 to 1; the maximum and minimum values ​​of the word frequency and the inverse document frequency are counted respectively, and then processed using the normalization formula.

[0070] It should be noted that for word-level text data, a series of low-dimensional vectors will be obtained after being mapped by the word vector model. These low-dimensional vectors can be regarded as a set, in which each low-dimensional vector has a fixed dimension. In order to normalize these low-dimensional vectors, we need to count the maximum and minimum values ​​for each dimension. Specifically, for the first dimension of all low-dimensional vectors, find the element with the largest value as the maximum value of the dimension, and the element with the smallest value as the minimum value of the dimension; then repeat this operation for the second dimension, and so on, until the maximum and minimum values ​​of all dimensions are counted.

[0071] Use the normalization formula for scaling: After obtaining the maximum and minimum values ​​of each dimension, you can use the linear normalization (also called Min-Max normalization) formula to process the low-dimensional vector. For any element in the low-dimensional vector, that is, the value of the low-dimensional vector of a word in a certain dimension, subtract the minimum value of the dimension from this value, and then divide it by the difference between the maximum and minimum values ​​of the dimension. The result is the normalized value of the element. All elements in each dimension of all low-dimensional vectors are processed, and finally a normalized low-dimensional vector set is obtained.

[0072] Count the maximum and minimum values: First, collect the frequency of each word in all word-level text data to form a frequency list. In this frequency list, find the word with the largest value as the maximum value and the word with the smallest value as the minimum value.

[0073] Use the normalization formula for scaling: After obtaining the maximum and minimum values ​​of the word frequency, use the linear normalization formula to process each word frequency in the word frequency list. For any word frequency in the list, subtract the minimum value of the word frequency from it, and then divide it by the difference between the maximum and minimum values ​​of the word frequency. The result is the normalized value of the word frequency. By performing this process on all word frequencies in the word frequency list, a normalized word frequency list is obtained.

[0074] The normalization of inverse document frequency is similar to word frequency normalization. First, collect the inverse document frequency of each word in all word-level text data to form an inverse document frequency list. Find the maximum and minimum values ​​in this list.

[0075] Use the normalization formula for scaling: After obtaining the maximum and minimum values ​​of the inverse document frequency, use the linear normalization formula to process each inverse document frequency in the inverse document frequency list. Subtract the minimum value of the inverse document frequency from each inverse document frequency in the list, and then divide it by the difference between the maximum and minimum values ​​of the inverse document frequency to obtain the normalized inverse document frequency. Perform this operation on all elements in the inverse document frequency list to complete the normalization of the inverse document frequency and obtain the normalized inverse document frequency list.

[0076] Through the above normalization processing of low-dimensional vectors, word frequencies and inverse document frequencies, these features of word-level text data have the same scale range before weighted summation feature fusion, thus preparing for subsequent feature fusion operations.

[0077] In some embodiments of the present application, it also includes: when the text data is at the sentence level, calculating the mean and standard deviation of the vector of semantic representation in each dimension, using the normalization formula to convert the value of each dimension into a standard normal distribution with a mean of 0 and a standard deviation of 1, and scaling the maximum and minimum values ​​of the vector based on the semantic representation to the interval of 0 to 1 through the normalization formula; counting the length of the text data at the sentence level, calculating the mean and standard deviation of all lengths, and processing through the normalization formula; counting the ratio of the total number of similar syntactic structures to the total number of all syntactic structures in each sentence-level text data, recorded as a syntactic structure quantification index, calculating the maximum and minimum values ​​of each category in all sentence-level text data, and scaling to the interval of 0 to 1 through the normalization formula.

[0078] Specifically, the mean and standard deviation are calculated as follows: First, for all sentence-level text data, the semantic representation vector corresponding to each sentence has been obtained through methods such as recurrent neural networks. These vectors have the same dimension.

[0079] For each dimension of the semantic representation vector, calculate the mean and standard deviation of all vector elements in that dimension. For example, if the semantic representation vector is 300-dimensional, you need to calculate the mean and standard deviation of all vector elements in the first dimension, the mean and standard deviation in the second dimension, and so on, until you have calculated the mean and standard deviation of the 300th dimension.

[0080] Use the normalization formula , process the elements on each dimension, where is the original value of that dimension, is the mean of all elements in this dimension, is the standard deviation of all elements in that dimension. After this processing, the elements in each dimension will be converted to a standard normal distribution with a mean of 0 and a standard deviation of 1.

[0081] Normalization: Find the maximum and minimum values ​​of the standardized semantic representation vector in each dimension.

[0082] Use the normalization formula Process the elements in each dimension, where is the standardized value, is the minimum value in this dimension, is the maximum value in that dimension. After this process, the elements in each dimension will be scaled to the interval of 0 to 1.

[0083] Processing of sentence-level text data length, statistical text length: For each sentence-level text data, count the number of words (or characters, depending on the needs) it contains to obtain the text length of each sentence.

[0084] Calculate mean and standard deviation: Calculate the mean and standard deviation of the length of all sentence-level text data.

[0085] Standardization: The text length of each sentence is processed using the standardization formula. After this process, the sentence length will be converted to a standard normal distribution with a mean of 0 and a standard deviation of 1.

[0086] Processing of syntactic structure quantitative indicators, calculation of syntactic structure quantitative indicators: For each sentence-level text data, analyze the various syntactic structures contained therein (such as noun phrases, verb phrases, etc.), and count the ratio of the total number of similar syntactic structures to the total number of all syntactic structures, and record this ratio as the syntactic structure quantitative indicator of this type of syntactic structure.

[0087] Calculate the maximum and minimum values: For each type of syntactic structure, find the maximum and minimum values ​​of the quantitative indicators of this type of syntactic structure in all sentence-level text data.

[0088] Normalization processing: The quantitative index of each type of syntactic structure is processed using a normalization formula, and the quantitative index of each type of syntactic structure will be scaled to the range of 0 to 1.

[0089] In some embodiments of the present application, when the features of text data of each category are fused based on the weighted summation method, it includes: the low-dimensional vectors corresponding to all word-level text data form a matrix W, where each row represents a low-dimensional vector of a word, each column represents a dimension, and for the i-th dimension, the standard deviation Var of all values ​​on the i-th dimension is calculated. i .

[0090] Get the standard deviation of all word frequencies Var t The standard deviation of all inverse document frequencies Var f .

[0091] The weight of the word-level text data is calculated based on the standard deviation of all values ​​in the i-th dimension, the standard deviation of all word frequencies, and the standard deviation of all inverse document frequencies.

[0092] It should be noted that the standard deviation reflects the degree of discreteness of features and is a statistic that measures the degree of discreteness of data. For the low-dimensional vectors corresponding to word-level text data, calculating the standard deviation of all values ​​on the dimension can clearly understand the fluctuation of data on each dimension. If the standard deviation of a certain dimension is large, it means that the low-dimensional vector values ​​of different words on this dimension are significantly different, indicating that this dimension contains rich distinguishing information and is of great significance for distinguishing different words; conversely, a small standard deviation means that the dimension has little variation between different words and has low discrimination. Similarly, obtaining the standard deviation of all word frequencies and the standard deviation of all inverse document frequencies can measure the degree of discreteness between different words in word frequency and inverse document frequency, respectively.

[0093] Importance of quantified features: Calculating the weight of word-level text data based on these standard deviations can convert the discreteness of features into specific weight values. Features with a large degree of discreteness have relatively high weights, indicating that the feature has stronger representativeness and distinguishing ability when describing word-level text data, and should receive more attention in the feature fusion process; while features with a small degree of discreteness have relatively low weights to avoid excessive influence on the fusion results. This weight calculation method provides a scientific and objective basis for feature fusion.

[0094] Realize reasonable feature weighting: When performing feature fusion, different features may contribute differently to the final result. By calculating weights based on standard deviation, it is possible to perform reasonable weighted summation of features such as low-dimensional vectors, word frequency, and inverse document frequency. This enables the fused feature vector to more comprehensively and accurately reflect the essential characteristics of word-level text data, avoids the problem of some important features being ignored or some minor features being over-emphasized, and improves the quality and effectiveness of feature fusion.

[0095] Improve the complementarity of features: The low-dimensional vector, word frequency, and inverse document frequency of word-level text data each have unique information. The low-dimensional vector captures the semantic information of the word, the word frequency reflects the frequency of occurrence of the word in the text, and the inverse document frequency reflects the uniqueness of the word in the entire document collection. Through reasonable weight distribution, the complementary role of these features can be fully utilized, so that the fused feature vector contains richer and more comprehensive information, thereby better describing the word-level text data.

[0096] In some embodiments of the present application, when the features of text data of each category are fused based on a weighted summation method, it includes: the semantic representation vectors of all sentence-level text data form a matrix M, where each row represents the semantic representation vector of a word, each column represents a dimension, and for the jth dimension, the standard deviation Var of all values ​​on the jth dimension is calculated. j .

[0097] Get the standard deviation of the sentence-level text data length Var s ; Calculate the standard deviation Var of the syntactic structure of the sentence-level text data e .

[0098] The standard deviation of the syntactic structure of sentence-level text data satisfies the following relationship: ; Among them, m is the number of sentence-level text data, k represents the syntactic structure type, is the syntactic structure quantitative index of the k-th syntactic structure in the n-th text data, It is the syntactic structure quantitative index of the kth syntactic structure.

[0099] It is understandable that the fused features can more accurately reflect the essential characteristics of the sentence, and the model trained based on these features can better learn the patterns and rules of the sentence, thus showing higher accuracy in tasks such as text classification, sentiment analysis, and machine translation. For example, in the text classification task, the fused features that comprehensively consider semantics, length, and syntactic structure can more accurately identify the category of the text.

[0100] Enhance the generalization ability of the model: Texts in different fields and styles may have large differences in sentence-level features. By calculating the standard deviation and performing feature fusion based on it, the model can adaptively adjust the degree of attention to different features and better adapt to various text data. This allows the model to maintain good performance when facing new and unseen texts, improves the generalization ability of the model, and expands the scope of application of the model.

[0101] Specifically, the semantic representation vectors of all sentence-level text data are constructed into a matrix M and the standard deviation of each dimension is calculated, which can accurately present the discrete degree of semantic representation in each dimension. j If Var is large, it means that the semantic difference between different sentences is significant and it is the key dimension to distinguish the semantics of sentences. j If the dimension is small, it means that the dimension is relatively stable in each sentence. This helps us to gain a deeper understanding of the structure of the semantic representation vector and grasp the changing pattern of the sentence semantics.

[0102] Text length and syntactic structure features: Obtaining the standard deviation of sentence-level text data length and the standard deviation of syntactic structure can reflect the discreteness of data from the perspective of sentence size and grammatical structure. s It can reflect the variation in length of different sentences and the diversity of text styles; while Var e By comprehensively considering various syntactic structure types, the complexity and degree of difference in the grammatical structure of sentences are measured, providing a quantitative basis for understanding the organization of sentences.

[0103] In some embodiments of the present application, when the features of word-level and sentence-level text data are fused based on a weighted summation method, it includes: The weight of word-level text data is: ; The weight of sentence-level text data is: ; The fused feature vector is: ; in, is the weight of word-level text data, is the weight of sentence-level text data, a, b, c are the low-dimensional vector weight coefficient, word frequency weight coefficient and inverse document frequency weight coefficient respectively, a+b+c=1, A, B, C are the syntactic structure weight coefficient, semantic weight coefficient and length weight coefficient respectively, T is the word-level feature vector, G is the sentence-level feature vector, F is the fused feature vector, I is the number of dimensions of word-level text data, and J is the number of dimensions of sentence-level text data.

[0104] It should be noted that the calculation formulas for considering the degree of feature discreteness: word-level text data weights and sentence-level text data weights all involve parameters related to the standard deviation. The standard deviation reflects the degree of discreteness of the data. By incorporating the standard deviation into the weight calculation, the weight distribution of the feature can be more in line with the actual situation of the data. For example, when the standard deviation of the word-level feature is large, it means that the difference between the word-level features in different text data is more significant. At this time, the word-level text data will be given a relatively high weight to highlight its importance in the fusion feature; conversely, if the standard deviation of the sentence-level feature is large, the weight of the sentence-level text data will increase to ensure that the features that can better reflect the data differences occupy a larger proportion in the fusion.

[0105] It is understandable that the word-level feature vector contains the semantic information, word frequency, and inverse document frequency of the word, which can reflect the lexical characteristics of the text from a micro level; the sentence-level feature vector S covers the semantic representation, syntactic structure, and length of the sentence, which reflects the sentence structure and overall semantics of the text from a macro level. By fusing the two by weighted summation, the information of the word level and sentence level can be fully integrated, and the features of the text at different levels can be fully captured, so that the fused feature vector F contains richer and more comprehensive text information.

[0106] Word-level features and sentence-level features are complementary. Word-level features can provide accurate lexical semantics and frequency information, but lack the description of the overall structure and semantic coherence of the sentence; sentence-level features can make up for this deficiency and reflect the grammatical structure and overall semantics of the sentence. Through reasonable weighted fusion, the advantages of both can be fully utilized, the information complementarity can be achieved, and the semantics and structure of the text can be more accurately represented.

[0107] Based on the fused feature vector F, the text is written. Since F integrates important information at the word level and sentence level, and the weight distribution is reasonable, the writing model can better understand the semantics and structure of the text, thereby generating a better text in terms of semantic accuracy, logical coherence, and fluency of expression. For example, in intelligent writing assistance, the generated articles can use vocabulary more accurately, the connection between sentences is more natural, and the overall structure is more reasonable.

[0108] Enhanced adaptability and generalization: This multi-level feature fusion method enables the system to adapt to texts of different types and styles. Different texts may show different characteristics in word-level and sentence-level features. By dynamically adjusting the weights for fusion, the system can flexibly process according to the characteristics of specific texts, improving the adaptability and generalization of the system, so that it can achieve better results in various practical application scenarios.

[0109] See also Figure 2As shown, an embodiment of the present invention provides an intelligent writing method based on multi-layer feature fusion, including: S1: collecting text data and cleaning the text data; performing preprocessing operations such as word segmentation, part-of-speech tagging, and named entity recognition on the text.

[0110] S2: Determine the category of the text data and extract features from the text data according to the category. The categories of text data include: word level and sentence level.

[0111] S3: The features of word-level and sentence-level text data are fused based on weighted summation.

[0112] S4: Write the generated text based on the fused features.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An intelligent writing system based on multi-layer feature fusion, characterized in that: include: A data collection module is configured to collect text data and clean the text data; Perform preprocessing operations on text data such as word segmentation, part-of-speech tagging, and named entity recognition; The multi-layer feature extraction module is configured to determine the category of the text data and extract features from the text data according to the category, the categories of the text data including: word level and sentence level; A feature fusion module is configured to fuse features of word-level and sentence-level text data based on a weighted summation method; The writing module is configured to write and generate text based on the fused features.

2. The intelligent writing system based on multi-layer feature fusion according to claim 1 is characterized in that: The cleaning includes: detecting the encoding format of the text data and converting it into a unified encoding format, then using Python's chardet library to detect the encoding, calculating the encoding confidence of the text data, and deleting all text data with the encoding confidence lower than 0.

2.

3. The intelligent writing system based on multi-layer feature fusion according to claim 1 is characterized in that: When judging the category of text data, it includes: Build a grammatical connective word library, traverse the text data, obtain the text length of the text data, and pre-set a length threshold; When the text length is less than or equal to the length threshold, the text data is determined to be at the word level; when the text length is greater than the length threshold, the text data is determined to be at the sentence level.

4. The intelligent writing system based on multi-layer feature fusion according to claim 1 is characterized in that: When feature extraction is performed based on text data categories, it includes: When the text data is determined to be word-level, the word vector model is used to map each text data into a low-dimensional vector, capture the semantic information of the word, and extract the word frequency and inverse document frequency; When the text data is determined to be at the sentence level, the text data is encoded based on a recurrent neural network to obtain the semantic representation of the text data, and to extract the length and syntactic structure of the text data.

5. The intelligent writing system based on multi-layer feature fusion according to claim 4 is characterized in that: Also includes: Before the features of the text data are fused based on the weighted summation method, the features of the word-level and sentence-level text data are unified; When the text data is at the word level, the maximum and minimum values ​​of all low-dimensional vectors in each dimension are counted, and then the normalization formula is used to scale the value of each dimension to the range of 0 to 1; The maximum and minimum values ​​of word frequency and inverse document frequency are counted respectively, and then processed using the normalization formula.

6. The intelligent writing system based on multi-layer feature fusion according to claim 5 is characterized in that: Also includes: When the text data is at the sentence level, the mean and standard deviation of the semantically represented vector in each dimension are calculated, and the value of each dimension is converted into a standard normal distribution with a mean of 0 and a standard deviation of 1 using a normalization formula. The maximum and minimum values ​​of the semantically represented vector are scaled to the range of 0 to 1 using a normalization formula. The length of the sentence-level text data is counted, and the mean and standard deviation of all lengths are calculated, and then processed using a normalization formula. The ratio of the total number of similar syntactic structures in each sentence-level text data to the total number of all syntactic structures is counted and recorded as the syntactic structure quantitative index. The maximum and minimum values ​​of each category in all sentence-level text data are calculated and scaled to the interval of 0 to 1 through the normalization formula.

7. The intelligent writing system based on multi-layer feature fusion according to claim 6 is characterized in that: When the features of text data of each category are fused based on weighted summation, it includes: The low-dimensional vectors corresponding to all word-level text data form a matrix W, where each row represents a low-dimensional vector of a word, each column represents a dimension, and for the i-th dimension, the standard deviation Var of all values ​​on the i-th dimension is calculated i ; Get the standard deviation of all word frequencies Var t The standard deviation of all inverse document frequencies Var f ; The weight of the word-level text data is calculated based on the standard deviation of all values ​​in the i-th dimension, the standard deviation of all word frequencies, and the standard deviation of all inverse document frequencies.

8. The intelligent writing system based on multi-layer feature fusion according to claim 1 is characterized in that: When the features of text data of each category are fused based on weighted summation, it includes: The semantic representation vectors of all sentence-level text data form a matrix M, where each row represents the semantic representation vector of a word, each column represents a dimension, and for the jth dimension, the standard deviation Var of all values ​​on the jth dimension is calculated j ; Get the standard deviation of the sentence-level text data length Var s ; Calculate the standard deviation Var of the syntactic structure of the sentence-level text data e ; The standard deviation of the syntactic structure of sentence-level text data satisfies the following relationship: ； Among them, m is the number of sentence-level text data, k represents the syntactic structure type, is the syntactic structure quantitative index of the k-th syntactic structure in the n-th text data, It is the syntactic structure quantitative index of the kth syntactic structure.

9. The intelligent writing system based on multi-layer feature fusion according to claim 1 is characterized in that: When the features of word-level and sentence-level text data are fused based on weighted summation, it includes: The weight of word-level text data is: ； The weight of sentence-level text data is: ； The fused feature vector is: ； in, is the weight of word-level text data, is the weight of sentence-level text data, a, b, c are the low-dimensional vector weight coefficient, word frequency weight coefficient and inverse document frequency weight coefficient respectively, a+b+c=1, A, B, C are the syntactic structure weight coefficient, semantic weight coefficient and length weight coefficient respectively, T is the word-level feature vector, G is the sentence-level feature vector, F is the fused feature vector, I is the number of dimensions of word-level text data, and J is the number of dimensions of sentence-level text data.

10. An intelligent writing method based on multi-layer feature fusion, applied to the intelligent writing system based on multi-layer feature fusion according to any one of claims 1 to 9, characterized in that: include: S1: Collect text data and clean it; perform preprocessing operations such as word segmentation, part-of-speech tagging, and named entity recognition on the text; S2: Determine the category of the text data, and extract features from the text data according to the category. The categories of the text data include: word level and sentence level; S3: The features of word-level and sentence-level text data are integrated based on weighted summation; S4: Write the generated text based on the fused features.

Images