A construction network generation method based on word collocation analysis

Abstract

The application discloses a construction network generation method based on word collocation analysis. The method first extracts the construction from the corpus by using a construction extraction tool based on a pre-trained model to obtain a corresponding construction library; secondly, the model fully utilizes the information of the access word of the construction through word collocation analysis, and obtains the linguistic characteristics of the construction according to the collocation strength of the access word; then, the algorithm of the pipeline hierarchical clustering is used to form the construction category and generate the first layer network; finally, the second layer and higher layer network are generated based on the construction category, and the maximum common item between the abstract degree of the construction and the construction category is taken as the feature. The method fully utilizes the construction related knowledge learned by the pre-trained model, can capture and construct the characteristics of the construction, simultaneously establishes the network relationship between the constructions, and generates the construction network.

Description

Technical Field

[0001] This invention relates to the fields of construction grammar and natural language processing, and in particular to a construction network generation method based on word collocation analysis. Background Technology

[0002] There are two main research approaches to grammatical theory in linguistics: one, represented by generative grammar, argues that "grammar is grammar, and usage is usage"; the other, represented by cognitive linguistics and functional grammar, argues that "usage precedes grammar," or more precisely, "usage is grammar." Construction grammar is an exploration within this academic context. Primarily grounded in cognitive linguistics, it adheres to a usage-based view of language experience and linguistic knowledge acquisition, constructing a linguistic theoretical model that contrasts with generative grammar. It is highly helpful in studying many problems in Natural Language Processing (NLP), such as sentiment analysis, semantic role labeling, and natural language reasoning tasks.

[0003] In construction grammar, a "construction" is a unit of knowledge, which manifests as a language unit in a language system; a construction network is formed by the connections between constructions. That is to say, a language system is a construction network, in which constructions are the initial units of the language system, language acquisition is mainly based on constructions, language generalization is constructional, and language is composed of and only of constructions.

[0004] Corresponding to the linguistic understanding of a "lexicon," the collection of various constructions within a language system can be called a "construction lexicon." A construction lexicon includes all conventional semantic-form matching entities within the language system, encompassing not only entity-based morphemes, words, and idioms, but also various schematic constructions (such as lexical formats, syntactic formats, and sentence structures) and semi-schematic constructions that express specific grammatical relationships. It also includes information about the connections and usage biases between constructions. The construction lexicon exhibits diverse connections between constructions, forming a complex and dynamic network system—this is the "construction network."

[0005] For example, the construction "pron (personal pronoun) + V (verb) + pron (personal pronoun) + Noun (noun)" has the central meaning of "X causes Y to receive Z", but depending on the verb part, it has the following polysemous forms:

[0006] 1. "X causes Y to receive Z" (central meaning)

[0007] Example: Joe gave Sally the ball.

[0008] 2. The fulfillment of the condition means "X causes Y to receive Z".

[0009] Example: Joe promised Bob a car.

[0010] 3. "X enables Y to receive Z"

[0011] Example: Joe permitted Bob an apple. (Joe allowed Bob to have an apple.)

[0012] 4. "X causes Y to not receive Z"

[0013] Example: Joe refused Bob a cookie.

[0014] 5. "X intentionally caused Y to receive Z."

[0015] Example: Joe baked Bob a cake.

[0016] Compared to the central meaning, each of the above extensions of meaning constitutes a construction with minimal difference. Although the specific verbs in the constructions differ, the relationship between the transferred central meaning and the central meaning that caused the movement, as well as the implication of these extensions, are the same. Therefore, they can be regarded as the inheritance of meaning. Construction 1 is a higher-level construction of constructions 2-5, located at the upper level of the network. Constructions 2-5 are at the same level, belonging to the same category.

[0017] The generation of construction networks is of great significance to existing natural language processing research: on the one hand, the generation of construction networks helps to clarify the relationships between constructions, forming a more usable construction library, and significantly improving knowledge extraction; on the other hand, the generation of construction networks can help the model select the construction with more important information from multiple constructions, and more effectively inject construction grammar knowledge into NLP pre-trained models.

[0018] Against this background, this invention proposes a construction network generation method based on collocation analysis. This method fully utilizes construction-related information learned by pre-trained models, capturing and constructing construction features while establishing network relationships between constructions, which is of great significance to natural language processing research. Summary of the Invention

[0019] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for generating construction networks based on term collocation analysis.

[0020] The objective of this invention is achieved through the following technical solution: a method for generating construction networks based on term collocation analysis, the method comprising the following steps:

[0021] (1) Using a construction extraction tool based on a pre-trained model, constructions contained in the corpus are extracted, and constructions with a frequency lower than ω% of the number of sentences in the corpus are filtered out, forming a construction library C = {c1, c2, c3, ..., c n}; where 3≤ω≤8;

[0022] (2) For each configuration c in the configuration library C i Lexical analysis is performed to construct construction c based on the collocation strength of the obtained terms. i The corresponding feature vector v i This yields the corresponding feature vector set V = {v1, v2, v3, ..., v n};

[0023] (3) Combine the corresponding eigenvectors v of the configuration i The configuration library C is clustered using a pipelined hierarchical clustering approach to obtain the first-level configuration classes. The first layer of the generative network

[0024] (4) Using first-level construction classes The included configurations Using the length of the longest common sub-item and the average configuration length as characteristics, we obtain the adjacency matrix S1 of the first-order configuration class;

[0025] (5) Based on the adjacency matrix S1, the first layer of the network is clustered using a pipelined hierarchical clustering method to obtain the second-level configuration classes, thus generating the second layer of the network.

[0026] (6) Repeat steps (4) and (5) to obtain the adjacency matrix S of the configuration class again. H-1 Then, using a pipelined hierarchical clustering approach, the (H-1)th layer of the network is clustered to obtain the Hth level configuration class, thus generating the Hth layer of the network. That is, to generate a structured network.

[0027] Furthermore, the configuration c i It contains several access terms U i .

[0028] Furthermore, in step (2), when performing construction term analysis, Fisher's exact test method is used, specifically: first, construction c is selected from the corpus. i and c i One of the access terms U iAnd perform statistical analysis to obtain the configuration c i The total frequency is a+c; secondly, the access term U is retrieved from the construction library. i The frequency of occurrence 'a'; then, the access term 'U' is retrieved from the corpus. i The total frequency is a + b; then, all constructions in the corpus are counted, that is, the total number of constructions n in the corpus; finally, the entry term U is calculated by subtraction. i The configuration c appears i The frequency of absence b = (a + ba), and the construction c i The term U appears but is an access term. i The frequency of non-occurrence c = (a + ca), configuration c i and access term U i The frequency d = (nabc) of none of them is calculated using Fisher's exact test formula, i.e.

[0029] The calculated p-value represents the term collocation strength; calculating the term collocation strength of all admitted terms yields the corresponding feature vector set V = {v1, v2, v3, ..., v n}

[0030] Furthermore, in step (3), before performing clustering, an initial adjacency matrix is ​​generated, specifically by treating each configuration as a cluster and then calculating the distance between clusters to form an initial adjacency matrix S0.

[0031] Furthermore, any value between 5 and 10 times the minimum value in the initial adjacent matrix S0 is used as the threshold for clustering in step (3).

[0032] Furthermore, in step (4), any value between 5 and 10 times the minimum value in the adjacency matrix S1 of the first-level configuration class is used as the clustering threshold in step (5).

[0033] Further, in step (3) or step (5), the pipeline hierarchical clustering method is as follows: during clustering, search for the value with the smallest distance in the adjacent matrix and its corresponding nodes v1 and v2, merge the two clusters with the smallest distance between v1 and v2, and repeat this process until the minimum value in the adjacent matrix is ​​greater than the threshold; after one hierarchical clustering is completed, select the classes with no less than t aggregated configurations in the cluster, re-perform hierarchical clustering of the remaining clusters, and repeat this process until the threshold is greater than the farthest distance in the adjacent matrix; where t≥2;

[0034] In step (3), the neighbor matrix searched during clustering is S0;

[0035] In step (5), the neighbor matrix searched during clustering is S1.

[0036] Therefore, it is easy to deduce that the neighbor matrix searched during each clustering is the neighbor matrix of the previous level configuration class, and any value between 5 and 10 times the minimum value in the neighbor matrix of the previous level configuration class is used as the threshold for the current clustering.

[0037] Furthermore, in step (4), for the construction class and construction class The distance score calculation process is as follows:

[0038] For construction classes Each construction c in i Calculate its sum construction class Each construction c in j The length L of the longest common sub-item, and substitute it into the formula.

[0039] R i =L / max(L) y ,L z )

[0040]

[0041] Where R i Representative construction c i and construction class The higher the correlation between them, the stronger the construction c. i and construction class The higher the degree of correlation, the more L y and L z Representative construction c i and configuration c j The length of the first-order configuration class can be used to calculate the adjacency matrix S1 by combining the score values.

[0042] The beneficial effects of this invention are: it enables the generation of constructional connections, that is, the generation of relationships between constructions, and further, the generation of constructional networks. This allows the model to select the more important constructions from among multiple constructions, thereby more effectively injecting constructional grammar knowledge into pre-trained natural language processing models. Attached Figure Description

[0043] Figure 1 This is a flowchart of the present invention; Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings.

[0045] like Figure 1 As shown, this invention provides a method for generating construction networks based on term collocation analysis, comprising the following steps:

[0046] (1) Using a construction extraction tool based on a pre-trained model, constructions contained in the corpus are extracted, and constructions with a frequency lower than ω% of the number of sentences in the corpus are filtered out, forming a construction library C = {c1, c2, c3, ..., c n};(where 3≤ω≤8).

[0047] Specifically, the process involves first inputting the corpus into a construction extraction tool generated by a pre-trained model. The tool then detects and outputs all constructions contained within the corpus. The specific value of ω can be adjusted based on the actual corpus data. For example, if the corpus contains a large number of sentences and therefore a large number of constructions, a larger ω value should be used to retain more significant constructions; conversely, a smaller ω value should be used to ensure a sufficient number of extracted constructions. Through this extraction process, a construction library C corresponding to the corpus is formed.

[0048] Wherein, configuration c i It contains several access terms U i .

[0049] (2) Perform term analysis on the construction library C extracted in step (1) to obtain term collocation strength and construct construction c. i The corresponding feature vector v i This yields the corresponding feature vector set V = {v1, v2, v3, ..., v n};

[0050] Lexical analysis calculates the lexical terms that appear in a construction and their collocation strength relative to the constructions in that corpus.

[0051] The term analysis employs the Fisher exact test method, and the calculation process is described as follows:

[0052] First, select construction c from the corpus. i and c i One of the access terms U i And perform statistical analysis to obtain the configuration c i The total frequency is a+c; secondly, the access term U is retrieved from the construction library. i The frequency of occurrence 'a'; then, the access term 'U' is retrieved from the corpus. i The total frequency is a + b; then, all constructions n in the corpus are counted, that is, the total number of constructions in the corpus; finally, the entry term U is calculated by subtraction. i The configuration c appears i The frequency of absence b = (a + ba), and the construction c i The term U appears but is an access term. i The frequency of non-occurrence c = (a + ca), configuration c i and access term U iThe frequency d = (nabc) of none of them is calculated using Fisher's exact test formula, i.e. Matching strength can be used to distinguish different configurations and is a characteristic of configurations.

[0053] Using a matrix to store the term analysis results yields a collocation strength matrix M, where each row represents a collocation c. i The vector corresponding to this row, i.e., construction c, represents the collocation strength relationship with all terms in the corpus. i The corresponding feature vector v i The matrix is ​​the feature vector set V = {v1, v2, v3, ..., v} corresponding to the structure library. n}

[0054] (3) Combine the corresponding eigenvectors v of the configuration i The configuration library C is clustered using a pipelined hierarchical clustering approach to obtain the first-level configuration classes. The first layer of the generative network

[0055] Take each feature vector v i As an independent category, an initial neighbor matrix S0 is formed by calculating the distance between vectors, and any value between 5 and 10 times the minimum value in the initial neighbor matrix S0 at this time is used as the threshold.

[0056] The pipeline hierarchical clustering adopts the following method: each time clustering occurs, the minimum distance value in S0 and its corresponding nodes v1 and v2 are searched, the two categories with the minimum distance between v1 and v2 are merged, and this process is repeated until the minimum value remaining in S0 is greater than the threshold.

[0057] After one hierarchical clustering step, clusters with at least t aggregated configurations are selected and merged into the clustering results. The selection process removes some configurations that were forcibly split into different categories because their distance from the set threshold, and these are added to the next clustering step. (t≥2, the specific value is adjusted based on the number of sentences in the corpus)

[0058] The remaining clusters from the previous clustering and filtering step are then re-clustered hierarchically. This process is repeated until a threshold is greater than the furthest distance between configurations. The configuration classes obtained after each filtering step are merged, which constitutes the clustering results, thus generating the first layer of the network.

[0059] (4) Using construction classes The included construction c i Using the length of the longest common sub-item and the average configuration length as characteristics, we obtain the adjacency matrix S1 of the first-level configuration class.

[0060] For construction classes Each construction c in i Calculate its sum construction class Each construction c in j The length L of the longest common sub-item, and substitute it into the formula.

[0061] R i =L / max(L) y ,L z )

[0062]

[0063] Where R i Representative construction c i and construction class The higher the correlation between them, the stronger the construction c. i and construction class The higher the degree of correlation, the more L y and L z Representative construction c i and configuration c j The length of the construct. The higher the score value, the better the construct class. and construction class The closer the values ​​are, the more likely the adjacency matrix S1 of the first-level configuration class can be generated through the score value. The minimum value of the adjacency matrix S1 of the first-level configuration class at this time is 5-10 times as the threshold for clustering in step (5).

[0064] (5) Using the adjacency matrix S1, cluster the first layer of the network through pipeline hierarchical clustering to generate the second layer of the network. The pipeline hierarchical clustering method is consistent with step (3).

[0065] (6) Repeat steps (4) and (5) to obtain the adjacency matrix S of the configuration class again. H-1 Then, using a pipelined hierarchical clustering approach, the (H-1)th layer of the network is clustered to obtain the Hth layer configuration class, which can generate the Hth layer of the network. That is, to generate a structured network.

[0066] Based on the constructional network generated by this invention, and combined with application scenarios, the generated constructional network can be applied to the training of attention-based natural language models to improve the performance of tasks such as sentiment classification, opinion mining, and causal reasoning. Specific examples include:

[0067] First, using word embedding techniques and construction extraction tools, the input corpus is converted into a corresponding vector representation e0, and the construction library C contained therein is extracted. Second, for a construction c in the construction library C... i Retrieve all texts containing the construction c from its corpus. iThe statements are used to generate configuration c by inputting these statements into the attention layer. i The corresponding vector representation is e1. Then, according to the construction network, the vector representation of the construction is passed through the attention layer to obtain the vector representation e2 corresponding to the first-level construction class. This process is repeated until the top layer of the construction network, resulting in the top-level representation e of the network formed by layer-by-layer aggregation. i Finally, the top-level expression e of the structural network is... i By expressing the interaction through the attention layer and all sentence vectors containing its constructions, inter-sentence information can be exchanged through the construction network, which enhances the understanding of sentences by the natural language model and improves the performance of subsequent tasks such as sentiment classification or opinion mining.

[0068] The invention will be further explained below through an example. A corpus, processed using a construction extraction tool and frequency filtering, yields the following constructions:

[0069] 1. 'DET'AUX'VERB'

[0070] 2. 'he"AUX"VERB'

[0071] 3. 'it' AUX' VERB'

[0072] 4. 'NOUN'AUX'ADV'VERB'

[0073] 5. 'NOUN' AUX' VERB'

[0074] 6. 'AUX' ADV' VERB'

[0075] 7.'NOUN"AUX"ADV'

[0076] First, by performing term analysis on the admission terms of part-of-speech tags such as 'DET' and 'AUX', the feature vectors corresponding to the constructions are obtained. Through the first clustering, 1, 2, and 3 form one category, 4 and 5 form another category, and 6 and 7 form yet another category.

[0077] Next, in the second clustering, since these configurations all have the structure of 'AUX' VERB, and because the characteristics of their greatest common sub-items are similar, they are all clustered into the same class, forming the second-level configuration class.

[0078] In higher-level clustering, these constructions form higher-level construction classes with other constructions, eventually forming a construction network on the corpus.

[0079] In the training of natural language models using construction networks, the network first generates a vector representation of each construction. Then, through an attention mechanism, it aggregates information upwards along the network, ultimately generating an aggregated vector representation of these seven constructions. This aggregated vector representation interacts with the sentence vectors containing the constructions within it, enabling information transfer between sentences through the construction network. This allows for more effective injection of construction grammar knowledge into pre-trained NLP models.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0081] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

[0082] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

Claims

1. A method for generating a construction network based on collocation analysis of lexical items, characterized in that, The method comprises the following steps: (1) Through the construction extraction tool based on the pre-training model, the construction contained in the corpus is extracted, and the construction with the frequency lower than the number of corpus sentences ω% is screened out to form the construction library C corresponding to the corpus = { c 1 , c 2 , c 3 , ... , c n}; 3≤ω≤8; (2) Each of the construction formula in the construction formula library C is subjected to term analysis to obtain the corresponding feature vector of the construction formula c i with the term collocation strength c i v i , and the corresponding feature vector group V = { v 1 , v 2 , v 3 , ... , v n} is obtained.​ (3) the corresponding eigenvectors of the combination configuration v i , the first-level configuration class is obtained by clustering the configuration library C in a pipeline level clustering manner , the first layer of the network is generated ; (4) the first-order composition class is determined by the maximum common sub-item length and the average composition length of the included compositions of the included compositions, and the adjacent matrix S1 of the first-order composition class is obtained.​ (5) On the basis of the adjacent matrix S1, the first layer of the network is clustered in a pipeline hierarchical clustering manner to obtain a second-level construct class, and a second layer of the network is generated ; (6) repeating step (4) and step (5) to obtain the adjacent matrix S of the class of configurations again H-1 and the H-1 layer of the network is clustered in the way of pipeline hierarchical clustering to obtain the H-level class of configurations, and the H layer of the network is generated , that is, the configuration network is generated; In the step (3) or step (5), the pipeline hierarchical clustering is performed in the following manner: when clustering, the minimum value and the corresponding node in the adjacent matrix are searched v 1 and v 2, the two clusters with the minimum distance are merged v 1 and v 2, and the process is repeated until the minimum value in the adjacent matrix is greater than a threshold value; after one hierarchical clustering is completed, the classes in which the number of aggregated constructs is not less than t are screened out, the remaining clusters are re-hierarchically clustered, and the process is repeated until the threshold value is greater than the farthest distance in the adjacent matrix; wherein t≥2.

2. The method of claim 1, wherein the method further comprises: The construction c i comprises several admission terms U i .

3. The method of claim 1, wherein, In the step (2), the Fisher exact test method is used in the constructional word analysis, specifically: first, select the constructional word from the corpus c i and c i one access word U i , and conduct statistical analysis to obtain the total frequency a + c of the constructional word c i ; second, retrieve the frequency a of the access word U i from the constructional word library. The total frequency of the access word U in the corpus is a + b; then all the patterns in the corpus are counted, that is, the total number of patterns n in the corpus; finally, the frequency of the access word U that appears but the pattern does not appear is calculated by subtraction, that is, b = (a + b - a) i i The total frequency of the access word U in the corpus is a + b; then all the patterns in the corpus are counted, that is, the total number of patterns n in the corpus; finally, the frequency of the access word U that appears but the pattern does not appear is calculated by subtraction, that is, b = (a + b - a) i c i The total frequency of the access word U in the corpus is a + b; then all the patterns in the corpus are counted, that is, the total number of patterns n in the corpus; finally, the frequency of the access word U that appears but the pattern does not appear is calculated by subtraction, that is, b = (a + b - a) i c i The total frequency of the access word U in the corpus is a + b; then all the patterns in the corpus are counted, that is, the total number of patterns n in the corpus; finally, the frequency of the access word U that appears but the pattern does not appear is calculated by subtraction, that is, b = (a + b - a) i c 1 The total frequency of the access word U in the corpus is a + b; then all the patterns in the corpus are counted, that is, the total number of patterns n in the corpus; finally, the frequency​​​​ The calculated p value is the collocation strength of the word item; The term collocation strength of all the access terms is calculated to obtain the corresponding feature vector set V = { v 1 , v 2 , v 3 , ... , v n}.

4. The method of claim 1, wherein the method further comprises: In the step (3), before clustering, an initial adjacent matrix is generated, specifically, each construction is taken as a cluster, and then the initial adjacent matrix S0 is formed by calculating the distance between clusters.

5. The method of claim 4, wherein the method further comprises: Any value in the range of 5-10 times of the minimum value in the initial adjacent matrix S0 is taken as the threshold value for clustering in the step (3).

6. The method of claim 5, wherein the method further comprises: In the step (4), any value in the range of 5-10 times of the minimum value in the adjacent matrix S1 of the first-level construction class is taken as the threshold value for clustering in the step (5).

7. The method of claim 1, wherein the method further comprises: In the step (3), the adjacent matrix searched during clustering is S0; in the step (5), the adjacent matrix searched during clustering is S1.

8. The method of claim 1, wherein the method further comprises: In the step (4), specifically, For each conformation in the conformation class c i , the maximum common sub-item length L of each conformation in the conformation class c j is calculated and brought into the formula​​ R i = L / max(L y , L z ) Where R i Representative construction c i and construction class The higher the correlation between them, the stronger the construction. c i and construction class The higher the degree of correlation, the more L y and L z Representative construction c i and construction c j The length of the first-level construction class is used to calculate the adjacency matrix S1 by combining the score values.

Images