Semantic analysis and matching method and system based on large model

Abstract

The invention discloses a semantic analysis and matching method and system based on a large model. In an existing process of matching party expression language information with laws, no special is made in terms of keyword screening, matching is easily carried out by using nouns, and analysis cannot be carried out through behaviors. According to the method, the original text data is cleaned, the keywords are matched based on the verbs, the nouns are filtered, classification is performed according to the keywords, and the same behaviors are classified and divided, so that the accuracy of behavior analysis in the dialogue content is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of language processing, and relates to a semantic analysis and matching method and system based on a large model. Background Art

[0002] As users' demands for information types and topics become more and more extensive, retrieval platforms need to provide text data that is as diverse and comprehensive as possible, which in turn generates massive amounts of text data. Efficiently retrieving the information users need from such massive amounts of text data has become a core challenge currently faced by retrieval platforms, especially in the political and legal fields, which require even more accurate matching of texts.

[0003] In the existing process of matching the language information expressed by the parties and the legal provisions, there is no specific reference to keyword screening, which is easy to match by nouns and cannot be analyzed by behavior. Analysis of behavior is easily disturbed by nouns, resulting in poor analysis results. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the present invention proposes a semantic analysis and matching method and system based on a large model.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] On the one hand, the present invention provides a semantic analysis and matching method based on a large model, comprising:

[0007] First, semantic disambiguation is performed on the communication text to obtain the disambiguated text content;

[0008] Extract keywords from the disambiguated text content, and use a decision tree to classify the disambiguated text content based on the keywords;

[0009] The extracted classified text content of each category is matched with the text content to be matched to obtain the matching content corresponding to the text content of each category.

[0010] Furthermore, the communication text is text data obtained by extracting audio content from the communication audio.

[0011] Furthermore, the semantic disambiguation method is:

[0012] Setting a first dictionary network to divide the communication text according to words, and obtaining a text list consisting of words and connecting characters;

[0013] Take each word as the target word according to the order of precedence and extract the context information of the target word;

[0014] All the meanings of the target word are extracted from the first dictionary network, and the target word is combined with the preceding and following connected characters and words to obtain the meaning of the target word pointed to by the preceding and following words of the target word.

[0015] Furthermore, the method for extracting keywords from the text content after disambiguation is:

[0016] The disambiguated text content is compared through the first dictionary network, and the words except verbs in the dictionary network are deleted, and the remaining verbs create text sequences;

[0017] Define a co-occurrence matrix C, where C ij Indicates word v i and word v j The number of co-occurrences within a certain window range w;

[0018] Construct a word graph, where each node represents a word, and the edge represents the co-occurrence relationship between words. The weight of the edge is calculated based on the co-occurrence matrix C, as follows:

[0019]

[0020] Where W ij is from the word v i To word v j The weight of the edge;

[0021] The initial weight of each word is set to the same, calculated using the following formula:

[0022]

[0023] in Indicates word v i The initial weight of

[0024] Use TextRank to iteratively calculate the weight of each word. The formula is as follows:

[0025]

[0026] in is the word v at the tth iteration i The weight of the word v, d is the damping coefficient, which controls the consideration of the importance of other words, and In(i) is the reference word v. i The set of all words in , Out(j) is the word v j The set of all words pointed to;

[0027] Repeat the above calculation steps until the weight converges, that is, the weight change between two adjacent iterations is less than the set threshold ∈. The calculation method is:

[0028] ∥r (t+1) -r (t) ∥<∈

[0029] After the iteration converges, use the final weight r i To sort the words, select the top N words with the highest weight as keywords.

[0030] Furthermore, the method for classifying the disambiguated text content according to keywords is:

[0031] First, the verbs appearing in a single window are normalized in the dictionary network, and the verbs with similar meanings are normalized to obtain the top verbs in each window;

[0032] Based on the similarity of the top verbs in each window, a decision tree is used to classify the disambiguated text content of each window.

[0033] The decision tree uses the top verbs that appear more than a certain frequency in all windows as a classification method;

[0034] The branches of the decision tree are the frequencies of occurrence of the top-level verbs.

[0035] Furthermore, the top verb is the initial verb that appears most frequently in a single window.

[0036] Furthermore, the method for matching the classified text content semantic information with the text content to be matched is:

[0037] Extract verbs from the text content to be matched based on the dictionary network;

[0038] Analyze the relevance of the top verbs in each category of text content and the verbs extracted from the text content to be matched. The specific formula is:

[0039]

[0040] Where x is the top-level verb, y i is the i-th text content to be matched, P(x,y m ) is the verb similarity of the x verb in the mth item of the text content to be matched, and H is the similarity of any top-level verb in different items of the text content to be matched;

[0041] By comparing the similarity of any top-level verb in different items of to-be-matched text content, the text content with the highest similarity is selected as the to-be-matched text content that matches the semantic information of the current category text content.

[0042] On the other hand, the present invention provides a semantic analysis and matching system based on a large model, comprising:

[0043] Data input module, where users input the communication text content to be processed;

[0044] The data processing module disambiguates the communication text content, extracts keywords from the disambiguated text content, classifies the disambiguated text content according to the keywords, and matches the semantic information of the classified text content with the text content to be matched;

[0045] A comparison module compares the frequency of keywords and the similarity between the semantic information of the classified text content and the text content to be matched;

[0046] The data output module outputs the text content to be matched corresponding to the user communication text content.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention cleans the original text data, and the obtained data has high accuracy; the present invention matches keywords based on verbs, filters nouns, and uses all computing power to analyze the behaviors in the conversation content, and has high accuracy in behavior analysis. The present invention classifies based on keywords, categorizes and divides the same behaviors, and improves the accuracy of behavior analysis in the conversation content. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the method of the present invention;

[0050] Figure 2 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] like Figure 1-Figure 2 As shown, the technical solution adopted by the present invention is as follows: a semantic analysis and matching method and system based on a large model, comprising:

[0053] The user inputs the communication text content to be processed in the data input module, and the communication text is the text data after the communication audio is extracted from the audio content. At this time, the text data contains homophones or synonyms, which is dirty data and cannot be directly semantically analyzed. The data processing module needs to clean the text data.

[0054] The cleaning operation performed on text data is called disambiguation. The data processing module performs semantic disambiguation on the communication text to obtain the disambiguated text content.

[0055] The semantic disambiguation method is:

[0056] A first dictionary network is set up to divide the communication text according to words to obtain a text list consisting of words and connecting characters. The first dictionary network can be an internal dictionary resource freely developed by the user and called through an API interface or a data exchange method.

[0057] Take each word as the target word according to the order of precedence, and extract the context information of the target word. Disambiguate each word one by one to avoid data deadlock caused by simultaneous processing or data misjudgment caused by delay in the extracted text content.

[0058] All the meanings of the target word are extracted from the first dictionary network, and the target word is combined with the preceding and following connected characters and words to obtain the meaning of the target word pointed to by the preceding and following words of the target word.

[0059] At this point, the text in the text data has been cleaned to obtain the original data of the user conversation. Errors such as homophones in the text data have been filtered jointly with the context, and the text data has become accurate data that meets the accuracy requirements in the legal scenario. There will be no deviation in the subsequent next processing, so that the text analysis meets the accuracy requirements in the legal scenario.

[0060] After the cleaning operation on the text data is completed, the data processing module also extracts keywords from the disambiguated text content.

[0061] The method for extracting keywords is:

[0062] The disambiguated text content is compared through the first dictionary network, and the words other than verbs in the dictionary network are deleted, and the remaining verbs create a text sequence. In this way, the disambiguated text content can be analyzed based on behavior, which is more in line with the legal judgment method and is judged based on the behavioral characteristics of the parties.

[0063] Define a co-occurrence matrix C, where C ij Indicates word v i and word v j The number of co-occurrences within a certain window range w. The matrix is ​​generated using a sliding window, and the window size w determines the context range of the co-occurrence. The larger the window range, the wider the feature word extraction range, the more reference content, and the more accurate the feature word analysis. Conversely, the smaller the window range, the more vague the feature word analysis.

[0064] Construct a word graph, where each node represents a word, and the edge represents the co-occurrence relationship between words. The weight of the edge is calculated based on the co-occurrence matrix C, and the formula is as follows:

[0065]

[0066] Where W ij is from the word v i To word v j The weight of the edge is calculated. The correlation weight between two words is calculated to obtain the correlation between a single word and other words.

[0067] The initial weight of each word is set to the same, calculated using the following formula:

[0068]

[0069] in Indicates word v i The initial weight of each word is the same, which reduces the inaccurate judgment of keywords by other factors and ensures that the extracted keywords accurately match the legal content.

[0070] Use TextRank to iteratively calculate the weight of each word. The formula is as follows:

[0071]

[0072] in is the word v at the tth iteration i The weight of the word v, d is the damping coefficient, which controls the consideration of the importance of other words, and In(i) is the reference word v. i The set of all words in , Out(j) is the word v j The set of all words pointed to.

[0073] Repeat the above calculation steps until the weight converges, that is, the weight change between two adjacent iterations is less than the set threshold ∈. The calculation method is:

[0074] ∥r (t+1) -r (t) ∥<∈

[0075] After the iteration converges, use the final weight r i To sort the words, select the top N words with the highest weights as keywords, and get the most representative keywords.

[0076] After the keywords are selected, the data processing module also uses a decision tree to classify the disambiguated text content based on the keywords.

[0077] The method for classifying the disambiguated text content according to keywords is:

[0078] First, the verbs that appear in a single window are normalized in the dictionary network, and verbs with similar meanings are normalized. Due to the characteristics of Chinese, words with the same meaning have multiple expressions, and using multiple expressions is not suitable for matching legal provisions. Therefore, it is necessary to normalize verbs with similar meanings to obtain the top verbs in each window. The top verb is the initial verb with the highest frequency in a single window. While being the most representative, it reduces the modification of the original text data to avoid ambiguity and lead to errors in matching legal provisions.

[0079] According to the similarity of the top verbs in each window, a decision tree is used to classify the disambiguated text content corresponding to each window.

[0080] The decision tree uses the top verbs that appear more than a certain frequency in all windows as a classification method. This prevents a single word from causing unnecessary analysis, saves system resources, and filters out words that appear only a few times.

[0081] The branch judgment method of the decision tree mentioned above is the frequency of occurrence of top-level verbs. By judging the frequency of top-level verbs, the text content after disambiguation is classified to obtain the category corresponding to each top-level verb. It can well match the legal content for a certain behavior of the parties.

[0082] The text content of each category after classification is extracted and matched with the text content to be matched to obtain the legal provisions corresponding to the text content of each category.

[0083] The method for matching the classified text content semantic information with the text content to be matched is:

[0084] Verbs are extracted from the content of legal provisions based on the dictionary network. Behavior is the main matching method in legal provisions. Using verbs for matching can well match the behavior in the communication between the parties and obtain the corresponding content of legal provisions.

[0085] Analyze the relevance of the top verbs in each category of text content and the verbs extracted from the legal provisions to be matched. The specific formula is:

[0086]

[0087] Where x is the top-level verb, y i is the i-th text content to be matched, P(x,y m ) is the verb similarity of verb x in the mth item of text content to be matched, and H is the similarity of any top-level verb in different items of text content to be matched.

[0088] By comparing the similarity of any top-level verb in different legal provisions, the legal provision with the highest similarity is selected as the legal provision content that matches the semantic information of the current category text content. Through the data output module, the legal provision content corresponding to the user communication text content is output.

[0089] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A semantic analysis and matching method based on a large model, characterized in that: Included are: First, semantic disambiguation is performed on the communication text to obtain the disambiguated text content; Extract keywords from the disambiguated text content, and use a decision tree to classify the disambiguated text content based on the keywords; The extracted classified text content of each category is matched with the text content to be matched to obtain the matching content corresponding to the text content of each category.

2. A semantic analysis and matching method based on a large model according to claim 1, characterized in that ,The communication text is the text data after the audio content is extracted from the communication audio.

3. The semantic analysis and matching method based on a large model according to claim 1, characterized in that ,The semantic disambiguation method is: Setting a first dictionary network to divide the communication text according to words, and obtaining a text list consisting of words and connecting characters; Take each word as the target word according to the order of precedence and extract the context information of the target word; All the meanings of the target word are extracted from the first dictionary network, and the target word is combined with the preceding and following connected characters and words to obtain the meaning of the target word pointed to by the preceding and following words of the target word.

4. The semantic analysis and matching method based on a large model according to claim 3 is characterized in that ,The method for extracting keywords from the text content after disambiguation is: The disambiguated text content is compared through the first dictionary network, and the words except verbs in the dictionary network are deleted, and the remaining verbs create text sequences; Define a co-occurrence matrix C, where C ij Indicates word v i and word v j The number of co-occurrences within a certain window range w; Construct a word graph, where each node represents a word, and the edge represents the co-occurrence relationship between words. The weight of the edge is calculated based on the co-occurrence matrix C, as follows: Where W ij It is from the word v i To word v j The weight of the edge; The initial weight of each word is set to the same, calculated using the following formula: r i (0) ＝1 / n in Indicates word v i The initial weight of Use TextRank to iteratively calculate the weight of each word. The formula is as follows: where r i (t) is the word v at the tth iteration i The weight of the word v, d is the damping coefficient, which controls the consideration of the importance of other words, and In(i) is the reference word v. i The set of all words in , Out(j) is the word v j The set of all words pointed to; Repeat the above calculation steps until the weight converges, that is, the weight change between two adjacent iterations is less than the set threshold ∈. The calculation method is: ||r (t+1) -r (t) ||<∈ After the iteration converges, the final weight r is used i To sort the words, select the top N words with the highest weight as keywords.

5. A semantic analysis and matching method based on a large model according to claim 4, characterized in that ,The method for classifying the disambiguated text content based on keywords is: First, the verbs appearing in a single window are normalized in the dictionary network, and the verbs with similar meanings are normalized to obtain the top verbs in each window; Based on the similarity of the top verbs in each window, a decision tree is used to classify the disambiguated text content of each window. The decision tree uses the top verbs that appear more than a certain frequency in all windows as a classification method; The branches of the decision tree are the frequencies of occurrence of the top-level verbs.

6. A semantic analysis and matching method based on a large model according to claim 5, characterized in that ,The top verb is the initial verb with the highest ,frequency in a single window.

7. The semantic analysis and matching method based on a large model according to claim 5, characterized in that ,The method for matching the classified text content semantic information with the text content to be matched is: Extract verbs from the text content to be matched based on the dictionary network; Analyze the relevance of the top verbs in each category of text content and the verbs extracted from the text content to be matched. The specific formula is: Where x is the top-level verb, y i is the i-th text content to be matched, P(x,y m ) is the verb similarity of the x verb in the mth item of the text content to be matched, and H is the similarity of any top-level verb in different items of the text content to be matched; By comparing the similarity of any top-level verb in different items of to-be-matched text content, the text content with the highest similarity is selected as the to-be-matched text content that matches the semantic information of the current category text content.

8. A semantic analysis and matching system based on a large model, characterized by , Data input module, where users input the communication text content to be processed; The data processing module disambiguates the communication text content, extracts keywords from the disambiguated text content, classifies the disambiguated text content according to the keywords, and matches the semantic information of the classified text content with the text content to be matched; A comparison module compares the frequency of keywords and the similarity between the semantic information of the classified text content and the text content to be matched; The data output module outputs the text content to be matched corresponding to the user communication text content.

Images