A Chinese syntactic analysis method and system
Through the recognition of sentence segmentation, word segmentation and phrases of Chinese syntactic analysis method, combined with idiomatic and general grammatical rules, the syntactic analysis process is simplified, the problem of insufficient accuracy and speed in the existing technology is solved, and more efficient syntactic analysis is achieved .
Patent Information
- Application Number
- CN202110023058.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-08
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-01-08
AI Technical Summary
The existing syntactic analysis methods have shortcomings in accuracy and speed, especially when dealing with complex sentences and large amounts of documents, the analysis level and process are complicated and cumbersome, making it difficult to improve the accuracy and speed.
A Chinese syntactic analysis method is adopted to identify sentence structure and grammatical relationships through sentence segmentation, word segmentation, recognition of phrases and phrases, combining idiomatic and general grammatical rules, and identify sentence structure and grammatical relationships, and use word databases, idiomatic phrase node databases and sentence structure databases. Carry out intelligent analysis to simplify the analysis process.
Improve the accuracy and speed of syntactic analysis, especially when parsing sentences with special structures and a large number of documents, a simpler and faster analysis process is achieved.
Smart Images

Figure CN112699664B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language analysis, and in particular to a Chinese syntactic analysis method and system. Background Art
[0002] Natural language processing is a branch of artificial intelligence and linguistics. It is a technology that uses computer tools to process and process natural language information in written and oral forms that are unique to humans. Natural language processing can be roughly divided into two levels. One is shallow analysis, such as word segmentation and part-of-speech tagging. This technology only needs to analyze and process the local scope of the sentence. The other level is deep processing of language, which requires a comprehensive analysis of the syntax, semantics, and pragmatics of the sentence.
[0003] At present, the mainstream syntactic analysis method mainly performs syntactic analysis through the dependency relationship between words. The analysis level and parsing process are complicated and cumbersome, the relationship between sentence components is cumbersome, and the parsing accuracy and speed still need to be improved. Syntactic analysis is a key technical link in natural language processing and is widely used. For example, in an automatic question-answering device, by combining syntactic analysis technology, the corresponding answer is found in the knowledge base by calculating the similarity between the sentence asked and the corresponding sentence in the knowledge base. In information filtering technology, information that users may not want to see is automatically filtered out by calculating text similarity. However, due to the defects of existing technologies, the accuracy, speed and recall rate are low. Summary of the invention
[0004] In response to the above problems, the present invention proposes a Chinese syntactic analysis method and system, which starts from the language features, fully explores the characteristics of Chinese characters and structures, utilizes existing Chinese grammar and habits, and realizes intelligent analysis of Chinese sentence-level semantics through rule design, so that the machine can "understand" the language.
[0005] According to one aspect of the present invention, a Chinese syntactic analysis method is provided, comprising the following steps:
[0006] Step S1: Segment the input text into sentences and words, and mark the parts of speech;
[0007] Step S2: Identify phrases in the sentence;
[0008] Step S3: identifying phrases in the sentence;
[0009] Step S4: Identify the structure of the sentence;
[0010] Step S5: Perform syntactic analysis on the sentence, and annotate and output the grammatical relations of the sentence components according to Chinese grammar.
[0011] Further, in the step S2, the phrases include idiomatic phrases and general phrases. The idiomatic phrases refer to idiomatic expressions in the form of phrases and idiomatic expressions in the form of phrases containing specific characters or nouns, and the general phrases refer to compound nouns.
[0012] Further, in the step S3, the phrases include idiomatic phrases and general phrases. The idiomatic phrases refer to idiomatic expressions in the form of phrases with characters, words or their collocations in the phrase node library as nodes, which are composed of nodes and components. The components include nouns, phrases, verbs, adjectives, adverbs, phrases or clauses.
[0013] Further, the step 3 includes:
[0014] S31. Compare each clause from left to right with the phrase node library to identify single nodes or double nodes;
[0015] S32. Mark the content between the closest double node and the double node from left to right in the sentence as a double-node idiomatic phrase;
[0016] S33. Identify general phrases, which refer to phrases composed of verbs + nouns and "de" character adjective phrases, where before the "de" character are nouns, pronouns, adjectives or "noun + verb";
[0017] S34. If a pre-single node is identified, mark the part from the pre-single node to the "verb + noun", or verb, or phrase or noun that follows and is not in the "de" character adjective phrase or double-node idiomatic phrase as a single-node idiomatic phrase. If a post-single node is identified, mark the part from the post-single node to the "verb + noun", or verb, or phrase or noun immediately before it as a single-node idiomatic phrase.
[0018] Further, the step S4 includes:
[0019] S41. Compare with the idiomatic sentence pattern library to determine whether the sentence is a compound sentence;
[0020] S42. Compare with the idiomatic sentence pattern library to identify and mark idiomatic sentence patterns;
[0021] S43. Identify and mark the core verb of the sentence;
[0022] S44. Identify the structure to which the "de" character belongs; when there are more than two "de" characters in a clause and both before and after the "de" character are nouns, phrases or pronouns, parse the "de" character as the possessive relationship between the nouns and pronouns according to Chinese grammar, and take the last "de" character and the content before it as the attributive of the content after it;
[0023] S45. Parse the pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, auxiliary words, numeral-classifiers, relative words, and punctuation marks in the sentence according to Chinese grammar, and mark the grammatical components or relationships.
[0024] S46. Determine and mark the subject, predicate, and object of the sentence. For general sentence patterns, the core verb is the predicate of the sentence. Excluding function words (adverbs, prepositions, conjunctions, auxiliary words, interjections, and onomatopoeias), relative words, phrases used as attributives or adverbials, and clauses, mark the components to the left of the core verb as the subject of the sentence, and mark the components to the right of the core verb as the object of the sentence. The determination of the subject, predicate, and object of idiomatic sentence patterns is based on manual settings.
[0025] Further, the step S43 includes:
[0026] 1) Find the verbs that cannot be used as core verbs in general sentence patterns. The judgment method is as follows: The verb at the beginning of the main sentence is not used as the core verb. The verb at the beginning of the sentence includes the verb with a phrase, adverb, adjective, or numeral-classifier before it; the verb in a phrase is not used as the core verb; the verb in an adjective phrase is not used as the core verb; the verb in an idiomatic phrase with a non-verb node is not used as the core verb; the verb before / after the word "的" is not used as the core verb.
[0027] 2) Except for the verbs that are not used as core verbs, mark the first verb from left to right in the main sentence or clause as the core verb of the sentence.
[0028] According to another aspect of the present invention, a Chinese syntactic analysis system is proposed, including a word recognition module, a phrase recognition module, a sentence recognition module, a syntactic analysis module, and a database. Among them,
[0029] The database includes a word library, an idiomatic phrase node library, and an idiomatic sentence pattern library;
[0030] The word recognition module is used to recognize words, including nouns and phrases;
[0031] The phrase recognition module is used to recognize phrases, including idiomatic phrases and general phrases;
[0032] The sentence recognition module is used to recognize the main structure of the sentence, including general sentence patterns and idiomatic sentence patterns;
[0033] The syntactic analysis module is used to perform syntactic analysis on the sentence, mark the grammatical components of the sentence components, and output the syntactic analysis results, including the applied grammatical rules. The sentence components include words, phrases, core verbs, adjectives, adverbs, numeral-classifiers, etc.
[0034] Further, in the phrase recognition module, the clause is compared with the node library to find whether single nodes or double nodes exist; the content between the nearest double nodes from left to right in the sentence is marked as a double-node idiomatic phrase; one double-node idiomatic phrase may contain one or several other double-node idiomatic phrases, which are called nested phrases and are marked; general phrases are recognized, where the general phrases refer to phrases composed of a verb + a noun and "de" character adjective phrases, and before the "de" character are a noun, a pronoun, an adjective, or a "verb + noun"; if there is a single node in the front, the part from this node to the "verb + noun", or verb, or phrase or noun that follows and is not in the "de" character adjective phrase and the idiomatic phrase is marked as a single-node idiomatic phrase, and if there is a single node in the back, the part from this node to the "verb + noun", or verb, or phrase or noun that is紧靠 its front is marked as a single-node idiomatic phrase.
[0035] Further, in the sentence recognition module, the steps of recognition and marking include:
[0036] S41. Compare with the idiomatic sentence pattern library to determine whether the sentence is a compound sentence;
[0037] S42. Identify and mark whether the main clause or the subordinate clause is a general sentence pattern or an idiomatic sentence pattern according to the idiomatic sentence pattern library. When judging, if the sentence is not an idiomatic sentence pattern, it is a general sentence pattern;
[0038] S43. Identify and mark the core verb of the sentence;
[0039] S44. Identify the structure to which the "de" character belongs; the nouns of the belonging relationship are grouped as one noun, and other phrases with "de" as an attributive are distinguished. When there are at least two "de" characters in the phrase, and both before and after at least two "de" characters are nouns, phrases, or pronouns, the "de" character is parsed as the belonging relationship between the nouns, phrases, or pronouns according to Chinese grammar, and the last "de" character and the content before it are marked as the attributive of the content after it, so as to integrate the nouns, phrases, or pronouns with the said relationship and can be distinguished from other phrases with "de" as an attributive;
[0040] S45. Analyze and mark the pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, auxiliary words, numeral-classifiers, relative words, and punctuation marks in the sentence according to Chinese grammar;
[0041] S46. Determine and mark the subject, predicate, and object of the sentence. Specifically: for a general sentence pattern, the core verb is the predicate of the sentence. Excluding function words (adverbs, prepositions, conjunctions, auxiliary words, interjections, and onomatopoeias), relative words, phrases used as attributives or adverbials, and clauses, the components on the left side of the core verb are marked as the subject of the sentence, and the components on the right side of the core verb are marked as the object of the sentence; the judgment of the subject, predicate, and object of an idiomatic sentence pattern is based on manual settings.
[0042] Further, the step of identifying and labeling the core verb of the sentence in step S43 includes:
[0043] Identifying idiomatic sentence patterns and their core verbs;
[0044] Identifying the core verb of a general sentence, the steps are:
[0045] 1) Find the verbs in the sentence that cannot be used as the core verb. The judgment method is: the verb at the beginning of the main sentence is not used as the core verb. The verb at the beginning of the sentence includes the verb preceded by a phrase, adverb, preposition, adjective or numeral; the verb in a phrase is not used as the core verb; the verb in an adjective phrase is not used as the core verb; the verb in an idiomatic phrase with a non-verb node is not used as the core verb; the verb before / after the character "的" is not used as the core verb;
[0046] 2) Except for the verbs that cannot be used as the core verb, the first verb from left to right in the main sentence or clause is used as the core verb of the sentence.
[0047] The beneficial effects of the present invention are: the syntactic analysis level and process are more concise, intuitive and fast than the prior art, and the accuracy is improved, especially for parsing sentences with special structures and a large number of documents. The present invention can be widely applied to various scenarios in the field of artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 is a schematic flow chart of a Chinese syntactic analysis method according to an embodiment of the present invention;
[0049] Figure 2 is a schematic diagram of the effect of sentence parsing according to the prior art;
[0050] Figure 3 is a schematic diagram of the effect of sentence parsing according to the prior art;
[0051] Figure 4 is a schematic diagram of the effect of sentence parsing according to the prior art;
[0052] Figure 5 is a schematic structural diagram of a Chinese syntactic analysis system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0053] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. The following described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0054] According to one aspect of the present invention, a Chinese syntactic analysis method is proposed. Figure 1 As shown, the following steps are included:
[0055] Step S1: Segment the input text into sentences and words, and mark the parts of speech;
[0056] Step S2: Identify and label the words and phrases in the sentence;
[0057] Step S3: Identify and mark idiomatic phrases in the sentence;
[0058] Step S4: Identify and annotate the structure of the sentence;
[0059] Step S5: Perform syntactic analysis on the sentence, and annotate and output the grammatical relations of the sentence components according to Chinese grammar.
[0060] In step S1, existing methods and tools can be used to segment sentences, words, and mark parts of speech, such as using Harbin Institute of Technology LTP. A sentence is a complete sentence from the beginning to the end of the period, and the sentences separated by commas are clauses. The first clause is the main clause; a sentence with only one clause is a single sentence.
[0061] In step S2, the segmented words in each sentence are matched with the words and phrases in the word library, and the phrases are marked as nouns. The phrases include idiomatic phrases and general phrases. Idiomatic phrases refer to idiomatic expressions in phrase form and idiomatic expressions in phrase form containing specific characters or nouns; general phrases refer to common combined nouns; idiomatic phrases and general phrases can be pre-processed and acquired and stored in a vocabulary library by machine learning, or by a combination of machine learning and manual methods; the machine learning processing method refers to using a computer to extract a number of repeated characters based on relevant corpus to form a phrase, or to extract the same characters or words that appear repeatedly in different phrases as specific characters or words in idiomatic phrases; the pre-processing by a combination of machine learning and manual methods refers to manually setting specific characters or nouns in a phrase, and then using a computer to extract a number of repeated and continuous characters containing the specific characters or nouns based on relevant corpus to form a phrase; idiomatic phrases can also be dynamically acquired in the sentence parsing process according to the pre-set "specific characters or nouns" plus the first noun, verb or phrase before it, and stored in a vocabulary library, thereby obtaining some new idiomatic phrases.
[0062] ① Idiomatic phrases, such as the words in <> in the following example.
[0063] <In summary> is an idiomatic expression in the form of a phrase;
[0064] <Ventilation device> is a customary expression including the specific noun "device".
[0065] ②General phrases;
[0066] For example: internal combustion engine.
[0067] In step S3, phrase recognition is performed on each clause to maximize the integration of the clauses and label them into several major parts, which is beneficial to the subsequent syntactic analysis and judgment.
[0068] Phrases are divided into idiomatic phrases and general phrases. Idiomatic phrases refer to idiomatic expressions in the form of phrases composed of specific words or terms (or their collocations) and components. The specific words or terms (or their collocations) are called the "nodes" of the phrase, and the "components" refer to nouns, phrases, verbs, adjectives, adverbs, phrases or clauses, which are represented by "——". Idiomatic phrases are divided into single-node phrases and double-node phrases. Among them, single-node phrases have one node (left node or right node), and double-node phrases have two nodes (left node and right node); the nodes can be manually set and pre-processed; the method for judging double-node idiomatic phrases is that the two nodes closest in distance in the sentence from left to right and the content between the nodes form a double-node idiomatic phrase; the nodes of double-node idiomatic phrases can be manually set and pre-processed, and can also be pre-processed by combining machine learning methods and stored in the idiomatic phrase node library; the method of combining machine learning pre-processing refers to using a computer to extract a pair of words or terms that appear repeatedly at a certain distance (words or terms) from relevant corpora as a double-node pair, which is stored in the idiomatic phrase node library after manual review and processing. For example, "due to" and "the reason for" in the following examples are double-nodes; a double-node idiomatic phrase may contain one or several other double-node idiomatic phrases, which are called nested phrases. The nested phrases can share one node. The nested phrase uses the double-node with the longest distance and the content between them as the double-node idiomatic phrase. See the following examples for details. General phrases refer to non-idiomatic phrases, specifically phrases composed of "verb + noun" and "adjective phrases with 'de'". Among them, before the "de" are nouns, pronouns, adjectives or "verb + noun"; if a single node is recognized in the front, the part from this node to the "verb + noun", or verb, or phrase or noun that follows and is not in the "adjective phrase with 'de'" and double-node idiomatic phrase is labeled as a single-node idiomatic phrase; if a single node is recognized in the back, the part from this node to the "verb + noun", or verb, or phrase or noun that is紧靠 its front is labeled as a single-node idiomatic phrase.
[0069] The following specifically gives examples to illustrate various phrase forms.
[0070] (1) Double-node idiomatic phrases
[0071] <Due to - - the reason for>,
[0072] Soybean seeds < because planting environment Causes >> result in a low germination rate;
[0073] soybean seeds < because are damaged by pests Causes > resulting in a low germination rate;
[0074] soybean seeds < because pests appear in the soil Causes > resulting in a low germination rate.
[0075] (2) Embedded phrases
[0076] <after ——>, <on the surface of ——>
[0077] 《seeds < after after soaking surface"
[0078] Among them, <after ——> is included in 《on the surface of ——》 and shares a node "after".
[0079] (3) General phrases
[0080] <verb + noun +'s>,
[0081] This is a < keep away connecting rod of the crank axis;
[0082] Among them, "far from the crank axis" is an adjectival phrase with the particle "de".
[0083] (4) Single-node idiomatic phrases
[0084] ① <in ——>
[0085] Currently, soybean seeds <
[0086] during the planting process> have a low germination rate due to pest infestation and planting environment reasons;
[0087] Among them, "zhong" is a post-single node and "planting process" is a phrase.
[0088] < pass install an exhaust purification device>, and purify the CO contained in the exhaust.
[0089] Among them, "through" is a pre-single node, "exhaust purification device" is an idiomatic phrase (phrases are all nouns), "install an exhaust purification device" is a "verb + noun" phrase, and "through installing an exhaust purification device" is a pre-single node idiomatic phrase.
[0090] In this step, (1) compare each clause with the node library to check whether single nodes or double nodes exist. The double nodes closest to each other from left to right in a sentence and the content between them form a double-node idiomatic phrase; one double-node idiomatic phrase may contain one or several other double-node idiomatic phrases, called nested phrases. Nested phrases can share a node, and the nested phrase with the longest distance between double nodes and the content between them is used as the double-node idiomatic phrase; (2) identify general phrases, that is, the part of the clause excluding the idiomatic double-node phrases. Look for a verb from left to right in the clause. If a noun follows the verb (at this time, the noun is not followed by "的", and the noun may be a word or a phrase), mark it as a "verb + noun" phrase. If a noun, pronoun, adjective, or "verb + noun" is followed by the word "的", mark it as an adjective phrase; (3) if there is a pre-single node, mark the part from this node to the "verb + noun", or verb, or phrase or noun that is not in the "的" word adjective phrase and the double-node idiomatic phrase immediately following it as a single-node idiomatic phrase. If there is a post-single node, mark the part from this node to the "verb + noun", or verb, or phrase or noun immediately preceding it as a single-node idiomatic phrase. Specifically, for a pre-single node, for the components following this node and not in the "的" word adjective phrase and the double-node idiomatic phrase, check in turn whether it is a "verb + noun", or verb, or phrase or noun. Then the part from this node to the found "verb + noun", or verb, or phrase or noun forms a single-node idiomatic phrase; for a post-single node, for the components immediately preceding this node, check in turn whether it is a "verb + noun", or verb, or phrase or noun. Then the part from this node to the found "verb + noun", or verb, or phrase or noun forms a single-node idiomatic phrase.
[0091] In step S4, both the main clause and the subordinate clause can only have one core verb. The main clause refers to the first clause in the whole sentence composed of several subordinate clauses. A simple sentence is regarded as the main clause. The specific steps are as follows:
[0092] S41. Compare with the idiomatic sentence pattern library to determine whether the sentence is a compound sentence. Compound words, such as "如果——,那么——". If there is a compound word, the content after the compound word is used as a simple sentence to be analyzed.
[0093] For example: if The electric motor operates to rotate the idler shaft, and the additional rotation will accelerate the idler gear.
[0094] S42. Identify and label whether the main clause or subordinate clause is a general sentence pattern or an idiomatic sentence pattern according to the idiomatic sentence pattern library. A general sentence pattern refers to a sentence pattern in which the sentence components are arranged in a general order (for example, subject - core verb - object), and an idiomatic sentence pattern refers to a sentence pattern in which the conventional component arrangement order is relatively special (for example, a special interrogative sentence) or has local characteristics (specific words, phrases or components); an idiomatic sentence pattern is composed of specific words, phrases or components and other components. The "specific components" refer to specific parts of speech or phrases, etc.; the "other components" refer to general nouns, verbs or phrases, etc., represented by "——"; the idiomatic sentence pattern is set and pre - processed manually, and the pre - processed idiomatic sentence pattern is marked with local grammatical features of the sentence pattern, such as the core verb, etc.
[0095] Examples of general sentence patterns: The present invention Involved in the technical field of crop planting. This sentence has a subject - predicate - object structure. When judging, if the sentence is not an idiomatic sentence pattern, it is a general sentence pattern. Examples of idiomatic sentence patterns in the sentence pattern library are as follows:
[0096] ① [—— is + adjective + de], where "is", "de" are specific words, "adjective" is a specific component, "——" is other components, and "is + adjective" is the core predicate (predicative structure);
[0097] An internal combustion engine operating on the Atkinson cycle is well - known.
[0098] ② [—— is helpful for ——], where "is helpful for" is a specific word, "——" is other components, and "is helpful for" is the core predicate;
[0099] Reducing the occurrence of arteriosclerosis Helps Improving physical health;
[0100] ③ [noun (phrase) + adjective], where "noun (phrase), adjective" are specific components, and "adjective" is the core predicate (predicative structure);
[0101] The growth rate of soybeans is fast, and the plants robust , and the stress resistance is strong;
[0102] The idiomatic sentence pattern is set and pre - processed manually and stored in the idiomatic sentence pattern library.
[0103] S43. Identify and label the core verb of the sentence. First, identify the idiomatic sentence patterns and their core verbs. The core verb or core predicate of the idiomatic sentence pattern is set during preprocessing. For example, in
—— contributes to ——
[0104] For example:
[0105] ① First, identify the idiomatic sentence patterns and their core verbs (predicates);
[0106] a. Complex sentence pattern
[0107]
If ——, then ——
[0108] if Motor To rotate the idler shaft, the additional rotation will cause the idler gear
[0109] where “action” and “accelerate” are the core verbs of the front and back simple sentences respectively;
[0110] b. Idiomatic sentence pattern
noun + adjective
[0111] Soybean growth rate Plant Stress resistance
[0112] c.
—— contributes to ——
[0113] reduce The occurrence of arteriosclerosis Good physical health;
[0114] ② The intake valve in the cylinder head To allow fuel to enter the cylinder bore.
[0115] Among them, "set" is the verb at the beginning of the sentence and is not the core verb. Excluding "set", the first verb "open" from left to right is the core verb.
[0116] ③ Every day 30 - 60 mg of soy isoflavone can significantly the content of cholesterol in the blood;
[0117] Among them, "take" is preceded by the adverb "every day" and is still regarded as the verb at the beginning of the sentence and not the core verb. The first verb "reduce" after it is the core verb.
[0118] ④ <After sowing> The plastic film can soil temperature;
[0119] Among them, "cover" is preceded by the idiomatic phrase <—— after>, and is still regarded as the "verb at the beginning";
[0120] ⑤ Soybeans The cost is constantly
[0121] Among them, "produce" is the verb in the phrase "production cost" and is not the core verb;
[0122] ⑥ Farmers <on the surface of the seeds soaked after > sodium alginate adhesive; spray Among them, "soak" is the verb in the idiomatic phrase <—— after> and is not the core verb;
[0123]
[0124] ⑦ The soybeans planted grow fast;
[0125] Among them, "plant" is the verb before "of" and is not the core verb;
[0126] sports ⑧ Due to the movement of the connecting rod, the piston reciprocates in the cylinder bore ;
[0127] Among them, "movement" is the verb after "of" and is not the core verb;
[0128] S44. Identify the structure to which the word "of" belongs; group the nouns of the belonging relationship together as one noun, and distinguish other phrases with "of" as attributive phrases. When there are at least two "of" words in a phrase, and both before and after the "of" words are nouns, phrases or pronouns, parse the "of" word as the belonging relationship between the nouns, phrases or pronouns according to Chinese grammar, that is, mark the last "of" and the content before it as the attributive of the content after it. In this way, the nouns, phrases or pronouns with the said relationship are integrated and can be distinguished from other attributive phrases with "of".
[0129] For example: The thickness of the outer skin is < Decide >Important dimensions for the performance of exhaust gas purification equipment.
[0130] Among them, "the performance of the device" is an attributive that modifies the noun "important dimensions" following the last "of".
[0131] S45. Pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, auxiliary words, quantifiers, relative words and punctuation marks in sentences are parsed and annotated according to Chinese grammar; for example, adjectives are parsed as the attributives of the nouns that follow them; adverbs are parsed as the adverbials of the verbs that follow them; idiomatic phrases such as <on top of> are parsed as adverbials, etc.
[0132] For example: seed dressing Tightly bonded <In seeds surface >
[0133] Bond - the core verb; seed dressing agent - noun, subject; only - adverb, adverbial modifier modifying "bond"; <on - the surface> - an idiomatic phrase, adverbial modifier of the verb "bond".
[0134] S46. Determine and mark the subject, predicate and object of the sentence. For general sentence patterns, the core verb is the predicate of the sentence. Exclude function words (adverbs, prepositions, conjunctions, auxiliary words, interjections and onomatopoeia), relative words, phrases and clauses that serve as attributes or adverbials. Mark the component on the left side of the core verb as the subject of the sentence, and mark the component on the right side of the core verb as the object of the sentence.
[0135] For example: Piston < at top dead center and end of intake stroke between >The distance traveled is the length of the intake stroke; according to the core verb identification rules, "travel" before "of" is not a core verb, and the verb in the idiomatic phrase <in between ——> is not a core verb. The first verb "is" from the left is the core verb of the sentence; and the nouns "piston" and "distance" on the left side of the core verb are the subjects of the sentence, and the "intake stroke length" on the right side of the core verb is the object of the sentence; "intake stroke" is an idiomatic phrase, and the length is a noun marked by part of speech, all marked as objects.
[0136] The same applies to the analysis of clauses or sentences.
[0137] For example: Another benefit is that <the device can change the compression ratio of the internal combustion engine>;
[0138] Among them, after the core verb "is", the noun "this device" and the verb "change" constitute the basic elements of the sentence, so it is judged as a clause, and "change" is the core verb of the clause.
[0139] The following is a further example:
[0140] ① The present invention Involved An Located in intake passage of an internal combustion engine;
[0141] relates to - core verb,
[0142] - provided in - an adjectival phrase with "的", serving as the attributive of "passage",
[0143] The present invention - an idiomatic phrase, serving as the subject,
[0144] passage - noun, serving as the object,
[0145] A - a quantifier serving as an attributive.
[0146] ② The phase modulation device <is composed of the engine block> support <between the crankshaft and the control shaft between >
[0147] <composed of - a single - node idiomatic phrase, serving as an adverbial of manner,
[0148] <between - and - a double - node idiomatic phrase, serving as an adverbial of location,
[0149] "supported" - core verb.
[0150] The phase modulation device - an idiomatic phrase, serving as the subject.
[0151] ③ reduce The occurrence of arteriosclerosis Helps improve physical health;
[0152]
- contributes to -
[0153] Contributes to - core predicate;
[0154] Reducing the occurrence of arteriosclerosis - a "verb + noun" phrase, serving as the subject;
[0155] Improving physical health - a "verb + noun" phrase, serving as the object.
[0156] According to the result of step S2 - 4, the syntactic analysis result can be output as needed; for example: output the whole - sentence analysis result in the original sentence order; or output the subject part, the core verb, and the object part, etc. in the original sentence order.
[0157] The following compares the analysis levels and effects between the present invention and the existing syntactic analysis technology.
[0158] (SBV - subject - verb, VOB - verb - object, ADV - adverbial - center structure, RAD - right - attachment relationship, ATT - attributive - center relationship)
[0159] (1) Analyze the sentence "The content quality of Zhihu is very good."
[0160] Using the existing technology, the parsing hierarchy and process are cumbersome and complex, as Figure 2 shown.
[0161] Using the method of the present invention, according to the conventional sentence pattern [noun + adjective], where the "adjective" is the core predicate (predicative structure), it is very easy to analyze that:
[0162] Content quality - subject,
[0163] Of Zhihu - adjective phrase, used as an attributive,
[0164] Very good - core predicate (predicative structure).
[0165] (2) Analyze the sentence "How to increase the cash withdrawal limit."
[0166] Using the existing technology for parsing, the result is incorrect. As Figure 3 shown, where the relationship between "cash withdrawal" and "limit" should be "ATT", and the relationship between "increase" and "limit" should be "VOB".
[0167] Using the method of the present invention: According to the conventional sentence pattern [how + verb --], where the "verb" is the core verb and the "--" is a noun or a "verb + noun" phrase; it is very easy to analyze that:
[0168] How - interrogative word;
[0169] Increase - core verb;
[0170] Cash withdrawal limit - "verb + noun" phrase, used as the object.
[0171] (3) Analyze the sentence "Check the brakes of the car."
[0172] Using the existing technology for parsing, the result is incorrect; as Figure 4 shown, where "car" should modify "brakes", and the relationship between "check" and "car" should be VOB.
[0173] Using the method of the present invention: Parse according to the imperative sentence (in a specific context, such as operation steps, etc.) or clause, and it is very easy to analyze that:
[0174] Check - core verb;
[0175] Of the car - "de" character adjective phrase (possessive relationship), used as an attributive;
[0176] Brakes - object.
[0177] According to another aspect of the present invention, a Chinese syntactic analysis system is provided, as Figure 5 shown, including: a word recognition module, a phrase recognition module, a sentence recognition module, a syntactic analysis module, and a database.
[0178] The database includes a word library (e.g., a word and phrase library), a phrase library (e.g., a conventional phrase node library), and a sentence library (e.g., a conventional sentence pattern library).
[0179] The word recognition module is used to recognize and label words and phrases.
[0180] The phrase recognition module is used to recognize and label phrases.
[0181] The sentence recognition module is used to recognize and label conventional sentence patterns and the core verb of a sentence.
[0182] The syntactic analysis module is used to perform syntactic analysis on a sentence, label the grammatical components of the sentence components, and output the syntactic analysis result.
[0183] In the word recognition module, the words and phrases in each clause are matched with the words and phrases in the word library and labeled. The phrases include conventional phrases and general phrases. The acquisition method of the phrases is as described above. In the phrase recognition module, the phrases are divided into conventional phrases and general phrases. Each clause is compared with the node library to find whether there is a single node or a double node. The content between the two closest double nodes from left to right in a sentence forms a conventional double-node phrase; one or several other conventional double-node phrases may be included in a double-node conventional phrase, which is called a nested phrase. The nested phrases can share a node, and the nested phrase takes the double node with the longest distance and the content between them as the double-node conventional phrase; general phrases are recognized, and the general phrases refer to phrases composed of a verb + a noun, or a "de" character adjective phrase, where before the "de" character is a noun, a pronoun, an adjective, or a "verb + noun"; if there is a single node in front, the part from this node to the "verb + noun", or a verb, or a phrase or a noun that is not in the "de" character adjective phrase and the double-node conventional phrase immediately following is labeled as a single-node conventional phrase. If there is a single node at the back, the part from this node to the "verb + noun", or a verb, or a phrase or a noun immediately before it is labeled as a single-node conventional phrase.
[0184] In the sentence recognition module, the following method is used for recognition and labeling.
[0185] S41. Compare with the conventional sentence pattern library to determine whether the sentence is a compound sentence.
[0186] S42. According to the conventional sentence pattern library, recognize and label whether the main clause or the subordinate clause is a general sentence pattern or a conventional sentence pattern. When judging, if the sentence is not a conventional sentence pattern, it is a general sentence pattern.
[0187] S43. Identify and label the core verb of the sentence. First, identify the idiomatic sentence patterns and their core verbs. The core verbs (core predicates) of the idiomatic sentence patterns are set together during preprocessing. Then, identify the core verbs of general sentences as follows: 1) Find the verbs in the sentence that cannot be core verbs. The judgment method is as follows: The verb at the beginning of the main clause is not the core verb. The verb at the beginning of the sentence includes the verb with a phrase, adverb, preposition, adjective, or numeral before it; the verb in a phrase is not the core verb; the verb in an adjective phrase is not the core verb; the verb in an idiomatic phrase with a non-verb node is not the core verb; the verb before / after "的" is not the core verb; 2) Except for the verbs that "cannot be core verbs", the first verb from left to right in the main clause or sub-clause is used as the core verb of the sentence. A sentence (main clause or sub-clause) can only have one core verb, and other verbs are phrasal verbs or verbs in clauses.
[0188] S44. Identify the structure to which "的" belongs; group the nouns of the belonging relationship together as one noun, and distinguish other attributive phrases with "的". When there are two or more "的" characters in a phrase, and both before and after the two or more "的" characters are nouns, phrases, or pronouns, parse the "的" character as the belonging relationship between the nouns, phrases, or pronouns according to Chinese grammar, that is, label the last "的" and the content before it as the attributive of the content after it. In this way, the nouns, phrases, or pronouns with a belonging relationship are integrated and can be distinguished from other attributive phrases with "的".
[0189] S45. Analyze and label pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, particles, numerals, relative words, and punctuation marks in the sentence according to Chinese grammar.
[0190] S46. Determine and label the subject, predicate, and object of the sentence. Specifically: For general sentence patterns, the core verb is the predicate of the sentence. Excluding function words (adverbs, prepositions, conjunctions, particles, interjections, and onomatopoeias), relative words, attributive or adverbial phrases, and clauses, label the components on the left side of the core verb as the subject of the sentence, and label the components on the right side of the core verb as the object of the sentence; The judgment of the subject, predicate, and object of the idiomatic sentence pattern is based on manual setting.
[0191] During the parsing process, when identifying words or phrases that are not in the preprocessed word library, especially phrases containing specific characters or words, such as noun + device / apparatus, the system stores these "new words" in the word library for future use.
[0192] In one embodiment, the steps of word segmentation and part-of-speech tagging (commas and periods are used for natural sentence division) can be omitted, and pre-processed words / phrases, idiomatic phrase node libraries, and idiomatic sentence pattern libraries can be directly compared with the document to be parsed to identify words and phrases, idiomatic phrases and idiomatic sentence patterns, and then a verb table (existing technology tool) and the recognition rules of the core verbs of the present invention can be used to quickly parse the sentences. This is particularly suitable for situations where large quantities of documents need to be parsed.
[0193] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.
Claims
1. A Chinese syntactic analysis method, characterized in that: It includes the following steps: Step S1: Split the input text into sentences, segment the words, and label the parts of speech; Step S2: Identify the phrases in the sentence; Step S3: Identify the phrases in the sentence; Step S4: Identify the structure of the sentence; Step S5: Conduct syntactic analysis on the sentence, and label and output the grammatical relationships of the sentence components according to Chinese grammar; The said Step S4 includes: S41. Compare with the idiomatic sentence pattern library, and judge whether the sentence is a compound sentence. If there is a compound word, the content after the compound word is regarded as a single sentence to be analyzed; S42. Compare with the idiomatic sentence pattern library, identify and label whether the main clause or the subordinate clause is a general sentence pattern or an idiomatic sentence pattern. When judging, if the sentence is not an idiomatic sentence pattern, it is a general sentence pattern; S43. Identify and label the core verb of the sentence; S44. Analyze and label the grammatical components or relationships of pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, auxiliary words, numeral-classifiers, relative words, and punctuation marks in the sentence according to Chinese grammar; S45. Determine and label the subject, predicate, and object of the sentence. For a general sentence pattern, the core verb is the predicate of the sentence. Excluding function words, relative words, phrases used as attributives or adverbials, and clauses, label the components on the left side of the core verb as the subject of the sentence, and label the components on the right side of the core verb as the object of the sentence; The judgment of the subject, predicate, and object of the idiomatic sentence pattern is based on manual setting; The said Step S43 includes: 1) Find the verbs that cannot be used as the core verb in the general sentence pattern. The judgment method is as follows: The verb at the beginning of the main clause is not used as the core verb. The verb at the beginning of the sentence includes the verb with a phrase, adverb, preposition, adjective, or numeral-classifier before it; The verb in the phrase is not used as the core verb; The verb in the adjective phrase is not used as the core verb; The verb in the idiomatic phrase with a non-verb node is not used as the core verb; The verb before and after the character "的" is not used as the core verb; 2) Except for the verbs that are not used as the core verb, label the first verb from left to right in the main clause or the subordinate clause as the core verb of the sentence.
2. The method according to claim 1, characterized in that In the said Step S2, the phrases include idiomatic phrases and general phrases. The idiomatic phrases refer to the idiomatic expressions in the form of phrases and the idiomatic expressions in the form of phrases containing specific characters or nouns. The general phrases refer to combined nouns.
3. The method according to claim 1, characterized in that In the said Step S3, the phrases include idiomatic phrases and general phrases. The idiomatic phrases refer to the idiomatic expressions in the form of phrases with the characters, words, or their collocation words in the phrase node library as nodes, which consist of nodes and components. The components include nouns, phrases, verbs, adjectives, adverbs, or clauses.
4. The method according to claim 3, characterized in that The said Step 3 includes: S31. Compare each subordinate clause with the phrase node library from left to right to identify single nodes or double nodes; S32. Label the content between the closest double node and the said double node from left to right in the sentence as a double-node idiomatic phrase; S33. Identify general phrases. The general phrases refer to the phrases composed of verbs + nouns and the "的" character adjective phrases, where before the "的" character are nouns, pronouns, adjectives, or "noun + verb"; S34. If a pre-single node is recognized, the part from the pre-single node to the verb, phrase or noun immediately following it that is not in an adjective phrase or a double-node idiomatic phrase is marked as a single-node idiomatic phrase. If a post-single node is recognized, the part from the post-single node to the verb, phrase or noun immediately preceding it is marked as a single-node idiomatic phrase.
5. The method according to claim 2, characterized in that: An idiomatic phrase includes a specific character or noun preset during the sentence parsing process, plus the first noun, verb or phrase before it.
6. The method according to claim 1, characterized in that After the step S43, the following steps are further included: Identify the structure to which the character "的” belongs. When there are more than two "的” characters in a clause, and both the front and back of the "的” character are nouns, phrases or pronouns, parse the "的” character as the possessive relationship between the nouns, phrases or pronouns according to Chinese grammar, and use the last "的” character and the content before it as the attributive of the content after it.
7. A Chinese syntactic analysis system, characterized in that: The system includes a word recognition module, a phrase recognition module, a sentence recognition module, a syntactic analysis module and a database. Among them, The database includes a word library, a single-node idiomatic phrase node library and an idiomatic sentence pattern library; The word recognition module is used to recognize words, including nouns and phrases; The phrase recognition module is used to recognize phrases, including idiomatic phrases and general phrases; The sentence recognition module is used to recognize the main structure of a sentence, including general sentence patterns and idiomatic sentence patterns; The syntactic analysis module is used to perform syntactic analysis on a sentence, mark the grammatical components of the sentence components and output the syntactic analysis results, including the applied grammatical rules. The sentence components include words and phrases, core verbs, adjectives, adverbs, conjunctions and quantifiers; In the sentence recognition module, the steps of recognition and marking include: S41. Compare with the idiomatic sentence pattern library to determine whether the sentence is a compound sentence. If there is a compound word, the content after the compound word is used as a single sentence to be analyzed; S42. Identify and mark whether the main clause or the subordinate clause is a general sentence pattern or an idiomatic sentence pattern according to the idiomatic sentence pattern library. When judging, if the sentence is not an idiomatic sentence pattern, it is a general sentence pattern; S43. Identify and mark the core verb of the sentence; S44. Parse and mark the pronouns, phrases, adjectives, adverbs, prepositions, conjunctions, auxiliary words, quantifiers, relative words and punctuation marks in the sentence according to Chinese grammar; S45. Determine and mark the subject, predicate and object of the sentence. Specifically: for a general sentence pattern, the core verb is the predicate of the sentence. Excluding function words, relative words, phrases used as attributives or adverbials, and clauses, the components on the left side of the core verb are marked as the subject of the sentence, and the components on the right side of the core verb are marked as the object of the sentence. The judgment of the subject, predicate and object of an idiomatic sentence pattern is based on manual setting; The step S43 includes: 1) Find the verbs in the general sentence pattern that cannot be used as core verbs. The judgment method is as follows: the verb at the beginning of the main clause is not used as the core verb. The verb at the beginning of the clause includes the verb with a phrase, adverb, preposition, adjective or quantifier before it; the verb in a phrase is not used as the core verb; the verb in an adjective phrase is not used as the core verb; the verb in a single-node idiomatic phrase with a non-verb node is not used as the core verb; the verb before and after the character "的” is not used as the core verb; 2) Except for the verbs that are not used as the core verb, mark the first verb from left to right in the main clause or subordinate clause as the core verb of that sentence.
8. The system according to claim 7, characterized in that In the phrase recognition module, compare the subordinate clause with the node library to find whether single nodes or double nodes exist; mark the double node that is closest to each other from left to right in the sentence and the content between them as a double-node idiomatic phrase; one or more other double-node idiomatic phrases are included in a double-node idiomatic phrase, which is called a nested phrase and is marked. Identify general phrases, where the general phrases refer to phrases composed of a verb + a noun and "de" character adjective phrases. Among them, before the "de" character are a noun, a pronoun, an adjective, or a "verb + noun"; if there is a single node in the front, mark the part from this node to the verb, phrase, or noun that is not in the adjective phrase and the idiomatic phrase immediately following it as a single-node idiomatic phrase. If there is a single node in the back, mark the part from this node to the verb, phrase, or noun immediately before it as a single-node idiomatic phrase.
9. The system according to claim 7, characterized in that In the sentence recognition module, after step S43, it also includes identifying the structure to which the "de" character belongs. When there are more than two "de" characters in the subordinate clause and both the front and back of the "de" character are nouns, phrases, or pronouns, parse the "de" character as the belonging relationship between the nouns, phrases, or pronouns according to Chinese grammar, and use the last "de" character and the content before it as the attributive of the content after it.
Citation Information
Patent Citations
Chinese corpus labeling method based on events
CN101782897A
System and method for converting Chinese phrase structure tree banks into interdependent structure tree banks
CN103020148A
Event-structure-based Chinese statement analysis method
CN103268311A
Method for identifying emotion key sentence on basis of lexical semantics and syntactic dependency
CN104281645A
Rule based automatic Chinese syntax analysis method
CN105320644A
