A method for optimizing compliance checking of requirement templates based on part-of-speech tagging and chunk analysis

Part-of-speech annotation and chunking analysis are carried out through the PCFG method to identify modeling elements in natural language requirements, solving the problem of low template compliance inspection accuracy and recall in the existing technology, and achieving higher requirements template compliance inspection accuracy and recall rate.

CN112101019BActive Publication Date: 2024-11-29NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202010804752.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-12
Publication Date
2024-11-29
Estimated Expiration
2040-08-12

AI Technical Summary

Technical Problem

In the prior art, part-of-speech annotation and chunk analysis methods are difficult to accurately identify modeling elements in natural language requirements, resulting in low accuracy and recall of template compliance inspection, especially when dealing with non-limiting natural language requirements. Poor performance.

Method used

Using PCFG-based part-of-speech annotation and chunk analysis methods, the BNF paradigm of the requirement template is constructed, combined with manual and automatic annotation results, and the probability context-independent grammar is used to calculate the grammatical analysis tree, identify modeling elements and judge whether the requirement meets the template.

Benefits of technology

The accuracy, recall and F2 measurement of the compliance check of the requirements template are improved. The simulation results show that the F2 measurement reaches 1 on some data sets, which significantly improves the accuracy of automated judgments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112101019B_ABST
    Figure CN112101019B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for optimizing compliance checking of requirement templates based on part-of-speech tagging and chunk analysis, which includes constructing a requirement template, determining the BNF paradigm of the requirement template, obtaining requirements, and manually tagging whether the requirements meet the requirement template, performing part-of-speech tagging and chunk analysis on each requirement according to the requirement template, and then automatically tagging whether the requirements meet the requirement template, and comparing the result of manual tagging with the result of automatic tagging to obtain the precision, recall rate and F2 metric of the requirement template compliance checking of the requirement document. By applying the part-of-speech tagging and chunk analysis methods to the requirement template compliance checking, the precision and recall rate of the requirement template compliance checking are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a requirement template conformity checking and optimization method based on part-of-speech tagging and chunk analysis. Background Art

[0002] Software requirements are usually described in natural language, which is generally easy to understand and applicable to the description of requirements in any field. However, natural language may be ambiguous if it is not restricted, and it is generally difficult to analyze it automatically. Requirement templates provide an effective tool to reduce ambiguity in natural language and automatically analyze natural language requirements. Requirement templates define the format of natural language requirements. For example, the Rupp template includes the following parts: (1) starting conditions; (2) system name; (3) modal verbs; (4) system functions; (5) objects; (6) detailed information of objects. When using templates in software requirements, it is particularly important to determine whether a natural language requirement conforms to the template. If the template conformity judgment of natural language requirements is manually performed, it is very time-consuming. In this case, how an automated tool can extract modeling elements in natural language requirements through part-of-speech tagging and chunk analysis, and perform template conformity checks to determine whether software requirements conform to the template and improve its precision and recall rate has become an important technical issue.

[0003] The simplest way to do part-of-speech tagging is to use the high-frequency part-of-speech corresponding to each word in the vocabulary as the default part-of-speech, but there is obviously room for choice. At present, the mainstream word segmentation method is to treat the part-of-speech tagging of sentences as a sequence tagging problem and apply it to the part-of-speech tagging task. However, since each English word has a different part-of-speech in different contexts, this will also affect the results of chunk analysis. The general method of chunk analysis is to combine the parts of speech according to certain grammatical rules. However, simply combining words according to their parts of speech does not take into account the relationship between the context. Therefore, it is necessary to reconsider the methods of part-of-speech tagging and chunk analysis. The existing part-of-speech tagging and chunk analysis methods are not applicable to a large number of non-restrictive natural language needs, because they require more accurate part-of-speech tagging and chunk analysis methods to analyze.

[0004] In this regard, there have been relevant studies abroad. When checking the template conformity of the requirement document, part-of-speech tagging and chunk analysis are particularly important. Chetan Arora et al. used Stanford and OpenNLP part-of-speech tagging tools, MUNPEX, ANNIE and OpenNLP noun chunk analysis tools, and ANNIE verb chunk analysis tools to perform part-of-speech tagging and chunk analysis on the requirement text and make corresponding marks. Then, based on these marks, modeling elements such as conditions, system names, modal verbs, functions, objects, detailed information of objects, etc. can be extracted from natural language requirements, and whether they conform to the template can be judged based on the modeling elements. However, in past studies, part-of-speech tagging and chunk analysis were rule-based, and it was difficult to accurately determine the part of speech and divide the chunks, and the structure of the entire sentence was not analyzed. Therefore, in response to these problems, a new method is needed to perform part-of-speech tagging and chunk analysis on the requirement document to improve the accuracy of modeling element extraction, thereby improving the precision, recall rate and F2 metric of the requirement template conformity check. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and to provide a requirement template conformity check optimization method based on part-of-speech tagging and chunk analysis. By applying the part-of-speech tagging and chunk analysis methods to the requirement template conformity check, the accuracy and recall rate of the requirement template conformity check are improved.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0007] A method for optimizing compliance checking of a requirement template based on part-of-speech tagging and chunk analysis, comprising:

[0008] Step 1: Build a requirement template and determine the BNF (Backus Normal Form) of the requirement template;

[0009] Step 2: Obtain requirements and manually mark whether the requirements meet the requirement template;

[0010] Step 3: According to the requirement template, perform part-of-speech tagging and chunk analysis on each requirement, and then automatically mark whether the requirement meets the requirement template;

[0011] Step 4: Compare the results of manual annotation with those of automatic annotation to obtain the precision, recall rate and F2 metric of the requirement template conformity check for the requirement document.

[0012] To optimize the above technical solutions, the specific measures taken also include:

[0013] The requirement template constructed in step 1 above is:

[0014] <conditional-keyword> <sequence-of-tokens> <np> <vp-starting-with-modal> <np> <opt-details>;

[0015] in, <conditional-keyword>Conditional keywords include the following eight words and phrases: if, after, before, as soonas, when, while, in case, and at;

[0016] <sequence-of-tokens>A token sequence representing a condition;

[0017] <np>It indicates a noun phrase;

[0018] <vp-starting-with-modal>It indicates a verb phrase beginning with a modal verb;

[0019] <opt-details>Represents detailed information of an object.

[0020] The above step 2 is specifically as follows:

[0021] Obtain the requirements and manually check the agreement based on the template compliance. Manually check and mark whether the requirements comply with the requirement template. If the requirements comply with the requirement template, mark it as N. If the requirements do not comply with the requirement template, mark it as P.

[0022] The above template compliance manual check protocol is:

[0023] R is the requirement to be checked for compliance with requirement template T;

[0024] Verify that R is a grammatically correct sentence, without regard to punctuation when determining correctness;

[0025] Verify that R uses acceptable modal verbs;

[0026] If R is conditional, the verification condition only appears at the beginning of requirement R, and the condition conforms to the structure described by T;

[0027] verify<system name> , <object>,as well as <whom>filled by noun phrases;

[0028] verify <process>Filled with verb phrases;

[0029] If all the above conditions are met, then R meets T;

[0030] Otherwise, R does not conform to T.

[0031] The above step three is specifically as follows:

[0032] According to the requirement template, the PCFG (Probabilistic Context Free Grammar) method is used to perform part-of-speech tagging and chunk analysis, identify modeling elements, and determine the relationship between modeling elements, and then automatically check and automatically mark whether the requirement conforms to the requirement template. When the requirement conforms to the requirement template, it is marked as N, and when it does not conform to the requirement template, it is marked as P.

[0033] The above PCFG method is specifically as follows:

[0034] Given a context-free grammar G and a sentence S, calculate the probability of the parse tree P(T|S,G) by multiplying the probability of each node in the tree generating its branch node;

[0035] If a sentence has multiple grammatical analysis trees, all grammatical analysis trees are sorted according to the probability value;

[0036] PCFG uses syntactic disambiguation and selects the grammatical analysis tree with the largest probability value when faced with multiple analysis results, namely argmax T P(T|S,G).

[0037] In the above step 4, the calculation formulas for precision, recall and F2-measure are:

[0038]

[0039]

[0040]

[0041] Among them, it is assumed that the template that meets the requirements is a negative example (N), and the template that does not meet the requirements is a positive example (P);

[0042] True positive (TP) = actually positive (P), marked as positive (P) in the system;

[0043] True negative (FN) = actually a positive example (P), but marked as a negative example (N) in the system;

[0044] False positive (FP) = actually a negative example (N), but marked as a positive example (P) in the system.

[0045] The above step 4 is specifically as follows:

[0046] The automatically annotated requirement template conformance check results obtained in step 3 are stored in the map in the form of (key, value), where the key represents each input requirement and the value represents the annotated result of the requirement (N or P);

[0047] Set TP, FP, and FN to 0, read the manually annotated requirement template conformance check results obtained in step 2 line by line, split each line with an English period (.), obtain a requirement text and the requirement annotation result label (N or P), and set label1 to map.get(text);

[0048] Then compare label and label1. If both are P, TP increases by 1. If label is P and label1 is N, FN increases by 1. If label is N and label1 is P, FP increases by 1.

[0049] Finally, calculate Precision, Recall, and F2-measure.

[0050] The present invention has the following beneficial effects:

[0051] By building the requirement module and grammar tree analysis, the requirement document is tagged with part of speech and analyzed in chunks, and then the modeling elements in the requirement document are extracted, and the relationship between the modeling elements is analyzed, so as to perform the requirement template conformity check, improve the precision and recall rate of the template conformity check, and thus improve its F2 metric. The simulation results show that this method can improve the precision and recall rate, and the F2 metric is 1 on one of the data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of the present invention;

[0053] Figure 2 This is a schematic diagram of the requirement template compliance check result;

[0054] Figure 3 Select the TemplateConformance tab for the example and highlight the requirements;

[0055] Figure 4 In the example, select the TemplateNonConformance tag and highlight the requirement;

[0056] Figure 5 Configure a flow chart for the pipeline of the block analysis part;

[0057] Figure 6 Configure a flow chart for the pipeline of requirement template compliance checking. DETAILED DESCRIPTION

[0058] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0059] The present invention proposes a method for optimizing the conformance check of requirement templates based on part-of-speech tagging and chunk analysis, which can be applied to the description of requirements in the fields of "tactical control systems" and "patient transfer management". According to the above-mentioned field requirement templates, the requirements of the two fields are checked and automatically judged whether they conform to the requirement templates. Since the requirement template needs to contain conditions, system names, processes, and objects, the conditions are unnecessary elements and can only appear at the beginning of a sentence, and the remaining elements are necessary elements, and the modeling elements satisfy a certain relationship, that is, the arrangement order is conditions, system names, processes, and objects.

[0060] Therefore, before identifying the use case elements, the requirements document must be tagged with parts of speech and analyzed by chunks. Part of speech tagging is to mark the part of speech of each token that appears in the requirements document, and chunk analysis is to divide noun chunks and verb chunks according to the part of speech. Before part of speech tagging, the requirements document needs to be segmented and divided into sentences. Word segmentation refers to dividing the requirements document into several tokens, and sentence segmentation refers to dividing the requirements document into several sentences. Based on the results of word segmentation, sentence segmentation, part of speech tagging, and chunk analysis, modeling elements are identified, among which the most important are conditions, system names, processes, objects, and details, so that the system can automatically determine whether the requirements document conforms to the template. Compare the results of the system's automatic recognition with the actual results, calculate the precision and recall rate, and thus obtain an optimal method to obtain the highest precision and recall rate.

[0061] A method for optimizing compliance checking of demand templates based on part-of-speech tagging and chunk analysis of the present invention comprises:

[0062] Step 1: Build a requirement template and determine the BNF paradigm of the requirement template;

[0063] Step 2: Obtain requirements and manually mark whether the requirements meet the requirement template;

[0064] Step 3: According to the requirement template, perform part-of-speech tagging and chunk analysis on each requirement, and then automatically mark whether the requirement meets the requirement template;

[0065] Step 4: Compare the results of manual annotation with those of automatic annotation to obtain the precision, recall rate and F2 metric of the requirement template conformity check for the requirement document.

[0066] In the embodiment, the Rupp template constructed in step 1 is:

[0067] <conditional-keyword> <sequence-of-tokens> <np> <vp-starting-with-modal> <np> <opt-details>;

[0068] in, <conditional-keyword>Indicates conditional keywords, including the eight words and phrases if, after, before, as soonas, when, while, in case, and at. When the conditional keyword appears at the beginning of a sentence, the first letter can be capitalized;

[0069] <sequence-of-tokens>A token sequence representing a condition (excluding condition keywords);

[0070] The first one <np>It indicates the first noun phrase that comes before the modal verb phrase;

[0071] <vp-starting-with-modal>It means a verb phrase beginning with a modal verb, abbreviated as modal verb phrase;

[0072] The second <np>It indicates the noun phrase following the modal verb phrase;

[0073] <opt-details>Indicates the second <np>Then know the part at the end of the sentence.

[0074] In the embodiment, the step 2 is specifically as follows:

[0075] Obtain a certain number of requirements and manually check the agreement based on the template compliance. Manually check and mark whether the requirements comply with the requirement template. When the requirements comply with the requirement template, mark it as N. When the requirements do not comply with the requirement template, mark it as P.

[0076] In the embodiment, the template compliance manual inspection protocol is:

[0077] R is the requirement to be checked for compliance with requirement template T;

[0078] Verify that R is a grammatically correct sentence, without regard to punctuation when determining correctness;

[0079] Verify that R uses acceptable modal verbs;

[0080] If R is conditional, the verification condition only appears at the beginning of requirement R, and the condition conforms to the structure described by T;

[0081] verify<system name> , <object>,as well as <whom>filled by noun phrases;

[0082] verify <process>Filled with verb phrases;

[0083] If all the above conditions are met, then R meets T;

[0084] Otherwise, R does not conform to T.

[0085] In the embodiment, in step three, OpenNLP Parser is used to perform syntax tree analysis, and OpenNLP Parser uses the PCFG algorithm to perform syntactic analysis.

[0086] First, define a quintuple (X, V, S, R, P);

[0087] Where X represents the terminal symbol set; V represents the non-terminal symbol set; S represents the start symbol of the grammar, which is included in V, that is, S∈V; R is the set of ordered pairs (α, β), that is, the generated rule set; P represents the statistical probability of each generated rule.

[0088] The PCFG is written as follows:

[0089] Form: A→α,P

[0090] Constraints: ∑ α P(A→α)

[0091] Given a sentence S, get n syntactic trees, calculate the probability of two syntactic trees P(T1), P(T2)...P(Tn), and select the syntactic tree with the largest probability value argmax T P(T|S,G), as the result of syntactic analysis.

[0092] According to the above selected syntactic analysis results, the parts of speech, noun blocks and verb blocks are extracted.

[0093] Step three is as follows:

[0094] According to the requirement template, the PCFG method is used to perform part-of-speech tagging and chunk analysis, identify modeling elements, and determine the relationship between modeling elements, and then automatically check and automatically mark whether the requirement conforms to the requirement template. When the requirement conforms to the requirement template, it is marked as N, and when the requirement does not conform to the requirement template, it is marked as P.

[0095] In the embodiment, the PCFG method is specifically as follows:

[0096] Given a context-free grammar G and a sentence S, calculate the probability of the parse tree P(T|S,G) by multiplying the probability of each node in the tree generating its branch node;

[0097] If a sentence has multiple grammatical analysis trees, all grammatical analysis trees are sorted according to the probability value;

[0098] PCFG uses syntactic disambiguation and selects the grammatical analysis tree with the largest probability value when faced with multiple analysis results, namely argmax T P(T|S,G).

[0099] According to the requirement template, part-of-speech tagging, and chunk analysis results, the modeling elements are identified, and according to the relationship between the modeling elements, it is determined whether each requirement in the requirement document conforms to the requirement template. Figure 2 As shown in the figure, on the right are some tags obtained by natural language processing of the requirement document. Check the checkbox in front of the tag to highlight the requirements with the tag in the requirement text. For example, if you select the TemplateComformance tag, the highlighted requirements are as follows: Figure 3 As shown; select the TemplateNonConformance tab, the highlighted requirements are as follows Figure 4 As shown.

[0100] In the embodiment, the step 4 is specifically as follows:

[0101] The automatically annotated requirement template conformance check results obtained in step 3 are stored in the map in the form of (key, value), where the key represents each input requirement and the value represents the annotated result of the requirement (N or P);

[0102] Set TP, FP, and FN to 0, read the manually annotated requirement template conformance check results obtained in step 2 line by line, split each line with an English period (.), obtain a requirement text and the requirement annotation result label (N or P), and set label1 to map.get(text);

[0103] Then compare label and label1. If both are P, TP increases by 1. If label is P and label1 is N, FN increases by 1. If label is N and label1 is P, FP increases by 1.

[0104] Finally, calculate Precision, Recall, and F2-measure.

[0105] like Figure 5 As shown, the method of the present invention improves the pipeline configuration in the chunk analysis, and has improvements in both precision and recall, thereby improving the f2 metric. However, there are still problems, and manual judgment and correction are still required. In order to verify the effect of the improvement of the method in this article, comparative experiments were carried out on the tcs dataset and the BILT dataset, respectively. The datasets have 202 and 45 requirements, respectively. The original method is to use the part-of-speech tagging method of OpenNLP. There are three improved methods. All three methods use OpenNLP Parser for grammatical analysis. The first method extracts the parts of speech in the requirement sentence; the second method extracts the noun chunks in the requirement sentence; and the third method extracts the parts of speech and noun chunks in the requirement sentence. Then Figure 5 The annotated requirement document is checked for template compliance. The process is as follows Figure 6 The template consistency check results obtained by the three methods are compared with the results of manual annotation. This paper uses precision, recall and F2-measure indicators to evaluate the effect of template consistency check, which are defined as follows:

[0106] Assume that the example that meets the requirement template is a negative example (N), and the example that does not meet the requirement template is a positive example (P);

[0107] True positive (TP) = actually positive (P), marked as positive (P) in the system;

[0108] True negative (FN) = actually a positive example (P), but marked as a negative example (N) in the system;

[0109] False positive (FP) = actually a negative example (N), but marked as a positive example (P) in the system;

[0110]

[0111]

[0112]

[0113] Taking the requirement document of the tcs system as an example, a compliance check is performed on it. It can be seen that the present invention can more correctly judge whether the document conforms to the requirement template, and the result is more ideal.

[0114] The experimental results are shown in Table 1:

[0115] Table 1 Comparison results of the two methods

[0116]

[0117] By analyzing the experimental results, it can be found that the improved method has improved precision and recall rate compared with the original method, and the recall rate is more satisfactory than the precision. The average values ​​of recall rate are 0.96 and 1 respectively, which shows that in the tcs requirement document, most of the requirements that do not meet the specifications can be identified. The average values ​​of precision are 0.81 and 1 respectively, which shows that in the tcs requirement document, a large number of requirements that meet the specifications are identified as non-compliant with the specifications. The rules need to be further improved and the block analysis part needs to be further improved. The reasons for the low precision are mainly the following:

[0118] 1) There may be some errors in part-of-speech tagging.

[0119] 2) The verb chunking function is not perfect and does not meet all needs.

[0120] 3) The extraction rules are not perfect, and some parts of the requirement text that conform to the template cannot be effectively extracted.

[0121] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.< / process> < / whom> < / object> < / np> < / np> < / np> < / np> < / vp-starting-with-modal> < / np> < / sequence-of-tokens> < / conditional-keyword> < / process> < / whom> < / object> < / np> < / np> < / vp-starting-with-modal> < / np> < / sequence-of-tokens> < / conditional-keyword>

Claims

1. A method for optimizing compliance checking of demand templates based on part-of-speech tagging and chunk analysis, characterized in that: include: Step 1: Build a requirement template and determine the BNF paradigm of the requirement template; Step 2: Obtain requirements and manually mark whether the requirements meet the requirement template; Step 3: According to the requirement template, perform part-of-speech tagging and chunk analysis on each requirement, and then automatically mark whether the requirement meets the requirement template; Step 4: Compare the manual annotation results with the automatic annotation results to obtain the precision, recall rate and F2 metric of the requirement template conformity check on the requirement document; The step 2 is specifically as follows: Obtain requirements and manually check the agreement based on template compliance. Manually check and mark whether the requirements comply with the requirement template. If the requirements comply with the requirement template, mark it as N. If the requirements do not comply with the requirement template, mark it as P. The template compliance manual check protocol is: R is the requirement to be checked for compliance with requirement template T; Verify that R is a grammatically correct sentence, without regard to punctuation when determining correctness; Verify that R uses acceptable modal verbs; If R is conditional, the verification condition only appears at the beginning of requirement R, and the condition conforms to the structure described by T; verify<system name> , <object>,as well as <whom>filled by noun phrases; verify <process> Filled with verb phrases;< / process> If all the above conditions are met, then R meets T; Otherwise, R does not conform to T; The step three is specifically as follows: According to the requirement template, the PCFG method is used to perform part-of-speech tagging and chunk analysis, identify modeling elements, and determine the relationship between modeling elements, and then automatically check and automatically mark whether the requirement conforms to the requirement template. If the requirement conforms to the requirement template, it is marked as N, and if the requirement does not conform to the requirement template, it is marked as P; The PCFG method is specifically: Given a context-free grammar G and a sentence S, calculate the probability of the parse tree P(T|S,G) by multiplying the probability of each node in the tree generating its branch node; If a sentence has multiple grammatical analysis trees, all grammatical analysis trees are sorted according to the probability value; PCFG uses syntactic disambiguation and selects the grammatical analysis tree with the largest probability value when faced with multiple analysis results, namely argmax T P(T|S,G).

2. According to claim 1, a method for optimizing the conformance check of a requirement template based on part-of-speech tagging and chunk analysis is characterized in that: The requirement template constructed in step 1 is: <conditional-keyword> <sequence-of-tokens> <np> <vp-starting-with-modal> <np> <opt-details> ;< / opt-details> < / np> < / vp-starting-with-modal> < / np> < / sequence-of-tokens> < / conditional-keyword> in, <conditional-keyword> Conditional keywords include eight words and phrases: if, after, before, as soon as, when, while, in case, and at; <sequence-of-tokens> A token sequence representing a condition; <np> It indicates a noun phrase;< / np> <vp-starting-with-modal> It indicates a verb phrase beginning with a modal verb; <opt-details> Represents detailed information of an object.

3. The method for optimizing the conformance check of a requirement template based on part-of-speech tagging and chunk analysis according to claim 1 is characterized in that: In step 4, the calculation formulas for precision, recall and F2-measure are: Among them, it is assumed that the template that meets the requirements is a negative example (N), and the template that does not meet the requirements is a positive example (P); True positive (TP) = actually positive (P), marked as positive (P) in the system; True negative (FN) = actually a positive example (P), but marked as a negative example (N) in the system; False positive (FP) = actually a negative example (N), but marked as a positive example (P) in the system.

4. The method for optimizing the conformity check of demand templates based on part-of-speech tagging and chunk analysis according to claim 3 is characterized in that: The step 4 is specifically as follows: The automatically annotated requirement template conformity check results obtained in step 3 are stored in the map in the form of (key, value), where key represents each input requirement and value represents the annotated result N or P of the requirement; Set TP, FP, and FN to 0, read the manually annotated requirement template conformance check results obtained in step 2 line by line, split each line with an English period (.), obtain a requirement text and the requirement annotation result label N or P, and set label1 to map.get(text); Then compare label with label1. If both are P, add 1 to TP. If label is P and label1 is N, then FN increases by 1; if label is N and label1 is P, then FP increases by 1; Finally, calculate Precision, Recall, and F2-measure. < / whom> < / object>

Citation Information

Patent Citations

  • Enquiry statement analytical method and system for information retrieval

    CN101510221A

  • Realization method and system for electronic medical record post-structuring and auxiliary diagnosis

    CN106383853A