A paragraph text information processing method and system

Abstract

The present invention relates to a paragraph text information processing method and system, including: designing a paragraph text processing language PTPL; developers write paragraph text processing rules according to PTPL specifications, parse the written paragraph text processing rules, and form execution process definitions; implement execution programs corresponding to various execution process definitions according to PTPL specifications, and then execute the execution programs corresponding to the execution process definitions in turn for input test paragraph texts according to the obtained execution process definitions and return results; if the returned result is incorrect or does not meet expectations, modify the written paragraph text processing rules and continue to test the input test paragraph text until it meets user needs; add descriptive information to the paragraph text processing rules that meet user needs, and save them together as files or publish them as application interfaces. The present invention can not only realize complex paragraph text processing functions through free combination, but also has good readability of processing rules, strong interactivity, low development difficulty, and is easy to collaborate and manage.

Description

Technical Field

[0001] The invention belongs to the field of text information processing, and in particular relates to a paragraph text information processing method and system. Background Art

[0002] In the prior art, currently in the field of text information processing, people mainly use programming languages ​​such as C, C++, Java and Python to cooperate with regular expressions to complete the segmentation, filtering, conversion and information extraction tasks of paragraph text. The main problem is that it is necessary to cooperate with host programming languages ​​such as C, C++, Java and Python, which is limited by the implementation of regular expression engines within each language. However, when working in a team, each rule developer is familiar with different knowledge backgrounds, programming skills, familiar programming languages, and different development platforms, which is not friendly to business personnel in understanding the rules, and it is easy to cause the business rules with the same meaning to be difficult to communicate with each other between different personnel. The developed rules cannot be used by the team, and it is difficult to be reviewed and tested by business personnel. As the number of rules increases, it becomes difficult to maintain. Summary of the invention

[0003] In order to overcome the above-mentioned problems existing in the prior art, the present invention provides a paragraph text information processing method and system for solving the current problems in paragraph text processing, such as high programming ability requirements for developers, rules being restricted to one or several specific programming languages, tight coupling between the design and implementation of text processing rules, and weak interactivity in the development and debugging of rules.

[0004] A method for processing paragraph text information, the method comprising the following steps:

[0005] S1. Language definition: Design a paragraph text processing language PTPL, which defines the pattern matching, processing rules and rule combination methods of paragraph text;

[0006] S2. Rule parsing: The developer writes paragraph text processing rules according to the PTPL specification, and the paragraph text processing rules written by the developer are parsed according to the definition of S1 to form an execution process definition;

[0007] S3. Rule implementation: Implement the execution procedures corresponding to each execution process definition according to the PTPL specification, and then execute the execution procedures corresponding to the class execution process definition on the input test paragraph text in sequence according to the execution process definition obtained in S2 and return the results;

[0008] S4. Rule debugging step: determine whether the returned result meets the user's expectations. If the result is incorrect or not as expected, modify the paragraph text processing rules written in S2 and continue to test the input test paragraph text until it meets the user's needs.

[0009] S5. Rule deployment: Add description information to the paragraph text processing rules obtained in S4 that meet user needs, and save them together as a file or publish them as an application interface.

[0010] According to the above aspects and any possible implementation, an implementation is further provided, wherein the paragraph text processing language in step S1 is defined at the level of lexical, grammatical and semantic, specifically including:

[0011] S11. Define lexical rules: Lexical tokens include English uppercase characters UCASE_LETTER, English lowercase characters LCASE_LETTER, Chinese characters CN_ZH_LETTER, words, unmarked short strings UNESCAPED_STRINTG, marked short strings INLINE_STRING, and marked long strings MULTILINE_STRING;

[0012] S12. Define grammar rules: grammar rules include comment item COMMENT, indentation rule INDENT, rule file description DOC_INFO, text processing step STATEMENT, text processing operator OP, parameter list INLINE_ARGUMENTS and pattern definition REGEX_LONG; wherein parameter list INLINE_ARGUMENTS supports Boolean parameter UNMAED_ARGUMENT and key value parameter KEYVAL_ARGUMENT; pattern definition REGEX_LONG is implemented with regular expression; text processing step is used to define the execution order between operators, and is led by operator name OP_NAME, indicating that the operator is executed after the operator with corresponding OP_NAME;

[0013] S13. Define semantic rules, where the semantic rules include: (1) further converting the paragraph text processing rules written by the developer to form an execution process definition; (2) the execution process definition is a combination of text processing steps STEP.

[0014] According to the above aspects and any possible implementation, an implementation is further provided, wherein step S2 specifically includes:

[0015] S21. Lexical scanning step, according to the definition of S11, using regular expressions to match the characters encountered in greedy mode, matching the paragraph text processing rules written by the developer from left to right, and parsing them into a sequence of tokens; if the regular expression does not match one of the above tokens, a scanning error is reported at the current position;

[0016] S22. Grammar parsing step, writing grammar parsing rules to parse the sequence of the tags into a grammar tree; if any grammar parsing rule cannot be matched at the current lexical tag position, an exception is thrown at the current lexical tag position to prompt the user that the position matches incorrectly;

[0017] S23. Semantic processing step: If the paragraph text processing rules written by the developer can be correctly parsed, the execution process definition is generated by traversing the entire grammar tree and according to the definition rules of S13.

[0018] According to the above aspects and any possible implementation, an implementation is further provided, wherein step S3 processes the input test paragraph text in sequence according to the execution process definition parsed by S23, wherein the result output after the previous step is used as the input of the next step, and the output of the last step is used as the output of the paragraph text processing rule written by the developer, thereby forming the result of paragraph text processing. According to the definition of PTPL, the text processing process, i.e., the execution process definition, is a free combination of the following four processing methods:

[0019] (1) Text segmentation: Take one or more patterns as parameters and a paragraph text as input. For each pattern associated with the execution process, return the start and end positions of the text matched by the pattern, and segment the paragraph text into multiple sub-texts according to the mapping rules defined by other parameters.

[0020] (2) Text extraction: Using the capture grouping function, when the input sub-text matches the defined pattern, the text matching the pattern is extracted from the sub-text and output;

[0021] (3) Text filtering: This type of processing takes multiple sub-texts formed after a paragraph text is processed in several steps as input, and decides whether to retain the sub-text based on the degree of match between one or more defined positive or negative patterns and each sub-text;

[0022] (4) Text conversion: The multiple sub-texts formed after the paragraph text is processed in several steps are taken as input, matched according to the defined pattern, and the matched sub-text range is converted to replace each paragraph sub-text to form new paragraph sub-texts.

[0023] According to the above aspects and any possible implementation, an implementation is further provided, wherein step S4 specifically includes:

[0024] S41. The user writes paragraph text processing rules on the debugging interface, and the parsing area in the debugging interface outputs the currently parsed rules in real time; if the current rule definition is correct, the test interface outputs the definition of the execution process to be executed according to the logic of step S2; if the current rule definition is wrong, the corresponding error message is output until the current rule meets the user's text processing needs;

[0025] S42. The user enters some paragraph texts that the user considers representative as sample texts on the debugging interface. When the sample text changes, the debugging interface returns the result of text processing in real time according to the processing logic of step S3 and displays it on the debugging interface. In the debugging interface, the user creates and modifies paragraph text processing rules and sample texts in real time until the result displayed on the interface meets the user's expectations.

[0026] According to the above aspects and any possible implementation, an implementation is further provided, wherein step S5 specifically includes:

[0027] S51. Save the paragraph text processing rules and test paragraph text that meet the user's expectations;

[0028] S52. Add processing rule description information to explain the business context in which the paragraph text processing rule applies;

[0029] S53. Associate the saved paragraph text processing rules, the tested paragraph text and the business background description, and publish them together as a Web service backend in conjunction with the rule parsing described in S2 and the rule implementation described in S3.

[0030] According to the aspects described above and any possible implementation methods, an implementation method is further provided, wherein the execution process definition is a sequential combination of text processing steps; each generated text processing step includes three parts: processing method, processing parameters and pattern definition; the processing method is one of the tasks of paragraph text segmentation, replacement, filtering and extraction; the processing parameters correspond to the parameter list INLINE_ARGUMENTS in the above step S12; the pattern definition corresponds to one or more REGEX symbols in the above step S12; if some steps lack necessary parameters or have type errors, the user is prompted that the processing step has a semantic error.

[0031] As described above and any possible implementation method, an implementation method is further provided, wherein each STEP is decomposed into three parts: processing method, processing parameters and pattern definition; the processing method of each STEP is one of the four tasks of segmentation, replacement, filtering or extraction of paragraph text; the pattern definition contained in each STEP is implemented using a regular expression variant, a word matching rule or other matching rules for a sequence.

[0032] The present invention also provides a paragraph text information processing system, comprising: a language definition module, used to design a paragraph text processing language PTPL, the paragraph text processing language defines pattern matching, processing rules and rule combination methods for paragraph text;

[0033] A rule parsing module is used to enable developers to write paragraph text processing rules according to the PTPL specification, and parse the paragraph text processing rules defined by the developer according to the definition of the language definition module to form an execution process definition, wherein the execution process definition is composed of one or more processing steps;

[0034] The rule implementation module is used to first implement the execution programs corresponding to various execution process definitions according to the PTPL specification, and then execute the execution programs corresponding to each processing step in sequence on the input test paragraph text according to the execution process definition obtained by the rule parsing module and return the result;

[0035] A rule debugging module is used to determine whether the returned result meets the user's expectations. If the result is incorrect or does not meet the expectations, the paragraph text processing rules written in the rule parsing module are modified and the input test paragraph text is continuously tested until it meets the user's requirements;

[0036] The rule deployment module is used to add description information to the paragraph text processing rules that meet user needs obtained by the rule debugging module, save them as files or publish them as application interfaces.

[0037] The present invention also provides a computer storage medium, on which a computer program is stored. The computer program is executed by a processor to implement the paragraph text information processing method of the present invention.

[0038] Beneficial Effects of the Invention

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The paragraph text information processing method of the present invention comprises: designing a paragraph text processing language PTPL, which defines the pattern matching, processing rules and rule combination mode of the paragraph text; developers write paragraph text processing rules according to the PTPL specification, and parse the paragraph text processing rules written by the developer according to the definition to form an execution process definition; according to the PTPL specification, the execution program corresponding to each type of execution process definition is implemented, and then according to the obtained execution process definition, the execution program corresponding to the class execution process definition is executed in sequence for the input test paragraph text and the result is returned; it is judged whether the returned result meets the user's expectations, if the result is incorrect or does not meet the expectations, the written paragraph text processing rules are modified and the input test paragraph text is continuously tested until it meets the user's needs; descriptive information is added to the obtained paragraph text processing rules that meet the user's needs, and they are saved as files or published as application interfaces together. The present invention can not only realize complex paragraph text processing functions through free combination, but also has good readability of processing rules, strong interactivity, low development difficulty, and is easy to collaborate and manage. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a work flow chart of an embodiment of the present invention;

[0042] Figure 2 It is the development paragraph text processing rules and debugging interface in the embodiment of the present invention:

[0043] Figure 3 It is an interface diagram provided externally for paragraph text information processing based on a domain-specific language in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to better understand the technical solution of the present invention, the content of the present invention includes but is not limited to the specific implementation methods described below, and similar technologies and methods should be considered to be within the scope of protection of the present invention. In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0045] It should be clear that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0047] like Figure 1 The figure shows the overall framework of the present invention. The present invention is a paragraph text information processing method. The system of the present invention is not limited to the specific operating hardware and programming language, and can be completed by writing in any language. Therefore, other working modes are not described in detail.

[0048] The specific operation process is as follows:

[0049] S1. In the language definition phase, the pattern matching, processing rules and rule combination rules of paragraph text are defined lexically, grammatically and semantically to form a language design specification through the paragraph text processing language (hereinafter referred to as PTPL). Subsequent developers will develop paragraph text processing rules based on this specification.

[0050] S2, rule parsing stage, designs the grammar parser of PTPL based on LALR grammar parsing, parses the paragraph text processing rules defined by the developer, and generates a grammar parsing tree; further, according to the semantic rules of PTPL, generates the execution process definition of paragraph text processing from the grammar parsing tree, where the execution process definition is composed of one or more text processing steps.

[0051] S3, language implementation stage, first implement the execution programs corresponding to various execution process definitions according to the PTPL specification, and then execute each execution process definition program in turn on the input test paragraph text according to the execution process definition obtained in S2 and return the result.

[0052] S4, rule debugging stage. Based on the above steps, the rule developer can input the paragraph text processing rules defined in S2 and the test paragraph text to be processed on the debugging interface. The debugging interface parses the processing rules input by the developer and outputs the rule definition. The developer can observe whether the parameter composition and processing order of the processing rules meet the requirements in the debugging interface. At the same time, after the debugging interface accepts the processing rules and the test paragraph text to be processed, it returns a text processing result. The developer observes whether the processing result meets his processing expectations in the debugging interface. If the parameter composition and processing order of the processing rules do not meet the requirements or the processing results do not meet the developer's expectations, the developer can modify the processing rules or test paragraph text in real time in the interface until they meet the user's needs. The debugging interface implements dynamic modification of processing rules and test paragraph texts, and allows developers to observe the parsed processing rule definitions and paragraph text processing results in real time, which enhances the readability and interactivity of the processing rules and facilitates verification of the correctness of the text processing rules at any time;

[0053] S5, rule deployment stage, if the processing rules after debugging in S4 above meet the user requirements, some business description information is further added to the processing rules, and saved together as a file or published as an application program interface.

[0054] The present invention adopts a paragraph text rule definition language that supports Chinese prompt information, allows users to describe the purpose and purpose of paragraph text processing rules through a "description" field, realizes the combination of multiple paragraph text processing methods through marking and concatenation, adjusts the processing completed by each processing step through multiple parameter forms such as named parameters and inline parameters, and realizes interactive rule development through a real-time debugging interface. It can not only realize complex paragraph text processing functions through free combination, but also has good readability, strong interactivity, low development difficulty, and easy collaboration and management. It solves the current problems of high programming ability requirements for developers in paragraph text processing, rules being limited to one or several specific programming languages, tight coupling between the design and implementation of text processing rules, and weak interactivity in rule development and debugging.

[0055] Specifically, the step S1 specifically includes:

[0056] (1) Define lexical rules. The lexical rules of PTPL include the following lexical tags: English uppercase characters UCASE_LETTER, English lowercase letters LCASE_LETTER, Chinese characters CN_ZH_LETTER, words, untagged short strings UNESCAPED_STRINTG, tagged short strings INLINE_STRING, and tagged long strings MULTILINE_STRING. Among them, untagged short strings support direct scanning of a tag in the language without explicitly using single and double quotes to enclose it, which increases the intuitiveness of rule writing; tagged long strings implement cross-line functionality, and can identify multiple lines of text in the rule file as a single tag, which is convenient for adding comments or complex pattern definitions in the rule language. The lexical rules in this embodiment are defined as follows:

[0057] LCASE_LETTER:"a".."z"

[0058] UCASE_LETTER:"A".."Z"

[0059] CN_ZH_LETTER: / [u"\u4e00-\u9fa5"] /

[0060] LETTER:UCASE_LETTER|LCASE_LETTER|CN_ZH_LETTER

[0061] WORD:LETTER+

[0062] MULTILINE_STRING: / """([^"\\]*(\\.[^"\\]*)*)""" / / / Multi-line string

[0063] INLINE_STRING: / "([^"\\]*(\\.[^"\\]*)*)" / / / Single-line string, starts with double quotes

[0064] SINLINE_STRING: / '([^'\\]*(\\.[^'\\]*)*)' / / / Single-line string, starts with a single quote

[0065] UNESCAPED_STRING: / [^=:\n\r\f,#]+ / / / String that does not start with quotes

[0066] (2) Define grammar rules. The grammar parsing rules include the comment item COMMENT, the indentation rule INDENT, the rule file description DOC_INFO, the text processing step STATEMENT, the text processing method OP, the parameter list INLINE_ARGUMENTS and the pattern definition REGEX_LONG. The parameter list INLINE_ARGUMENTS supports the Boolean parameter UNMAED_ARGUMENT and the key value parameter KEYVAL_ARGUMENT; the pattern definition REGEX_LONG is implemented in the form of regular expressions, etc., and is a special parameter of the operator. An operator can have zero or more pattern definitions, and each pattern is implemented by various string tags (cross-line MULTILINE STRING, or INLINE_STRING); the grammar symbol STEP of the text processing step also defines the execution order between each processing step, and is guided by the processing method name OP_NAME, indicating that the step is executed after the step with the corresponding OP_NAME. The grammar rules in this embodiment are defined as follows:

[0067] ? start:_NL*[doc_info_NL]statement+ / / rule file

[0068] / / Define the rule file description information

[0069] doc_info:"Instructions"(MULTILINE_STRING|INLINE_STRING|UNESCAPED_STRING)

[0070] / / Text processing step item (operator)

[0071] statement:step

[0072] step:[var_name":"]op_name[inline_arguments]_NL[_INDENT named_arguments_DEDENT? ]

[0073] / / Parameters for text processing steps

[0074] op_name:var_name / / Operator name

[0075] named_arguments: named_argument+ / / named parameters

[0076] named_argument:_NL*var_name_NL*regex_long?

[0077] keyval_argument:var_name / =|: / string / / key-value pair parameter

[0078] unamed_argument: string

[0079] inline_arguments:(keyval_argument|unamed_argument)+ / / inline parameters

[0080] / / Mode definition item

[0081] regex_long:regex_list_NL[_INDENT regex_long+_DEDENT]

[0082] regex_list:regex+

[0083] regex:string

[0084] / / Basic grammar items

[0085] var_name:UNESCAPED_STRING / / variable name

[0086] string:MULTILINE_STRING|INLINE_STRING|SINLINE_STRING|UNESCAPED_STRING / / String

[0087] COMMENT: / #[^\n]* /

[0088] %ignore COMMENT / / Ignore comments

[0089] / / Indentation mark

[0090] %declare_INDENT_DEDENT

[0091] (3) Define semantic rules. Semantic rules include: (1) further transforming the paragraph text processing rules written by the developer to form an execution process definition; (2) the execution process definition is a combination of text processing steps (hereinafter referred to as STEPs), where each STEP is decomposed into three parts: processing method, processing parameters, and pattern definition; the processing method of each STEP is one of the four tasks of paragraph text segmentation, replacement, filtering, or extraction; the pattern definition contained in each STEP can be implemented using regular expression variants of various programming languages, word matching rules, or other matching rules for sequences.

[0092] Specifically, the step S2 specifically includes:

[0093] (1) Lexical scanning stage: Regular expressions are used to match characters encountered in greedy mode, and the paragraph text processing rules written by the developer are scanned from left to right to be parsed into a sequence of lexical tokens. If the regular expression cannot match any lexical token in S11, a scanning error is reported at the position where the match cannot be made.

[0094] (2) Grammar parsing stage: Make the grammar rules defined in step (2) of S1 conform to the grammar rules required by grammar parsers such as LALR or LR(1), and write grammar parsing rules to parse the token sequence obtained in the above steps into a grammar tree; if any grammar parsing rule cannot be matched at the current lexical token position, an exception is thrown at the current lexical token position to prompt the user that the position is matched incorrectly;

[0095] (3) In the semantic processing stage, the grammar tree generated in the previous step is traversed by using syntax-directed translation and other means to generate an execution process definition consisting of text processing steps. Each text processing step includes three parts: processing method, processing parameters and pattern definition. The processing method can be segmentation, replacement, filtering and extraction; the processing parameters correspond to the parameter list INLINE_ARGUMENTS in step (2) of S1 above; the pattern definition corresponds to the REGEX or REGEX_LONG symbol in step (2) of S1 above, and can contain one or more pattern definitions. If some processing steps lack necessary parameters or the parameter type is incorrect, the user will be prompted that the processing step has a semantic error.

[0096] Specifically, step S3 writes a function implementation consistent with the parameters and mode definition of the processing step for each type of text processing step in S1 as the execution program of the processing step. The function receives the execution process definition generated by step (3) of S2, and processes the input test paragraph text in sequence according to each step in the execution process definition, wherein the result output after the previous step is used as the input of the next step, and the output of the last step is used as the output of the paragraph text processing rule written by the developer, thereby forming the result of paragraph text processing; S3 implements the execution programs corresponding to various types of execution process definitions according to the PTPL specification, specifically including but not limited to the following types of text processing steps:

[0097] (1) Text segmentation: Take one or more patterns as parameters and a paragraph text as input. For each pattern associated with the execution process, return the start and end positions of the text matched by the pattern, and segment the paragraph text into multiple sub-texts according to the mapping rules defined by other parameters.

[0098] (2) Text extraction: Using the capture grouping function, when the input sub-text matches the defined pattern, the text matching the pattern is extracted from the sub-text and output;

[0099] (3) Text filtering: This type of processing takes multiple sub-texts formed after a paragraph text is processed in several steps as input, and decides whether to retain the sub-text based on the degree of match between one or more defined positive or negative patterns and each sub-text;

[0100] (4) Text conversion: The multiple sub-texts formed after the paragraph text is processed in several steps are taken as input, matched according to the defined pattern, and the matched sub-text range is converted to replace each paragraph sub-text to form new paragraph sub-texts.

[0101] Specifically, the step S4 is implemented as a debugging interface, such as Figure 2 As shown, the step S4 specifically includes:

[0102] (1) In the development and debugging interface of paragraph text processing rules, users interactively write processing rules in the upper left corner, and the rule parsing area in the lower left corner of the interface parses the processing rules according to S23 and S3. Whenever the parsing rules change, the debugging interface sends a request to the background API and outputs the current parsed execution process definition in real time. If the current processing rule semantics are correct, the interface will output the information of each processing step to be executed; if the current processing rule semantics are wrong, the corresponding error message will be output until the current processing rule definition text meets the user's text processing requirements;

[0103] (2) The user enters some paragraph text that the user considers representative of the test as sample text on the debugging interface. When the sample text changes, the debugging interface returns the result of the text processing in real time according to the processing logic of step S3, and displays it highlighted in the debugging interface; in the debugging interface, the user creates and modifies paragraph text processing rules and sample text in real time until the result displayed on the interface meets the user's expectations.

[0104] Specifically, the step S5 includes:

[0105] (1) In S4, if the text processing rule developed by the developer meets the expectations on the test paragraph text, the corresponding text processing rule definition and the test paragraph text are saved;

[0106] (2) Add text processing rule description information to explain the business context in which the rule applies;

[0107] (3) Associate the saved text processing rules, paragraph text and business background description, and publish them together with the rule parsing code described in S2 and the rule implementation code described in S3 as a Web service backend. When the Web service backend is started, it can read the saved rule definition and test paragraph text, and generate an API, which can accept the paragraph text input by the user and return the processing result when running. An example of an interface provided to the outside can be seen in Figure 3 .

[0108] Preferably, the present invention also provides a paragraph text information processing system, comprising: a language definition module, used to design a paragraph text processing language PTPL, the paragraph text processing language defines pattern matching, processing rules and rule combination methods for paragraph text;

[0109] A rule parsing module is used to enable developers to write paragraph text processing rules according to the PTPL specification, and parse the paragraph text processing rules defined by the developer according to the definition of the language definition module to form an execution process definition, wherein the execution process definition is composed of one or more processing steps;

[0110] The rule implementation module is used to first implement the execution programs corresponding to various execution process definitions according to the PTPL specification, and then execute the execution programs corresponding to each processing step in sequence on the input test paragraph text according to the execution process definition obtained by the rule parsing module and return the result;

[0111] A rule debugging module is used to determine whether the returned result meets the user's expectations. If the result is incorrect or does not meet the expectations, the paragraph text processing rules written in the rule parsing module are modified and the input test paragraph text is continuously tested until it meets the user's requirements;

[0112] The rule deployment module is used to add description information to the paragraph text processing rules that meet user needs obtained by the rule debugging module, save them as files or publish them as application interfaces.

[0113] Preferably, the present invention further provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the paragraph text information processing method of the present invention.

[0114] Compared with the existing technical means, the present invention can not only realize complex paragraph text processing functions through free combination, but also has good readability, strong interactivity, low development difficulty, and easy collaboration and management. It is suitable for the development and deployment scenarios of paragraph text processing rules, such as those that require low programming ability of developers, rule development needs to be independent of one or several specific programming languages, and interactive rule development and debugging.

[0115] The above description shows and describes several preferred embodiments of the present invention, but as mentioned above, it should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the application concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A paragraph text information processing method, characterized in that: The method comprises the following steps: S1. Language definition: Design a paragraph text processing language PTPL, which defines the pattern matching, processing rules and rule combination of paragraph text. The paragraph text processing language is defined at the lexical, grammatical and semantic levels, including: S11. Define lexical rules; S12. Define grammatical rules; S13. Define semantic rules, where the semantic rules include: (1) further converting the paragraph text processing rules written by the developer to form an execution process definition; (2) The execution process is defined as a combination of text processing steps STEP; S2. Rule parsing: The developer writes paragraph text processing rules according to the PTPL specification, and the paragraph text processing rules written by the developer are parsed according to the definition of S1 to form an execution process definition; S3. Rule implementation: Implement the execution procedures corresponding to various execution process definitions according to the PTPL specification, and then execute the execution procedures corresponding to the execution process definitions on the input test paragraph text in sequence according to the execution process definitions obtained in S2 and return the results; S4. Rule debugging step: determine whether the returned result meets the user's expectations. If the result is incorrect or not as expected, modify the paragraph text processing rules written in S2 and continue to test the input test paragraph text until it meets the user's needs. S5. Rule deployment: add description information to the paragraph text processing rules that meet user needs obtained in S4, and save them together as a file or publish them as an application interface; The S2 specifically includes: S21. Lexical scanning step, according to the definition of S11, using regular expressions to match the characters encountered in greedy mode, matching the paragraph text processing rules written by the developer from left to right, and parsing them into a sequence of tokens; if the regular expression does not match one of the above tokens, a scanning error is reported at the current position; S22. Grammar parsing step, writing grammar parsing rules to parse the sequence of the tags into a grammar tree; if any grammar parsing rule cannot be matched at the current lexical tag position, an exception is thrown at the current lexical tag position to prompt the user that the position matches incorrectly; S23. Semantic processing step, traversing the entire grammar tree, performing grammar analysis on the paragraph text processing rules written by the developer, and generating an execution process definition according to the definition rules of S13; the step S3 processes the input test paragraph text in sequence according to the execution process definition parsed by S23, according to each step in the execution process definition, wherein the result output after the processing of the previous step is used as the input of the next step, and the output of the last step is used as the output of the paragraph text processing rules written by the developer, thereby forming the result of the paragraph text processing.

2. The paragraph text information processing method according to claim 1, characterized in that: The S11. defining lexical rules specifically includes: lexical tokens include English uppercase characters UCASE_LETTER, English lowercase characters LCASE_LETTER, Chinese characters CN_ZH_LETTER, words, unmarked short strings UNESCAPED_STRINTG, marked short strings INLINE_STRING, and marked long strings MULTILINE_STRING; The S12. definition of grammar rules specifically includes: the grammar rules include comment item COMMENT, indentation rule INDENT, rule file description DOC_INFO, text processing step STATEMENT, text processing operator OP, parameter list INLINE_ARGUMENTS and pattern definition REGEX_LONG; wherein the parameter list INLINE_ARGUMENTS supports Boolean parameter UNMAED_ARGUMENT and key value parameter KEYVAL_ARGUMENT; the pattern definition REGEX_LONG is implemented with a regular expression; the text processing step is used to define the order of execution between operators, and is guided by the operator name OP_NAME, indicating that the operator is executed after the operator with the corresponding OP_NAME.

3. The paragraph text information processing method according to claim 1, characterized in that: It also includes that according to the definition of PTPL, the text processing process, i.e., the execution process, is defined as a combination of the following four processing methods: (1) Text segmentation: Take one or more patterns as parameters and a paragraph text as input. For each pattern associated with the execution process, return the start and end positions of the text matched by the pattern, and segment the paragraph text into multiple sub-texts according to the mapping rules defined by other parameters; (2) Text extraction: Using the capture grouping function, when the input sub-text matches the defined pattern, the text matching the pattern is extracted from the sub-text and output; (3) Text filtering: Taking multiple sub-texts formed after a paragraph text is processed in several steps as input, decide whether to retain the sub-text based on the matching degree between one or more defined positive or negative patterns and each sub-text; (4) Text conversion: The multiple sub-texts formed after the paragraph text is processed in several steps are used as input, matched according to the defined pattern, and the matched sub-text range is converted to replace each paragraph sub-text to form new paragraph sub-texts.

4. The paragraph text information processing method according to claim 3, characterized in that: The step S4 specifically includes: S41. The user writes paragraph text processing rules on the debugging interface, and the parsing area in the debugging interface outputs the currently parsed rules in real time; if the current rule definition is correct, the test interface outputs the definition of the execution process to be executed according to the logic of step S2; if the current rule definition is wrong, the corresponding error message is output until the current rule meets the user's text processing needs; S42. The user inputs some test paragraph text as sample text on the debugging interface. When the sample text changes, the debugging interface returns the result of text processing in real time according to the processing logic of step S3 and displays it in the debugging interface. In the debugging interface, the user creates and modifies paragraph text processing rules and sample text in real time until the result displayed on the interface meets the user's expectations.

5. The paragraph text information processing method according to claim 4, characterized in that: The step S5 specifically includes: S51. Save the paragraph text processing rules and test paragraph text that meet the user's expectations; S52. Add processing rule description information to explain the business context in which the paragraph text processing rule applies; S53. Associate the saved paragraph text processing rules, the tested paragraph text and the business background description, and publish them together as a Web service backend in conjunction with the rule parsing described in S2 and the rule implementation described in S3.

6. The paragraph text information processing method according to claim 1, characterized in that: The execution process definition is a sequential combination of text processing steps; each generated text processing step includes three parts: processing method, processing parameters and mode definition; the processing method is one of the tasks of segmentation, replacement, filtering and extraction of paragraph text; the processing parameters correspond to the parameter list INLINE_ARGUMENTS in the above step S12; the mode definition corresponds to one or more REGEX symbols in the above step S12; if some steps lack necessary parameters or have type errors, the user is prompted that the processing step has a semantic error.

7. The paragraph text information processing method according to claim 1, characterized in that: Each STEP is decomposed into three parts: processing method, processing parameters and pattern definition; the processing method of each STEP is one of the four tasks of segmentation, replacement, filtering or extraction of paragraph text; the pattern definition contained in each STEP is implemented using regular expression variants, word matching rules or other matching rules for sequences.

8. A paragraph text information processing system, characterized in that: include: The language definition module is used to design a paragraph text processing language PTPL. The paragraph text processing language defines the pattern matching, processing rules and rule combination methods for paragraph text. The paragraph text processing language is defined at the lexical, grammatical and semantic levels, including: defining lexical rules; defining grammatical rules; defining semantic rules, wherein the semantic rules include: (1) further converting the paragraph text processing rules written by the developer to form an execution process definition; (2) the execution process definition is a combination of text processing steps STEP; The rule parsing module is used to enable developers to write paragraph text processing rules according to the PTPL specification, and parse the paragraph text processing rules defined by developers according to the definition of the language definition module to form an execution process definition; The rule implementation module is used to first implement the execution programs corresponding to various execution process definitions according to the PTPL specification, and then execute the execution programs corresponding to each processing step in sequence on the input test paragraph text according to the execution process definition obtained by the rule parsing module and return the result; A rule debugging module is used to determine whether the returned result meets the user's expectations. If the result is incorrect or does not meet the expectations, the paragraph text processing rules written in the rule parsing module are modified and the input test paragraph text is continuously tested until it meets the user's requirements; The rule deployment module is used to add description information to the paragraph text processing rules that meet user needs obtained by the rule debugging module, save them as files or publish them as application interfaces; the rule parsing module specifically includes: Lexical scanning: According to the definition, the paragraph text processing rules written by the developer are matched from left to right using regular expressions in greedy mode to parse into a sequence of tokens; if the regular expression does not match one of the above tokens, a scanning error is reported at the current position; Grammar parsing: Write grammar parsing rules to parse the sequence of tokens into a grammar tree; if any grammar parsing rule cannot be matched at the current lexical token position, an exception is thrown at the current lexical token position to prompt the user that the position is matched incorrectly; Semantic processing: traverse the entire grammar tree, perform grammar analysis on the paragraph text processing rules written by the developer, and generate an execution process definition based on the defined rules; according to the parsed execution process definition, process the input test paragraph text in sequence according to the steps in the execution process definition, where the result output after the previous step is used as the input of the next step, and the output of the last step is used as the output of the paragraph text processing rules written by the developer, thereby forming the result of paragraph text processing.

9. A computer storage medium, characterized in that The medium stores a computer program, and the computer program is executed by a processor to implement the paragraph text information processing method according to any one of claims 1 to 7.

Images