Text analysis method and device, electronic equipment and storage medium

Abstract

The invention provides a text analysis method and device, electronic equipment and a storage medium, and can be applied to the technical field of computer science and financial science and technology. The method comprises the steps that in response to a received to-be-analyzed text, lexical analysis is conducted on the to-be-analyzed text based on a symbol set, a mark sequence of the to-be-analyzed text is obtained, and the symbol set comprises an identifier, an operator and a separator; based on grammar rule information, grammar analysis is carried out on the mark sequence to obtain a grammar tree, and the grammar rule information is obtained based on template information generated by identifiers, operators and separators and rule framework information corresponding to the template information; and selecting a target traversal strategy from the candidate traversal strategy library on the basis of demand information of a target service corresponding to the to-be-analyzed text, traversing each node in the syntax tree to obtain a node processing result of each node, and combining the node processing result with a logic relationship between the nodes in the syntax tree to obtain an analysis result of the to-be-analyzed text.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer science and financial technology, in particular to a text analysis method and device, electronic equipment and storage medium. BACKGROUND

[0002] In embedded systems, scientific research, automation control and other scenarios, developers often need to define and use specific symbols and rules to express complex logic and data structures. Although traditional programming languages are powerful, they lack flexibility or are inefficient when dealing with these specific symbols, so there is an urgent need to develop a simple language analysis engine that can efficiently analyze custom symbols.

[0003] In related technologies, tools such as regular expression analysis, JavaScript object notation parser, extensible markup language parser, etc. are usually designed for standardized formats and cannot be directly applied to scenarios containing a large number of custom symbols. The interpreter pattern provides a framework for building custom language parsers, but its implementation complexity is high and requires high professional skills from developers. SUMMARY

[0004] In view of the above problems, the present disclosure provides a text analysis method, device, electronic equipment and storage medium.

[0005] According to a first aspect of the present disclosure, a text analysis method is provided, comprising: in response to receiving a text to be parsed, performing lexical analysis on the text to be parsed based on a symbol set to obtain a token sequence of the text to be parsed, wherein the symbol set includes identifiers, operators and separators; performing syntax analysis on the token sequence based on grammar rule information to obtain a syntax tree, wherein the grammar rule information is determined by: in response to a selection operation on template information, obtaining rule framework information corresponding to the template information, which contains multiple framework parameters, and in the case of detecting a parameterized configuration operation, updating a target parameter in the multiple framework parameters to obtain the grammar rule information, the template information being generated by the identifiers, operators and separators; selecting a target traversal strategy from a candidate traversal strategy library based on requirement information of a target business corresponding to the text to be parsed, traversing each node in the syntax tree to obtain a node processing result of each node, and combining the node processing result and the logical relationship between each node in the syntax tree to obtain an analysis result of the text to be parsed, wherein the requirement information contains at least one of information indicating that the target business is processing a logical expression with a nested structure, and information indicating that the target business is processing batch data in parallel or sequentially.

[0006] The second aspect of the present disclosure provides a text analysis device, comprising: a processing module configured to, in response to receiving a text to be analyzed, perform lexical analysis on the text to be analyzed based on a symbol set to obtain a token sequence of the text to be analyzed, wherein the symbol set comprises identifiers, operators and delimiters; an analysis module configured to, based on grammar rule information, perform syntax analysis on the token sequence to obtain a syntax tree, wherein the grammar rule information is determined by: in response to a selection operation on template information, obtaining rule framework information corresponding to the template information and containing a plurality of framework parameters, in a case where a parameterized configuration operation is detected, updating a target parameter in the plurality of framework parameters to obtain the grammar rule information, and the template information is generated by the identifiers, the operators and the delimiters; a selection module configured to, based on requirement information of a target service corresponding to the text to be analyzed, select a target traversal strategy from a candidate traversal strategy library, traverse each node in the syntax tree to obtain a node processing result of each node, and combine the node processing result and a logical relationship between each node in the syntax tree to obtain an analysis result of the text to be analyzed, wherein the requirement information contains at least one of information indicating that the target service is processing a logical expression with a nested structure, and information indicating that the target service is processing batch data in parallel or sequentially.

[0007] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0008] The fourth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0009] The fifth aspect of the present disclosure further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0010] According to the embodiments of the present disclosure, by providing the template information generated from the base symbol and its corresponding rule framework information containing configurable framework parameters, and generating the final grammar rule information when the parameterized configuration operation is detected, the complexity of the grammar definition is simplified from writing specific rule code to filling or adjusting the key parameters of the preset framework, which significantly reduces the professional threshold and operation complexity of the parser construction, improves the intuitiveness and flexibility of use, and enables users to quickly customize the parsing logic that meets specific needs without deep compiler technology; in the interpretation execution stage, the optimal traversal strategy is dynamically selected according to the demand information containing specific processing intentions (processing nested logical expressions or batch data processing) to traverse and calculate the syntax tree, and the node processing results are combined according to the logical relationship between the nodes, which realizes the efficient interpretation and execution of the custom language, not only improves the processing performance of large-scale or complex rules and reduces the delay, but also enhances the domain adaptation ability of the parsing engine to different business scenarios (such as real-time decision and batch processing) through customizable traversal and calculation strategies. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 An application scenario diagram of a text parsing method, apparatus, electronic device and storage medium according to embodiments of the present disclosure is schematically shown;

[0013] Figure 2 A flowchart of a text parsing method according to embodiments of the present disclosure is schematically shown;

[0014] Figure 3 A flowchart of a generation method from a starting symbol to a complete syntax tree is schematically shown;

[0015] Figure 4 A structural block diagram of a text parsing apparatus according to embodiments of the present disclosure is schematically shown;

[0016] Figure 5 A block diagram of an electronic device suitable for implementing a text parsing method according to embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description is merely exemplary and is intended to provide a thorough understanding of the present disclosure. The following detailed description and examples are provided as an exemplary basis for understanding the present disclosure, and are not intended to limit the present disclosure in any way. Furthermore, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concept of the present disclosure.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the term "includes" and tautological expressions thereof, such as "including", means the inclusion of the stated features, steps, operations, and / or components but not the exclusion of one or more other features, steps, operations, or components.

[0019] All terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0020] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of A, B, or C, in the meaning as understood by one of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having both A and B, a system having both A and C, a system having both B and C, and / or a system having A, B, and C, etc.).

[0021] In the technical solutions of the present disclosure, the user information (including but not limited to user personal information, user image information, user equipment information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of related data comply with relevant laws, regulations, and standards, take necessary processing measures, do not violate public order and good customs, and provide corresponding operation portals for users to choose authorization or refusal.

[0022] In a scenario of making an automatic decision by using personal information, the method, device and system provided by the embodiments of the present disclosure provide a corresponding operation entry for the user to select to agree or reject the automatic decision result; if the user selects to reject, the expert decision process is entered. The expression "automatic decision" herein refers to an activity of automatically analyzing and evaluating a person's behavior habit, interest and hobby, or economic, health and credit status by a computer program, and making a decision. The expression "expert decision" herein refers to an activity of making a decision by a person who is engaged in a certain field, has special experience, knowledge and skills, and reaches a certain professional level.

[0023] In the related art, the self-defined language analysis tool and method often involve complex compiler technology, and require high professional skills of the developer, so that ordinary users are difficult to quickly start and customize the parser according to their own needs. Although the traditional analysis system allows users to define the symbol set and the grammar rule, the extensibility and flexibility are limited, and when a new symbol needs to be added or the grammar rule needs to be modified, the system code needs to be deeply modified or the parser needs to be regenerated, so that the maintenance cost is high.

[0024] Meanwhile, the process of defining the symbol set and the grammar rule by the user is often not intuitive, and needs to rely on professional grammar description languages or tools, which increases the learning cost and operation difficulty of the user. For a large-scale or complex self-defined language, the traditional parser is prone to performance bottleneck when interpreting and executing, which limits the application in real-time processing or high-performance computing scenarios. Moreover, the field adaptability of the traditional technology is poor, for example, it lacks optimization and support for specific fields, and in specific fields (such as embedded systems, scientific research, etc.), a large amount of self-defined symbols and specific logic may need to be processed, which is difficult to meet the actual application requirements.

[0025] In view of this, the embodiment of the present disclosure provides a text parsing method, device, electronic equipment and storage medium, the method comprises: in response to receiving a to-be-parsed text, performing lexical analysis on the to-be-parsed text based on a symbol set to obtain a token sequence of the to-be-parsed text, wherein the symbol set comprises an identifier, an operator and a separator; performing syntax analysis on the token sequence based on grammar rule information to obtain a syntax tree, wherein the grammar rule information is determined by: in response to a selection operation on template information, obtaining rule framework information corresponding to the template information, the rule framework information comprising a plurality of framework parameters, in the case of detecting a parameterization configuration operation, updating a target parameter in the plurality of framework parameters to obtain the grammar rule information, the template information being generated by the identifier, the operator and the separator; based on the requirement information of the target business corresponding to the to-be-parsed text, selecting a target traversal strategy from a candidate traversal strategy library, traversing each node in the syntax tree to obtain a node processing result of each node, and combining the node processing result and the logical relationship between each node in the syntax tree to obtain a parsing result of the to-be-parsed text, wherein the requirement information comprises at least one of information indicating that the target business is processing a logical expression with a nested structure, and information indicating that the target business is processing batch data in parallel or sequentially.

[0026] According to the embodiment of the present disclosure, by providing template information generated by basic symbols and its corresponding rule framework information containing configurable framework parameters, and generating the final grammar rule information when detecting the parameterization configuration operation, the complexity of syntax definition is simplified from writing specific rule code to filling or adjusting the key parameters of the preset framework, which significantly reduces the professional threshold and operation complexity of parser construction, improves the intuitiveness and flexibility of use, and enables users to quickly customize the parsing logic that meets specific needs without deep compiler technology; in the interpretation execution stage, by dynamically selecting the optimal traversal strategy according to the requirement information containing specific processing intentions (processing nested logical expressions or processing batch data) to traverse and calculate the syntax tree, and combining the node processing result with the logical relationship between the nodes, efficient interpretation and execution of the self-defined language is realized, which not only improves the processing performance of large-scale or complex rules and reduces the delay, but also enhances the domain adaptability of the parsing engine to different business scenarios (such as real-time decision-making and batch processing) through customizable traversal and calculation strategies.

[0027] Figure 1 The application scenario diagram of the text parsing method, device, electronic equipment and storage medium according to the embodiment of the present disclosure is schematically shown.

[0028] As Figure 1As shown, the application scenario 100 according to this embodiment can include a terminal device 101, a network 102 and a server 103. The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0029] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc.

[0030] The terminal device 101 is an interface directly operated and interacted by the user, and its core role is to provide visual editing of custom rules and intuitive display of analysis results. For example, it sends the original rule text input by the user to the server 103 through the network 102 for core analysis; at the same time, it receives and displays the analysis results returned by the server. The terminal device 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0031] The server 103 can be a server providing various services, and the server is the core computing unit of the entire system, encapsulating and executing the text analysis method. For example, it receives the rule text from the network 102, calls the built-in analysis engine for processing, and returns the final calculation or query result to the terminal device 101 through the network 102.

[0032] It should be noted that the text analysis method provided by the embodiments of the present disclosure can generally be executed by the server 103. Correspondingly, the text analysis apparatus provided by the embodiments of the present disclosure can generally be arranged in the server 103. The text analysis method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 103 and capable of communicating with the terminal device 101 and / or the server 103. Correspondingly, the text analysis apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 103 and capable of communicating with the terminal device 101 and / or the server 103.

[0033] It should be understood that Figure 1 The number of terminal devices, networks and servers in the application scenario 100 is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks and servers.

[0034] Figure 2 A flowchart of a text analysis method according to an embodiment of the present disclosure is schematically shown.

[0035] As Figure 2 shown, the text analysis method of this embodiment includes operations S210-S230.

[0036] In operation S210, in response to receiving the text to be parsed, the text to be parsed is subjected to lexical analysis based on a symbol set to obtain a token sequence of the text to be parsed. The symbol set includes identifiers, operators, and delimiters.

[0037] In an embodiment of the present disclosure, the text to be parsed can be a text input by a user through a terminal device. The symbol set can be a set of symbols predefined by the user to describe the constituting mode of all legal basic units (vocabulary) in the text to be parsed, for example, a configuration file of a "vocabulary table" and its identification rules. The identifier can be used to name entities such as variables, functions, and keywords, and is usually composed of letters, numbers, and underscores according to a specific rule. The operator can be one or more characters representing an operation or a logical operation. The delimiter can be a boundary character used to delimit different parts of the text.

[0038] The token sequence can be the output result of lexical analysis, which is an ordered list, and each element in the list is a "token" that can contain two core information: a morpheme (an actual string matched in the source text) and a type (a classification in the symbol set, such as an identifier, an addition operator).

[0039] For example, the lexical analysis of the text to be parsed based on the symbol set can include: using a state transition tool (for example, a deterministic finite automaton compiled according to a regular expression) to scan the input text, and identifying and classifying continuous character sequences into discrete symbols according to the rules defined by the symbol set.

[0040] In operation S220, the token sequence is subjected to syntax analysis based on grammar rule information to obtain a syntax tree. The grammar rule information is determined by: in response to a selection operation on template information, obtaining rule framework information corresponding to the template information and containing multiple framework parameters, and in a case where a parameterized configuration operation is detected, updating a target parameter in the multiple framework parameters to obtain the grammar rule information. The template information is generated by identifiers, operators, and delimiters.

[0041] In embodiments of the present disclosure, the grammar rule information can be derived based on the template information generated by the identifiers, operators and separators, and the rule framework information corresponding to the template information, which is a set of formalized rules for describing the structure of a legal sentence of a certain language, used to define how to combine the "tokens" (words) produced by lexical analysis into larger, meaningful syntactic units (e.g., expressions, statements, programs), and can be used as the core logic and basis for judgment of the syntax analysis. The template information can be a parameterized syntax structure prototype defined in advance according to the syntax pattern by the basic symbol types of identifiers, operators and separators. The rule framework information can be a structured container or a generation rule for accommodating and instantiating the "template information", which specifies how the template is concretized, how it is combined, and the priority relationship. The grammar book is the output result and core data carrier of the syntax analysis.

[0042] In embodiments of the present disclosure, the rule framework information can be a structured rule description container bound to a specific template information, which contains the complete syntax skeleton of the template and explicitly identifies all the placeholder or variable positions allowed to be customized by the user. The framework parameter can be a variable or placeholder built into the rule framework information, which can be modified by the user to customize specific rules. The framework parameter represents the part that needs to be concretized in the framework, such as the type of operator, the name of identifier or the structure of sub-expression.

[0043] The parameterized configuration operation can be an interactive process in which the user assigns, selects or modifies the framework parameters exposed in the rule framework information. The target parameter can be one or more specific framework parameters that the user is currently modifying or intends to modify in the parameterized configuration operation. The update can be a data processing action in which the system replaces the placeholder of the corresponding target parameter in the rule framework information with a specific value (such as a specific symbol, string or another rule fragment) provided by the user in response to the user's parameterized configuration operation.

[0044] For example, by showing the user a plurality of template information and rule framework information, the user selects or inputs specific symbols for the placeholders in the template, and the system instantiates the abstract template into a specific production rule according to the user's filled content, and integrates the instantiated rule into the complete grammar rule information library according to the rule framework in which it is located; the system compiles (or interprets) the text or structured grammar rule information generated by the user configuration into a data structure that can be directly used by the syntax analyzer.

[0045] For example, the process of performing syntax analysis can include: the analyzer starts from the first token of the token sequence, queries the internally compiled grammar rules, matches the current and subsequent tokens with the right part of a rule; if the match is successful, a corresponding syntax tree node (e.g., a "Condition" node) is created; if the matched rule right part contains a non-terminal (e.g., "Expression"), recursively start the analysis of the new token subsequence, and return the analysis of the subnode as the child node of the current node; this process continues until the entire token sequence is successfully matched to the start symbol (e.g., "Program") defined by the grammar rule, and finally forms a complete syntax tree with the start symbol as the root node.

[0046] In operation S230, based on the requirement information of the target service corresponding to the text to be parsed, a target traversal strategy is selected from a candidate traversal strategy library, each node in the syntax tree is traversed to obtain a node processing result of each node, and the node processing result and the logical relationship between each node in the syntax tree are combined to obtain a parsing result of the text to be parsed, wherein the requirement information includes at least one of information indicating that the target service is processing a logical expression with a nested structure, and information indicating that the target service is processing batch data in parallel or sequentially.

[0047] In an embodiment of the present disclosure, the requirement information can be a scenario attribute description that implicitly affects the computing process and performance of the current business rule to be processed, and can be meta information describing "how to process data", such as "the rule type is real-time risk control decision" and "the data processing mode is batch statistics". The candidate traversal strategy library can be a pre-defined strategy set containing multiple syntax tree node access sequences and computing scheduling schemes, each strategy being optimized for a specific computing mode. The target traversal strategy can be a specific strategy object dynamically selected from the candidate traversal strategy library according to the requirement information of the target service for this interpretation execution.

[0048] The node processing result can be an output value generated after applying the pre-associated business logic of the current visited syntax tree node to calculate during the traversal process. The combination can be an operation process of gradually aggregating and reducing each node processing result according to the operation rules represented by the inherent parent-child and sibling relationships between the nodes in the syntax tree, and finally synthesizing a whole result. The logical relationship can be the connection relationship between the nodes encoded by the syntax tree data structure itself. The parsing result can be the complete semantic calculation conclusion of the text to be parsed, which can be a Boolean value, a numerical value, a string, or complex structured data. The nested structure of the logical expression can be a description in the form of a syntax tree, which refers to the deep nesting of nodes in the tree (especially operator nodes), forming a multi-level parent-child relationship structure. Batch data can refer to the characteristics of the business scenario, that is, a large number of independent tasks or data units with similar structures but different data need to be processed continuously, in parallel, or sequentially.

[0049] According to an embodiment of the present disclosure, the method further includes: in response to a selection operation on the template information, obtaining rule framework information corresponding to the template information, wherein the rule framework information contains a plurality of framework parameters; and in response to a parameterization configuration operation, updating a target parameter in the plurality of framework parameters to obtain the grammar rule information.

[0050] For example, the system responds to the user's selection operation on the template information to obtain the rule framework information corresponding to the "simple comparison" template information from the background. The framework can be internally represented as a structure similar to {leftOperand: [PARAM], operator: [PARAM], rightOperand: [PARAM]}. A form is dynamically generated on the right side of the interface, and the three input boxes in the form correspond to the three framework parameters in the framework: "left operand" (leftOperand), "comparison operator" (operator), and "right operand" (rightOperand).

[0051] The user starts filling in the form. First, the user selects "object property" -> "age" in the "left operand" drop-down menu. At this time, the "left operand" is the target parameter. The system receives this operation and performs an update to set the value of the leftOperand parameter in the rule framework to an internal reference pointing to the "age" attribute. Further, the user selects "greater than (>)" in the "comparison operator" drop-down menu, and the target parameter becomes operator. The system updates the framework again. Finally, the user inputs "30" in the "right operand" text box, and the target parameter becomes rightOperand. The system updates its value to the constant "30". At this time, all framework parameters in the rule framework have been configured.

[0052] In one feasible implementation, to address extensibility and domain adaptability, extension interfaces and plugin mechanisms can be provided to allow users to add new symbols, modify grammar rules, or optimize the interpretation and execution process as needed. For example, the symbol extension interface supports dynamic registration of new symbol types; the grammar extension point plugin framework allows the injection of custom production rules; the syntax tree accessor interface provides Visitor or Transformer base classes for business parties to extend traversal logic; the execution engine plugin supports access to external computing engines (such as rule engines and risk control engines); and metadata extension adds business metadata through the AnnotationProcessor mechanism.

[0053] According to embodiments of this disclosure, by providing intuitive template information for users to select from, and transforming the complex rule-writing process into a fill-in-the-blank configuration of framework parameters in a graphical form, users without knowledge of compiler principles or formal grammar can quickly and accurately create grammatically correct business rules, achieving a leap in experience from "writing code" to "filling out a form." The rule framework information and its parameters form an intuitive mapping of the rule structure on the interface. When users modify target parameters through parameterized configuration operations, they can see the changes in rule semantics in real time. This interactive method makes the adjustment, trial and error, and optimization of business rules highly intuitive and efficient.

[0054] According to embodiments of this disclosure, by providing template information generated from basic symbols and its corresponding rule framework information containing configurable framework parameters, and generating the final grammar rule information when parameterized configuration operations are detected, the complexity of syntax definition is simplified from writing specific rule code to filling or adjusting key parameters of a preset framework. This significantly reduces the professional threshold and operational complexity of parser construction, improves intuitiveness and flexibility, and allows users to quickly customize parsing logic to meet specific needs without delving into compiler technology. In the interpretation and execution phase, the optimal traversal strategy is dynamically selected to traverse and compute the syntax tree based on the requirement information containing specific processing intentions (processing nested logical expressions or performing batch data processing), and the node processing results are combined with the logical relationships between nodes. This achieves efficient interpretation and execution of custom languages, not only improving the processing performance of large-scale or complex rules and reducing latency, but also enhancing the domain adaptability of the parsing engine to different business scenarios (such as real-time decision-making and batch processing) through customizable traversal and computation strategies.

[0055] According to an embodiment of the present disclosure, before the text to be parsed is lexically analyzed based on the symbol set, the method further comprises: receiving symbol definition data input through a visual interface or a configuration file to obtain initial symbols, wherein the symbol definition data comprises a symbol identifier, a symbol type, and a regular expression for matching the symbol; in response to receiving an add-symbol instruction, detecting whether there is a symbol conflict between the added symbol and the initial symbol; and in a case where the detection result indicates that there is a symbol conflict between the added symbol and the initial symbol, outputting a plurality of conflict resolution schemes according to the symbol type and the priority corresponding to the symbol type, for a user to select.

[0056] In an embodiment of the present disclosure, the symbol definition data can be used to describe the structured information tuple of the custom language symbol, which is a digital representation of the user's intention and a standard input format for the system to receive and store symbol information, providing raw materials for subsequent lexical analysis rule generation. The visual interface can allow users to interact with graphical elements (such as buttons, forms, drag-and-drop areas) to input or modify symbol definition data. The configuration file can be a text file following a specific format (e.g., JSON, YAML), in which users can write symbol definition data in a structured text form.

[0057] The symbol identifier can be a string name or a keyword in the symbol definition data that uniquely refers to the symbol within the system. The symbol type can be a label that functionally classifies the symbol, and is a predefined finite set (e.g., identifier, operator, separator). The regular expression can be used to describe a formalized string that matches the pattern of character sequences, and in the symbol definition data, the specific composition rules of the symbol in the source code text can be defined.

[0058] The add-symbol instruction can be a "save" operation from the user on the visual interface, or a reload of the configuration file. The symbol conflict can refer to the overlapping or containing relationship between the regular expression pattern of the added symbol and the pattern of the existing symbol (initial symbol), resulting in a state where the lexical analyzer may produce two or more legal matching interpretations for the same text when scanning the text. The conflict resolution scheme can be a set of processing suggestions generated by the system for the detected specific conflict, which can be selected by the user, each suggestion containing a specific operation (e.g., adjusting the priority, modifying the regular expression) and the expected result.

[0059] For example, a built-in symbol conflict detection mechanism automatically prompts when a newly added symbol conflicts with an initial symbol. After the system receives a request to add a symbol definition, the request can be regarded as a newly added symbol instruction. The definition data of all initial symbols can be loaded from the persistent storage first, and the regular expression of the newly added symbol is extracted and compared with the regular expression of each initial symbol. By executing a conflict detection algorithm, it is detected that the string "IN" can be matched by both the newly added operator "IN" and the existing identifier pattern, so it is determined that there is a symbol conflict.

[0060] Based on this, two (or more) conflict resolution schemes are generated and returned to the front end. For example, scheme A (recommended): follow the default priority and determine "IN" as an operator; automatically ensure that the operator "IN" is matched first during lexical scanning, and the identifier pattern will no longer match the string "IN". Scheme B: modify the regular expression of the newly added symbol to make it more accurate or different, for example, change it to "\sIN\s" (require space before and after), to eliminate the intersection with the identifier pattern. Scheme C: temporarily ignore this conflict, but the system records and warns that this may cause ambiguity in subsequent parsing. The front end displays the conflict resolution schemes to the user in the form of a pop-up window or a highlighted list; after the user reads them, the user selects scheme A (accepts the recommended solution).

[0061] According to an embodiment of the present disclosure, by performing symbol conflict detection during the design phase, the overlap problem of regular expression patterns can be actively discovered and handled. This avoids parsing errors or unpredictable behavior caused by uncertain lexical scanning results at runtime, and fundamentally improves the robustness of the parsing engine.

[0062] According to an embodiment of the present disclosure, the symbol set is used to perform lexical analysis on the text to be parsed to obtain a token sequence of the text to be parsed, including: constructing a state transition tool based on the matching patterns corresponding to each symbol in the symbol set; using the state transition tool to identify the text to be parsed to obtain a character sequence corresponding to the matching patterns; converting the symbol type corresponding to the character sequence in the symbol set to a corresponding token, and arranging the tokens in the identification order to obtain a token sequence.

[0063] In an embodiment of the present disclosure, the matching pattern can be a formal rule defined for each symbol in the symbol set to accurately describe all legal forms of the symbol in the source code text, in the form of a regular expression. The state transition tool can be an automated scanner generated by pre-compiling all symbol matching patterns, used for efficient text identification. The character sequence can be a continuous original string fragment successfully matched and segmented from the input text during the "identification" process.

[0064] The conversion can be a process of information enhancement and structuring. It refers to encapsulating the recognized character sequence and its corresponding symbol type in the symbol set into a tagged object with a unified structure. The tag can be a standardized output unit of lexical analysis and can be used as an atomic input of syntax analysis. The identification sequence can be an ordered list composed of tags arranged in the order of "recognition".

[0065] For example, the lexical analysis stage converts the input sequence into a tag sequence using a state transition tool (such as a scanner based on a finite automaton) generation technique. A scan pointer can be initialized to point to the beginning of the text. From the current scan pointer, characters are input one by one into the state transition tool. The tool starts from the initial state and performs state jumping according to the input characters. This process records the last "terminal state" and its position. When the input of the next character causes no legal state to jump, the scanning is paused.

[0066] At this time, the last "terminal state" recorded is checked. If it can be found, it can indicate that a character sequence (from the scan pointer to the recorded position) is successfully recognized. For example, "age" is recognized; if it cannot be found (i.e., the first character cannot jump), a lexical error (illegal character) is reported. Through the "terminal state-symbol" mapping table, the symbol corresponding to the terminal state is found to be "IDENTIFIER", and the symbol type of the symbol is found to be "identifier" from the symbol set, and the "conversion" operation is performed to create a tag object, and the tag is added to the tokens list; the scan pointer is moved to the end of the recognized character sequence (skipping the processed part); if there are remaining characters in the text, the recognition step is returned to start the next round of recognition; when the scan pointer reaches the end of the text, the loop ends. At this time, the tokens list stores all tags in order, which constitutes the tag sequence of the text, and the tag sequence is output to the syntax analyzer.

[0067] According to an embodiment of the present disclosure, by pre-compiling the user-defined matching pattern into a state transition tool, the runtime pattern matching is changed from "interpreting and executing regular expressions" to "doing table lookup jumping on a deterministic state machine". This process has linear time complexity and the result is completely determined, avoiding uncertain behaviors such as backtracking, thereby being significantly superior to the method of dynamically interpreting regular expressions in efficiency and performance predictability.

[0068] According to an embodiment of the present disclosure, based on the grammar rule information, the token sequence is parsed to obtain a syntax tree, including: taking the start symbol defined in the grammar rule information as a root node, starting from the start position of the token sequence, matching the syntax component sequence described in each production rule with the token types and contents in the token sequence in corresponding order; in the case of successful matching, creating a parent node corresponding to the production rule, and taking the child node matched with the production rule as a child node of the parent node; combining layer by layer upwards until the start symbol is matched to obtain a complete syntax tree.

[0069] In an embodiment of the present disclosure, the start symbol can be a top-level non-terminal symbol in the grammar rule, representing a complete language unit (for example, a program or a statement), and is a logical starting point and target of the syntax analysis process. The production rule can be a basic unit of the grammar rule information, used to describe the formal rule of how the syntax components are constructed. The syntax component sequence can be a list of ordered syntax elements listed on the right side of an arrow (→) in a production rule, which constitutes the rule. These components can be terminal symbols (corresponding to token types) or non-terminal symbols (corresponding to other production rules).

[0070] Creating a parent node can be instantiating a new syntax tree node object representing the left part non-terminal symbol of the production rule after successful matching of the production rule. The child node of the parent node can refer to the dependent node of the parent node. Layer-by-layer upward combination can describe the characteristics of the construction direction and flow of the syntax tree, indicating that the analysis process is bottom-up or recursive return, for example, first matching and creating the bottom layer specific syntax structure (child node), and then taking them as components to match and create higher layer abstract structure (parent node) until reaching the top layer. The complete syntax tree can be the final product of the syntax analysis process, taking the start symbol node as the root, taking each specific token as the leaf, and taking the non-terminal symbol node corresponding to each production rule as the intermediate, which is a tree-shaped data structure containing all the syntax structure information of the input text.

[0071] For example, a stack is initialized to assist in tracking, or recursion is implemented directly through the function call stack; the parseStatement() function is called, with the goal of matching a Statement; in the grammar, one production rule for Statement is: Statement→Expression, so the current task becomes matching the right-hand side of this production rule, the sequence of syntax components [Expression]; the parseExpression() function is called to match Expression, resulting in 3 leaf nodes, and in the case that the sequence of syntax components is all successfully matched, the parseExpression() function creates an Expression node as the parent node, and sets the three leaf nodes as the children of this Expression parent node, and the function returns this Expression node.

[0072] The control flow returns to the parseStatement() function, and receives an Expression node as the matching result of its child rule Expression, and since the Statement→Expression rule is successfully matched, the parseStatement() function creates a Statement node as the root parent node, and sets the received Expression node as the child of this Statement node. At this time, the start symbol Statement is matched, and all input tokens are consumed. The complete syntax tree is returned, which is rooted at the Statement node, and has one Expression child node, and the Expression node has three leaf nodes.

[0073] Figure 3 A flowchart of the generation method from the start symbol to the complete syntax tree is shown schematically.

[0074] As shown in Figure 3 , after the start symbol is determined, it is taken as the root node of the entire syntax tree. The generation method from the start symbol to the complete syntax tree includes operations S310-S390.

[0075] In operation S310, the current analysis task state is determined.

[0076] In operation S320, the sequence of syntax components is decomposed. By selecting a production rule from the grammar rule information, the abstract symbol is decomposed into a specific sequence of syntax components.

[0077] In operation S330, it is determined whether the sequence of syntax components is a terminal symbol. If yes, operation S341 is performed, and if no, operation S342 is performed.

[0078] In operation S341, the terminal symbol is compared with the input token and a subtask is created. If the comparison is successful, a leaf node is created.

[0079] In operation S342, a new analysis subtask is executed. The new analysis subtask is recursively started and a top-down decomposition process is executed.

[0080] In operation S350, it is determined whether all components of the production rule are matched successfully. If yes, operation S360 is executed, and if no, operation S380 is executed.

[0081] In operation S360, a parent node representing the production rule is created, and all child component nodes are associated as its child nodes. The new node is returned as a result, embodying a bottom-up combination process.

[0082] In operation S370, layer-by-layer upward processing is performed until completion at the root node, obtaining a syntax tree.

[0083] In operation S380, backtracking is performed by checking candidate rules to find whether there is a suitable candidate syntax structure. If yes, operation 320 is returned, and if no, operation 380 is executed.

[0084] In operation 390, operation S310 is returned and a failure signal is passed.

[0085] According to an embodiment of the present disclosure, by strictly following the production rule to match and layer-by-layer upward combination, the complex nested and priority relationship implied in the linear text can be explicitly restored to a well-structured syntax tree, eliminating the ambiguity that the source code may bring in the form of writing (such as the position of parentheses, spaces), and providing a single and correct structure view for subsequent semantic processing. The entire syntax analysis process is completely driven by external and configurable grammar rule information. When a new syntax structure needs to be supported, only the corresponding production rule needs to be added or modified, and the core analysis algorithm does not need to be rewritten. This decoupling design of grammar and algorithm enables the system to quickly adapt to the syntax changes of self-defined languages in different fields, significantly improving the scalability and maintainability of the system.

[0086] According to an embodiment of the present disclosure, based on the demand information of the target business corresponding to the text to be parsed, the target traversal strategy is selected from the candidate traversal strategy library, including: in the case where the demand information indicates that the target business is processing a logical expression with a nested structure, selecting a depth-first traversal strategy as the target traversal strategy; in the case where the demand information indicates that the target business is processing batch data in parallel or sequentially, selecting a breadth-first traversal strategy as the target traversal strategy.

[0087] In embodiments of the present disclosure, the logical expression of the nested structure can contain complex rules of multiple levels of parentheses, conditional branches (and / or). The depth-first traversal strategy can be a tree structure traversal algorithm, which can refer to visiting the first child node of a node immediately after visiting the node, until reaching a leaf node without child nodes, and then backtracking to the previous node and visiting the next child node of the node, for optimizing the logical expression of the nested structure.

[0088] Batch data, for example, applies the same rule to each user in the user list. "Sequential processing" in parallel or sequential processing can refer to processing data units one by one, and "parallel processing" can refer to processing multiple data units at the same time to improve throughput. The breadth-first traversal strategy can be a tree structure traversal algorithm, the core rule of which is to visit all nodes of the current level first, and then visit all nodes of the next level in turn. It is usually implemented with a queue data structure, for optimizing batch data processing.

[0089] For example, the text parsing method supports dynamic selection of appropriate traversal strategies (such as depth-first and breadth-first dual-mode switching) for syntax trees according to specific business rules, so as to realize custom language logic operations and data processing.

[0090] Dynamic selection of appropriate traversal strategies according to specific business rules can be: composite condition rules (such as multi-level logic and / or in recommendation rules) use depth-first traversal to ensure that child conditions are calculated first; batch data processing rules use breadth-first traversal to improve cache utilization of batch operations; business rules with priorities (such as financial product recommendation weights) use hierarchical traversal with priority queues; real-time decision rules (such as risk control interception) use the memo pattern to optimize repeated sub-tree calculations.

[0091] Taking the processing of different types of rule requests by an intelligent risk control system as an example, the system receives a parsing request, which contains rule text to be parsed and request metadata; a preprocessing module analyzes the preliminary characteristics of the request metadata and / or the rule text, and generates demand information of the target business. The system maintains a candidate traversal strategy library, which contains at least two registered strategies, for example, depth-first and breadth-first; a strategy selector component reads the demand information generated in the previous step, and in the case of information indicating "nested structure", the selector can select the depth-first strategy from the library as the target traversal strategy for this execution; in the case of information indicating "batch data", the selector selects the breadth-first strategy as the target traversal strategy.

[0092] According to an embodiment of the present disclosure, by taking the requirement information of the target service as a decision input, the system can avoid the fixed traversal mode of "one size fits all", and automatically select the depth-first traversal strategy when encountering a nested structure of the logical expression to minimize the calculation delay and memory occupation; automatically switch to the breadth-first traversal strategy when processing batch data to maximize data locality and throughput. This dynamic adaptation capability enables the system to always approach the locally optimal execution efficiency when processing diversified services.

[0093] According to an embodiment of the present disclosure, the nodes in the syntax tree are traversed to obtain the node processing result of each node, including: determining the target processing function corresponding to the current node according to the mapping relationship between the node type of each node in the syntax tree and the processing function; calling the target processing function to process the node data of the current node to determine the node processing result.

[0094] In an embodiment of the present disclosure, the mapping relationship can be a pre-established configuration rule set describing one-to-one or many-to-one correspondence between the node type and the executable processing function (or logic module). For example, a dictionary, a configuration file, or a hash table in memory. The target processing function can be a specific function, method, or executable code unit that will actually be called and executed, which is determined according to the node type of the current node after querying the mapping relationship. The node data can be the context information or parameters required for the target processing function to perform calculations, which can include original values extracted from the source code (such as identifier name "age", operator "+"), processing results of child nodes, and possibly additional type annotations.

[0095] For example, a mapping dictionary of syntax tree nodes to business logic can be established, mapping the ">" operator to the gt() method in the comparison function library; and a dynamic distributor is designed to automatically select the execution logic according to the node type (for example, automatically distinguishing between numerical comparison and string comparison).

[0096] For example, the interpreter can use a depth-first strategy to traverse the syntax tree. When a node is accessed, the node type field is read, and it is assumed that the current node represents the ">" operation in "age" > 30, and its node type can be identified as "operator_greater". Take "operator_greater" as the key to query the global mapping dictionary; the query result can be the function name (or function pointer) "op_greater_than", which can be determined as the target processing function of the current node.

[0097] Further, the op_greater_than function can be prepared with arguments by examining the node data of the current node, including references to the results of processing its children. Recursively (or by traversing the completed children) the node processing result of the first child (the identifier "age") is obtained (e.g., the value 35 obtained by calling fetch_age_from_context); the node processing result of the second child (the number "30") is obtained (i.e., the value 30).

[0098] With the node data prepared, the function op_greater_than(35, 30) is called. Inside the function, the standard numeric comparison logic is performed, determining that 35 > 30 is true; and the node processing result is recorded. The function returns the Boolean value true, which is determined to be the node processing result of the comparison operator node.

[0099] According to embodiments of the present disclosure, the node type is dynamically associated with the processing function through an external mapping relationship, so that when a new syntax element (node type) is added or the processing logic of an existing element is modified, the core traversal and distribution framework does not need to be modified, only the mapping table needs to be updated and the new processing function needs to be implemented, which can reduce the maintenance complexity of the system and support hot updating of business rules.

[0100] According to embodiments of the present disclosure, the node processing results and the logical relationships between the nodes in the syntax tree are combined to obtain the parsing result of the text to be parsed, including: according to the hierarchical structure of the syntax tree, the node processing results of the child nodes are passed to the corresponding parent nodes from bottom to top; according to the logical operations in the syntax rule corresponding to the parent node, the node processing results of the child nodes are combined to generate the upward transmission result of the parent node; and the upward transmission result of the root node is taken as the parsing result of the text to be parsed.

[0101] In embodiments of the present disclosure, the hierarchical structure can be a tree topology inherent to the syntax tree, composed of the parent-child relationships between the nodes, which defines the subordinate and combination relationships between the nodes. The bottom-up strategy can be the direction of calculation and data transmission, which means starting from the leaf node at the deepest layer of the tree structure without child nodes, calculating the result, passing it to the parent node at the next layer, and then continuing to pass it to the upper layer after the parent node calculates based on the results of all child nodes, until reaching the root node. The transmission can be a data movement process of providing the node processing result of the child node after calculation as input data to the direct parent node, for example, by returning a function value, shared memory or a context object.

[0102] Logical operations refer to semantic computation rules bound to specific grammatical rules (which can correspond to non-terminal nodes, such as "logical AND expression" and "addition expression"). They define the semantics of how to combine the results of child nodes (operands) to obtain the result of the parent node. For example, the logical operation of the "AND" node is "logical AND," and the logical operation of the "addition" node is "arithmetic addition (+)." Composition can refer to the process by which a parent node, after receiving the processing results from all child nodes, applies its own logical operations to compute, reduce, or aggregate these input results, thereby producing a new result.

[0103] The root node can be the top-level node in the syntax tree, without a parent node. It can correspond to the start symbol in the grammar and represent the complete unit of text being parsed. The result passed up can be a new result value generated by the parent node after completing the combination calculation. The result will replace the results of all its child nodes, becoming a single value representing the semantics of the entire subtree under the parent node's jurisdiction, and will continue to be passed up to its parent node.

[0104] The interpretation and execution process can be customized according to specific business needs to meet the requirements of different application scenarios. For example, product recommendation semantics can be built by customizing the structure, including constructing compound conditional expressions based on object attributes, behavioral data, and contextual information. Its semantic elements can include: object profile dimensions, behavioral features (such as purchase preferences), spatiotemporal conditions (such as geographical location), and logical operators (combinations and nesting of AND / OR / NOT).

[0105] For example, recommendation logic can parse object-oriented features using a syntax tree and map them to related nodes in a product knowledge graph. The syntax tree then preliminarily processes the input rules based on a custom symbol set. According to the grammar rule definition, it achieves rapid and automatic conversion between semantics and execution logic, supporting the efficient generation of corresponding tags for application in real-world business scenarios. The implementation library includes: establishing a mapping dictionary from syntax tree nodes to business logic, such as mapping the ">" operator to the gt() method in the comparison function library; designing a dynamic dispatcher to automatically select execution logic based on node type (e.g., automatically distinguishing between numerical and string comparisons); employing just-in-time (JIT) compilation technology to compile high-frequency syntax tree subtrees into bytecode; and pre-compiling and caching repetitive semantic patterns, such as pre-compiling the "age > X" pattern into a parameterized query template.

[0106] According to embodiments of this disclosure, operator precedence and associativity are enforced by strictly adhering to the hierarchical structure of the syntax tree and the bottom-up computation order. When calculating the logical operation of a parent node, all its operands (the results of its child nodes) must have already been calculated, which fully complies with the inherent requirements of mathematical and logical operations and fundamentally avoids semantic deviations caused by incorrect computation order.

[0107] According to an embodiment of the present disclosure, the method further comprises: identifying a frequently accessed syntax tree sub-tree structure during the traversal execution; pre-compiling the calculation logic corresponding to the sub-tree structure into an executable bytecode template; and directly executing the bytecode template to obtain a node processing result when a node matching the sub-tree structure is encountered again during the subsequent traversal.

[0108] In an embodiment of the present disclosure, the syntax tree sub-tree structure can be a complete local tree structure composed of a node and all its descendant nodes in the syntax tree, representing a syntaxically self-contained unit, such as a complex conditional expression, a function call, or a loop body. The calculation logic corresponding to the sub-tree structure can refer to a plurality of specific operation instructions that need to be executed to complete the semantic calculation of a node or a sub-tree in the syntax tree, including a basic operation sequence such as data loading, operation, and conditional judgment, and usually exists in an intermediate representation form during interpretation execution. Pre-compiling can refer to a process of converting a specific calculation logic from a higher-level, interpreted intermediate representation form to a lower-level code form that is closer to machine code and can be directly executed in advance during program execution.

[0109] The bytecode template can be a reusable, parameterized instruction sequence generated by the pre-compiling process. The matching node can refer to a syntax tree node encountered during subsequent traversal, and the sub-tree structure (node type, structure, child node relationship, etc.) led by the node is completely equivalent in form and semantics to the sub-tree structure for which the bytecode template has been previously generated.

[0110] For example, the high-frequency syntax tree sub-tree is compiled into a bytecode template using a just-in-time compilation module. The just-in-time compilation module receives the extracted calculation logic, converts it into a linear, platform-independent bytecode instruction sequence by traversing the logic, converts variable access into reading from a "context object" by name or index because age and other data are external variables, and finally generates a bytecode template and assigns it a unique ID, which is stored in the "hot template cache" with the hash value of its sub-tree structure as the key.

[0111] According to an embodiment of the present disclosure, by pre-compiling the frequently accessed sub-tree calculation logic, a highly optimized bytecode template is generated, so that subsequent execution completely avoids a series of interpretation overheads such as traversal of the original syntax tree, node type judgment, and function call distribution, which can bring performance improvement of orders of magnitude to hot code, especially for rules called frequently or within loops; the entire optimization process (identification, compilation, and replacement) is automatically completed at runtime, without the need for manual annotation or offline compilation by the developer, which dynamically discovers performance bottlenecks and implements optimization according to the actual execution profile, making performance tuning automated and accurate, and suitable for scenarios of dynamically generated or user-defined rules.

[0112] According to an embodiment of the present disclosure, the method further comprises: in the process of parsing the token sequence based on the grammar rule information, caching a successfully matched production rule and its corresponding analysis result; in the case where a subsequent analysis needs to match the same production rule, directly reading the analysis result in the cache.

[0113] In an embodiment of the present disclosure, the successfully matched production rule can be a historical event record that a specific production rule is verified to be consistent with subsequent tokens at a certain specific starting position of the input token sequence at a certain moment in the process of syntax analysis, the record containing the rule itself and the context position at the time of matching. The analysis result can be a specific output generated after the production rule is successfully matched at the specific position; can include the number of tokens consumed in matching; the (sub-) syntax tree node constructed according to the rule; the new state of the parser after successful matching (such as the new input position).

[0114] For example, the parser generation and optimization can include automatically generating efficient parser code or parsing logic based on the user-defined symbol set and grammar rule information. For example: using an improved parsing table expression grammar combined with the following optimizations: caching of parsing results to avoid repeated calculations; hot path prediction, preloading commonly used grammar rules based on historical parsing data; using a compact bitmap to store the symbol set to speed up set operations; using a hierarchical error recovery mechanism to design error handling strategies at the lexical / syntax / semantic layers.

[0115] For example, repeated calculations can be avoided by caching the parsing results. Due to the complexity of the grammar or backtracking (for example, after failing to try another branch rule), the parser can need to try to match the same rule again at the input position; the parser function is called again, the key is calculated and looked up in the cache, this time the lookup hits, there is already a record in the cache corresponding to the key; the parser directly reads the result in the cache (for example, success=true, node=ExpressionNode, next_position=5); the parser immediately returns this result, updates the next_position to 5, and returns the ExpressionNode to the caller, and the entire matching process is completed in an instant without executing any recursive or token consumption logic.

[0116] According to an embodiment of the present disclosure, when parsing a grammar with ambiguity or requiring backtracking (such as a PEG grammar), the same grammar rule can be tried multiple times at the same input position. This scheme reduces the exponential or polynomial time complexity of backtracking overhead to approximately constant time cache lookup operations by caching and directly reading historical matching results, thereby greatly improving the parsing speed of complex grammar structures.

[0117] Based on the above text analysis method, the present disclosure further provides a text analysis device. The following will be described in detail in combination with Figure 4 The device is described in detail.

[0118] Figure 4 The structure block diagram of the text analysis device according to the embodiment of the present disclosure is schematically shown.

[0119] As Figure 4 shown, the text analysis device 400 of the embodiment includes a processing module 410, an analysis module 420 and a selection module 430.

[0120] The processing module 410 is configured to, in response to receiving the text to be analyzed, perform lexical analysis on the text to be analyzed based on a symbol set to obtain a token sequence of the text to be analyzed, wherein the symbol set includes identifiers, operators and delimiters. In an embodiment, the processing module 410 can be configured to perform the operation S210 described above, and details are not repeated here.

[0121] The analysis module 420 is configured to perform syntax analysis on the token sequence based on grammar rule information to obtain a syntax tree, wherein the grammar rule information is determined by: in response to a selection operation on template information, obtaining rule framework information corresponding to the template information and containing a plurality of framework parameters, in the case of detecting a parameterized configuration operation, updating a target parameter in the plurality of framework parameters to obtain the grammar rule information, and the template information is generated by the identifiers, operators and delimiters. In an embodiment, the analysis module 420 can be configured to perform the operation S220 described above, and details are not repeated here.

[0122] The selection module 430 is configured to select a target traversal strategy from a candidate traversal strategy library based on requirement information of a target service corresponding to the text to be analyzed, traverse each node in the syntax tree to obtain a node processing result of each node, and combine the node processing result and a logical relationship between each node in the syntax tree to obtain an analysis result of the text to be analyzed, wherein the requirement information contains at least one of information indicating that the target service is processing a logical expression with a nested structure, and information indicating that the target service is processing batch data in parallel or sequentially. In an embodiment, the selection module 430 can be configured to perform the operation S230 described above, and details are not repeated here.

[0123] According to an embodiment of the present disclosure, based on the processing module 410, the analysis module 420 and the selection module 430 in the text analysis device 400, by providing template information generated by the basic symbol and its corresponding rule framework information containing configurable framework parameters, and generating final grammar rule information when a parameterized configuration operation is detected, the complexity of grammar definition is simplified from writing specific rule code to filling or adjusting key parameters of a preset framework, which significantly reduces the professional threshold and operation complexity of parser construction, improves the intuitiveness and flexibility of use, and enables users to quickly customize parsing logic that meets specific needs without deep compiler technology; in the interpretation execution stage, by dynamically selecting the optimal traversal strategy for traversing and calculating the syntax tree according to the demand information containing specific processing intentions (processing nested logical expression or batch data processing), and combining the logical relationship between nodes to combine node processing results, efficient interpretation and execution of the custom language are realized, which not only improves the processing performance of large-scale or complex rules and reduces the delay, but also enhances the domain adaptation ability of the parsing engine to different business scenarios (such as real-time decision and batch processing) through customizable traversal and calculation strategies.

[0124] According to an embodiment of the present disclosure, the device 400 described above further includes a receiving module, a detection module and an output module. The receiving module is configured to receive symbol definition data input through a visual interface or a configuration file to obtain initial symbols, wherein the symbol definition data includes symbol identification, symbol type and regular expression for matching the symbol; the detection module is configured to detect whether there is a symbol conflict between the newly added symbol and the initial symbol in response to receiving a newly added symbol instruction; and the output module is configured to output a plurality of conflict resolution schemes according to the symbol type and the priority corresponding to the symbol type for user selection in the case that the detection result represents that there is a symbol conflict between the newly added symbol and the initial symbol.

[0125] According to an embodiment of the present disclosure, the processing module 410 includes a construction submodule, an identification submodule and a conversion submodule. The construction submodule is configured to construct a state conversion tool based on the matching mode corresponding to each symbol in the symbol set; the identification submodule is configured to identify the text to be parsed using the state conversion tool to obtain a character sequence corresponding to the matching mode; and the conversion submodule is configured to convert the symbol type to a corresponding mark according to the symbol type corresponding to the character sequence in the symbol set, and arrange the marks in the identification order to obtain a mark sequence.

[0126] According to an embodiment of the present disclosure, the analysis module 420 comprises a matching submodule, a creating submodule and a combining submodule. The matching submodule is configured to take the start symbol defined in the grammar rule information as a root node, and match the syntax component sequence described in each production rule with the corresponding sequence of token types and contents in the token sequence from the start position of the token sequence. The creating submodule is configured to create a parent node corresponding to the production rule in the case of successful matching, and take the child node matched with the production rule as a child node of the parent node. The combining submodule is configured to combine layer by layer upwards until the start symbol is matched to obtain a complete syntax tree.

[0127] According to an embodiment of the present disclosure, the selection module 430 comprises a first selection submodule and a second selection submodule. The first selection submodule is configured to select a depth-first traversal strategy as the target traversal strategy in the case that the requirement information indicates that the target service is processing a logical expression with a nested structure. The second selection submodule is configured to select a breadth-first traversal strategy as the target traversal strategy in the case that the requirement information indicates that the target service is processing batch data in parallel or sequentially.

[0128] According to an embodiment of the present disclosure, the selection module 430 further comprises a function determining submodule and a function calling submodule. The function determining submodule is configured to determine a target processing function corresponding to the current node according to the mapping relationship between the node type of each node in the syntax tree and the processing function. The function calling submodule is configured to call the target processing function to process the node data of the current node and determine the node processing result.

[0129] According to an embodiment of the present disclosure, the selection module 430 further comprises a passing submodule, a result combining submodule and an as submodule. The passing submodule is configured to pass the node processing result of the child node to the corresponding parent node from bottom to top according to the hierarchical structure of the syntax tree. The result combining submodule is configured to combine the node processing results of the child nodes according to the logical operation in the grammar rule corresponding to the parent node, and generate the upward transmission result of the parent node. The as submodule is configured to take the upward transmission result of the root node as the analysis result of the text to be analyzed.

[0130] According to an embodiment of the present disclosure, the above-described device 400 further comprises an identification module, a compiling module and an execution module. The identification module is configured to identify a syntax tree sub-tree structure that is accessed frequently during the traversal execution process. The compiling module is configured to pre-compile the calculation logic corresponding to the sub-tree structure into an executable bytecode template. The execution module is configured to directly execute the bytecode template to obtain the node processing result when the node matching the sub-tree structure is traversed again.

[0131] According to an embodiment of the present disclosure, the apparatus 400 further includes a caching module and a reading module. The caching module is configured to cache a successfully matched production rule and its corresponding analysis result in a process of performing syntax analysis on the token sequence based on the grammar rule information. The reading module is configured to directly read the analysis result in the cache in a case where subsequent analysis needs to match the same production rule.

[0132] According to an embodiment of the present disclosure, the apparatus 400 further includes an information obtaining module and a parameter updating module. The information obtaining module is configured to obtain rule framework information corresponding to the template information in response to a selection operation on the template information, wherein the rule framework information includes a plurality of framework parameters. The parameter updating module is configured to update a target parameter in the plurality of framework parameters to obtain the grammar rule information in response to a parameterized configuration operation.

[0133] According to an embodiment of the present disclosure, any one or more of the processing module 410, the analysis module 420 and the selection module 430 can be combined in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the processing module 410, the analysis module 420 and the selection module 430 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware that can be integrated or packaged, or any one of software, hardware and firmware or any appropriate combination of any of them. Alternatively, at least one of the processing module 410, the analysis module 420 and the selection module 430 can be at least partially implemented as a computer program module which can perform corresponding functions when it is run.

[0134] Figure 5 A block diagram of an electronic device suitable for implementing the text analysis method according to an embodiment of the present disclosure is schematically shown.

[0135] As Figure 5As shown, the electronic device 500 according to embodiments of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 502 or a program loaded into a random access memory (RAM) 503 from a storage section 508. The processor 501 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a dedicated microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 501 can also include an on-board memory for cache use. The processor 501 can include a single processing unit or multiple processing units for performing the various actions of the method processes according to embodiments of the present disclosure.

[0136] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. Note that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0137] According to embodiments of the present disclosure, the electronic device 500 can also include an input / output (I / O) interface 505 that is also connected to the bus 504. The electronic device 500 can further include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read therefrom is installed into the storage section 508 as necessary.

[0138] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which when executed, implement the method according to the embodiments of the present disclosure.

[0139] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories other than the ROM 502 and / or the RAM 503 described above.

[0140] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the text analysis method provided by the embodiments of the present disclosure.

[0141] The above functions defined in the system / apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 501. According to an embodiment of the present disclosure, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0142] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, downloaded and installed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the detachable medium 511. The program codes contained in the computer program can be transmitted by any suitable network medium, including but not limited to wireless, wired, etc., or any suitable combination of the above.

[0143] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0144] According to embodiments of the present disclosure, program code of the computer programs provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using a high-level procedural and / or object-oriented programming language, and / or an assembly / machine language. The programming language includes, but is not limited to, a programming language such as Java, C++, Python, "C" language, or a similar programming language. The program code can be executed entirely on a user computing device, partially on a user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0145] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0146] Those skilled in the art can understand that the features described in various embodiments of the present disclosure can be combined and / or integrated in various combinations, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in various embodiments of the present disclosure can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present disclosure. All such combinations fall within the scope of the present disclosure.

[0147] The embodiments of the present disclosure are described above. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications shall fall within the scope of the present disclosure.

Claims

1. A text parsing method, characterized in that, include: In response to receiving the text to be parsed, lexical analysis is performed on the text to be parsed based on a symbol set to obtain a token sequence of the text to be parsed, wherein the symbol set includes identifiers, operators and delimiters; Based on grammar rule information, the marked sequence is parsed to obtain a syntax tree. The grammar rule information is determined as follows: in response to a selection operation on template information, rule frame information containing multiple frame parameters corresponding to the template information is obtained. When a parameterized configuration operation is detected, the target parameter among the multiple frame parameters is updated to obtain the grammar rule information. The template information is generated by the identifier, the operator, and the delimiter. Based on the requirement information of the target business corresponding to the text to be parsed, a target traversal strategy is selected from the candidate traversal strategy library, and each node in the syntax tree is traversed to obtain the node processing result of each node. The node processing result and the logical relationship between each node in the syntax tree are combined to obtain the parsing result of the text to be parsed. The requirement information includes at least one of the following: information indicating that the target business is to process logical expressions with nested structures, and information indicating that the target business is to perform parallel or sequential processing of batch data.

2. The method according to claim 1, characterized in that, Before performing lexical analysis on the text to be parsed based on the symbol set, the method further includes: Receive symbol definition data input through a visual interface or configuration file to obtain initial symbols, wherein the symbol definition data includes symbol identifier, symbol type and regular expression used to match symbols; In response to receiving a new symbol instruction, detect whether there is a symbol conflict between the new symbol and the initial symbol; If the detection results indicate that there is a symbol conflict between the newly added symbol and the initial symbol, multiple conflict resolution solutions are output according to the symbol type and the priority corresponding to the symbol type for the user to choose from.

3. The method according to claim 1, characterized in that, Lexical analysis of the text to be parsed is performed based on a symbol set to obtain a token sequence of the text to be parsed, including: A state transition tool is constructed based on the matching patterns corresponding to each symbol in the symbol set. The state transition tool is used to identify the text to be parsed, and a character sequence corresponding to the matching pattern is obtained; Based on the symbol type corresponding to the character sequence in the symbol set, the symbol type is converted into a corresponding tag, and the tags are arranged in recognition order to obtain the tag sequence.

4. The method according to claim 1, characterized in that, Based on grammar rule information, the marked sequence is subjected to grammatical analysis to obtain a syntax tree, including: Using the start symbol defined in the grammar rule information as the root node, starting from the beginning position of the tag sequence, the sequence of grammatical components described in each production rule is matched with the tag type and content in the corresponding order of the tag sequence; If a match is successful, a parent node corresponding to the production rule is created, and the child node that matches the production rule is taken as the child node of the parent node. The syntax tree is obtained by combining elements layer by layer upwards until the starting symbol is matched.

5. The method according to claim 1, characterized in that, Based on the target business requirement information corresponding to the text to be parsed, a target traversal strategy is selected from the candidate traversal strategy library, including: When the requirement information indicates that the target business is to process logical expressions with nested structures, a depth-first traversal strategy is selected as the target traversal strategy. When the demand information indicates that the target service is to process batch data in parallel or sequentially, a breadth-first traversal strategy is selected as the target traversal strategy.

6. The method according to claim 1, characterized in that, The node processing results of each node are obtained by traversing each node in the syntax tree, including: Based on the mapping relationship between the node type and the processing function of each node in the syntax tree, determine the target processing function corresponding to the current node. The target processing function is invoked to process the node data of the current node, and the node processing result is determined.

7. The method according to claim 4, characterized in that, By combining the node processing results and the logical relationships between nodes in the syntax tree, the parsing result of the text to be parsed is obtained, including: Based on the hierarchical structure of the syntax tree, the node processing results of the child nodes are passed to the corresponding parent nodes from bottom to top; Based on the logical operations in the syntax rules corresponding to the parent node, the node processing results of the child node are combined to generate the upward propagation result of the parent node; The result of the upward propagation of the root node is used as the parsing result of the text to be parsed.

8. The method according to claim 1, characterized in that, The method further includes: During the traversal process, the syntax tree subtree structures that are accessed most frequently are identified; The computational logic corresponding to the subtree structure is pre-compiled into an executable bytecode template; When traversing to a node that matches the subtree structure again, the bytecode template is executed directly to obtain the node processing result.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: During the grammar rule information-based parsing of the marked sequence, the successfully matched production rules and their corresponding analysis results are cached. If subsequent analysis requires matching the same production rule, the analysis results in the cache are read directly.

10. A text parsing device, characterized in that, The device includes: The processing module is configured to, in response to receiving the text to be parsed, perform lexical analysis on the text to be parsed based on a symbol set to obtain a token sequence of the text to be parsed, wherein the symbol set includes identifiers, operators, and delimiters; An analysis module is used to perform syntactic analysis on the marked sequence based on grammar rule information to obtain a syntax tree. The grammar rule information is determined in the following way: in response to a selection operation on template information, rule frame information containing multiple frame parameters corresponding to the template information is obtained; when a parameterized configuration operation is detected, the target parameter among the multiple frame parameters is updated to obtain the grammar rule information. The template information is generated by the identifier, the operator, and the delimiter. The selection module is used to select a target traversal strategy from the candidate traversal strategy library based on the requirement information of the target business corresponding to the text to be parsed, traverse each node in the syntax tree, obtain the node processing result of each node, and combine the node processing result and the logical relationship between each node in the syntax tree to obtain the parsing result of the text to be parsed. The requirement information includes at least one of the following: information indicating that the target business is to process logical expressions with nested structures, and information indicating that the target business is to perform parallel or sequential processing of batch data.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images