Identification method, device, system, equipment, storage medium and program product

Abstract

The present disclosure provides a recognition method, device, system, equipment, storage medium and program product, relates to the technical field of artificial intelligence, and is used for reducing cost and response time delay while ensuring accuracy in any scene and improving user experience in different scenes. The method comprises the following steps: obtaining an input text; determining an entity recognition path of the input text, wherein the entity recognition path comprises a path of performing entity recognition on the input text based on a rule matching algorithm and / or a large model; and outputting an entity recognition result of the input text based on the entity recognition path.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a recognition method, apparatus, system, device, storage medium, and program product. Background Technology

[0002] With the development of artificial intelligence technology, entity recognition methods have evolved from rule matching and statistical machine learning to deep learning and large language models. However, existing technologies often suffer from problems in entity recognition applications, such as low accuracy, high mismatch rate, slow processing speed, high resource consumption, and inability to handle incomplete entity information. Summary of the Invention

[0003] This disclosure provides an identification method, apparatus, system, device, storage medium, and program product, which can ensure accuracy in any scenario while reducing costs and response latency, and improving user experience in different scenarios.

[0004] In a first aspect, an identification method is provided in the embodiments of this disclosure, the method comprising: Get the input text; Determine the entity recognition path of the input text, wherein the entity recognition path includes the path of performing entity recognition on the input text based on rule matching algorithms and / or large models; The entity recognition result of the input text is output based on the entity recognition path.

[0005] As one possible implementation, determining the entity recognition path of the input text includes: The input text is used to perform entity recognition using a rule-matching algorithm to obtain a candidate entity list; The entity recognition path is determined based on the quality score of the candidate entity list.

[0006] As one possible implementation, the response latency of the entity recognition path to entity recognition of the input text is related to the quality score.

[0007] As one possible implementation, the quality score of the candidate entity list is determined in the following manner: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the degree of difference in matching scores between different candidate entities.

[0008] As one possible implementation, determining the quality score of the candidate entity list includes: The quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest matching score and the second highest matching score. The quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score.

[0009] As one possible implementation method, The entity recognition path is a path for performing entity recognition on the input text based on a rule matching algorithm, and the entity recognition result output by the entity recognition path is one or more candidate entities in the candidate entity list.

[0010] As one possible implementation method, The entity recognition path is to first perform entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then input the candidate entity list into a large model for entity recognition. The entity recognition result output by the entity recognition path is the output result of the large model.

[0011] As one possible implementation method, The number of candidate entities in the candidate entity list is within a specified range. The large model is used to filter entities from the candidate entity list that match the input text. The specified range is used to limit the length of the text input to the large model; and / or, The response latency of the entity recognition path to entity recognition of the input text is related to the number of candidate entities in the candidate entity list.

[0012] As one possible implementation, the entity recognition path is a path for performing entity recognition on the input text based on a large model, and the entity recognition result output by the entity recognition path is the output result of the large model; or, The entity recognition path is a path that first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm. The entity recognition result output by the entity recognition path is the corrected result.

[0013] As one possible implementation method, The entity recognition path consists of a first path and a second path executed in parallel. The first path is the path obtained by performing entity recognition on the input text based on a rule matching algorithm, and the second path is the path obtained by performing entity recognition on the input text based on a large model. The entity recognition result output by the entity recognition path is a fusion result of the first recognition result output by the first path and the second recognition result output by the second path.

[0014] As one possible implementation, the method further includes: The entity recognition path is updated based on the constraints of cost and / or response latency for performing entity recognition on the input text using the entity recognition path.

[0015] As one possible implementation, the rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0016] As one possible implementation method, When the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels. As the level increases, the minimum value of the matching score range decreases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0017] As one possible implementation method, Update at least one of the following in the rule matching algorithm: number of levels, matching score range for each level, and matching type for each level.

[0018] Secondly, this disclosure provides an identification method, which includes: Get the input text; The input text is subjected to entity recognition using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0019] As one possible implementation method, Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches the rule at level i, the entity recognition result is determined based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0020] Thirdly, embodiments of this disclosure provide an identification device, which includes a processing unit for performing the steps of the method of the first or second aspect.

[0021] Fourthly, embodiments of this disclosure also provide an interactive system, including any of the identification device, processing device, and display device described in the third aspect above, wherein: The processing device is configured to perform corresponding processing using the entity recognition result output by the recognition device, and transmit the processing result to the display device for display.

[0022] Fifthly, embodiments of this disclosure also provide an electronic device that includes any of the identification devices described in the third aspect.

[0023] Sixthly, embodiments of this disclosure also provide an identification device, the device comprising: The input module is used to obtain the input text; The path control module is used to determine the entity recognition path of the input text, wherein the entity recognition path includes the path for entity recognition of the input text based on rule matching algorithms and / or large models; The output module is used to output the entity recognition result of the input text based on the entity recognition path.

[0024] In a seventh aspect, embodiments of this disclosure also provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the method described in any of the first aspects above.

[0025] Eighthly, this disclosure provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of the first aspects.

[0026] These or other aspects of this disclosure will become more apparent in the following description of embodiments. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the implementation of an identification method provided in this disclosure; Figure 2A flowchart illustrating the implementation of a rule matching algorithm provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating the implementation of a named entity recognition method provided in this disclosure embodiment; Figure 4 A flowchart illustrating the implementation of a named entity recognition scenario provided in this disclosure. Figure 5 A schematic diagram of an entity recognition system architecture provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of entity recognition data stream transmission in an SQL generation scenario provided by an embodiment of the present disclosure; Figures 7A-7C An interface diagram of a question-and-answer interaction system provided in an embodiment of this disclosure; Figure 8 A schematic diagram of an interactive system provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram of an identification device provided in an embodiment of the present disclosure. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0030] In this disclosure, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0031] The application scenarios described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this disclosure are also applicable to similar technical problems. In the description of this disclosure, unless otherwise stated, "multiple" means two or more.

[0032] Before introducing the identification method provided in the embodiments of this disclosure, for ease of understanding, the technical background of the embodiments of this disclosure will be described in detail below.

[0033] Entity recognition is a fundamental natural language processing task that automatically extracts key semantic units from unstructured text and categorizes them. It is a prerequisite for machine understanding of text, information extraction, and knowledge graph construction. Named Entity Recognition (NER) is the most commonly used and basic subset of entity recognition. NER is one of the fundamental tasks in the field of natural language processing. NER extracts proper nouns such as names of people, places, organizations, times, amounts, and dates. Its aim is to identify entities with specific meanings from text, such as names of people, places, organizations, and products. The role of entity recognition (such as named entity recognition) is to automatically extract key information from unstructured text (such as user questions, sentences, product manuals, test reports, articles, chat logs, contracts, etc.), transforming it into structured data so that machines can understand what key objects are present in the text. Specifically, it can perform information filtering, retaining only key content such as names, locations, times, organizations, products, and technical terms. It can achieve semantic understanding, allowing the machine to identify which parts are names, locations, and business terms. It can perform structured transformation, converting text into tables of [entities + categories] for easy storage, retrieval, statistics, and analysis. Entity recognition (such as named entity recognition) is the foundation for downstream tasks and a prerequisite for information extraction, intelligent question answering, knowledge graph construction, and machine translation. It can also perform information anonymization, automatically identifying confidential / privacy data such as phone numbers, ID cards, names, addresses, and product parameters, and performing privacy masking.

[0034] With the development of artificial intelligence technology, named entity recognition methods have evolved from rule matching and statistical machine learning to deep learning and large language models. Currently, named entity recognition methods can be mainly categorized as follows, but all have varying degrees of shortcomings: (a) Rule-based matching method This type of method extracts entities from text using predefined regular expressions, dictionary matching, and other rules. Although rule-based methods are highly interpretable and do not require training data, they have the following shortcomings: (1) The rule design is complex and it is difficult to cover all edge cases; (2) There is a lack of flexible matching mechanisms, and the ability to handle changes in format and spelling errors is weak; (3) There is a lack of scoring mechanisms, and it is impossible to distinguish the quality of matching; (4) When dealing with complex entity formats (such as product models containing combinations of numbers, letters, and special characters), a large number of specific rules need to be written, resulting in high maintenance costs; (5) It is impossible to identify complete entity information (such as "36.6 FB") from incomplete entity information (such as "36.6"), and complete entity information is required for matching.

[0035] (II) Deep Learning-Based Methods

[0036] This type of method uses neural network models (such as BiLSTM-CRF (Bidirectional Long Short-Term Memory) and BERT (Bidirectional Encoder Representations from Transformers)) for end-to-end entity recognition. Although deep learning methods perform well on some standard datasets, they have the following shortcomings in practical applications in the field of quality detection: (1) They require a large amount of labeled training data, which is not friendly to cold start scenarios or new fields; (2) The model is a black box, with poor interpretability and is difficult to debug and optimize; (3) They have limited processing capabilities for special formats in Chinese environments (such as full-width and half-width characters, numbers + Chinese characters); (4) Model inference requires GPU (Graphics Processing Unit) resources, and the processing speed is relatively slow; (5) They have poor domain adaptability and need to retrain the model when the entity format changes; (6) They have weak recognition capabilities for incomplete entity information and require complete entity information for accurate recognition.

[0037] (iii) The method of directly using a large language model (LLM).

[0038] This type of method utilizes the powerful language understanding capabilities of large language models such as GPT (Generative Pre-trained Transformer) and Qwen (Tongyi Qianwen) for entity recognition. Although large language models have strong generalization capabilities, they have the following shortcomings: (1) Each recognition requires calling LLM, which consumes a lot of computational resources and has a slow processing speed (usually requiring a response time of seconds); (2) For large models deployed locally, a large amount of input text needs to be processed, resulting in high computational complexity and high resource consumption; (3) For enterprise-level applications, frequent calls to LLM will generate high computational costs; (4) LLM is a black box model, and the recognition results are difficult to interpret, which is not conducive to problem localization and optimization in industrial applications; (5) When it is necessary to select from a large number of candidate entities, all candidates need to be provided to LLM, which significantly increases the length of the input text and further exacerbates the computational complexity problem; (6) The ability to process incomplete entity information is limited. Although LLM has a certain reasoning ability, when processing a large number of candidates, a complete enumeration list needs to be provided, resulting in high computational complexity.

[0039] (iv) Simple methods for combining rules and deep learning

[0040] Although such methods attempt to combine the advantages of rules and models, they have the following deficiencies: (1) There is a lack of deep integration between rules and models, and it is just a simple pre- and post-processing relationship; (2) There is a lack of an intelligent decision-making mechanism and it is unable to dynamically select whether to use rules or models according to specific situations; (3) It is unable to fully utilize the high efficiency of rule matching and the generalization ability of models; (4) There is no decision-making mechanism and it is difficult to achieve a balance between cost and accuracy; (5) There is a lack of the ability to identify complete entities from incomplete information.

[0041] (5) Special Challenges in the Field of Quality Inspection

[0042] In the actual application in the field of quality inspection, named entity recognition faces more special challenges. First, the product model formats are complex and diverse, usually including combinations of letters, numbers, and special characters (such as "21.45 FHD SNB 100Hz", "23.6+0", etc.), and there may be format variations such as spaces and parentheses. Second, the bad name formats are diverse, including Chinese (such as "dark spot", "bright spot"), English (such as "Bad / Shift Via DE", "Broken(MDL)-SQA"), and there are also cases of Chinese-English mixtures. Third, special processing requirements in the Chinese environment, such as the unified processing of full-width and half-width characters ("()" and "()"), the overall recognition of the combination of numbers + Chinese ("45 degrees", "10.36 processes"), etc. Fourth, there are a large number of intention expression phrases (such as "analysis of bad situations") in the field of quality inspection, which need to be distinguished from actual entity names. Fifth, users often only provide partial entity information in actual use (such as only providing "36.6" instead of the complete "36.6 FB", only providing "45 degrees" instead of the complete "45-degree oblique direction stripe Mura"), and the existing technologies lack the ability to identify complete entities from incomplete information, resulting in recognition failures or misidentifications.

[0043] When faced with these specific challenges, existing technologies often suffer from problems such as low recognition accuracy, high mismatch rate, slow processing speed, high resource consumption, and inability to handle incomplete entity information. In summary, existing technologies, especially in the application of named entity recognition in the field of quality inspection, have the following main shortcomings: (1) Pure rule-based methods lack flexible matching mechanisms and scoring systems, making it difficult to handle complex entity format changes and unable to identify complete entity information from incomplete entity information; (2) Deep learning methods require a large amount of training data, have poor interpretability, limited ability to handle special Chinese formats, and weak ability to identify incomplete entity information; (3) Pure LLM methods require calling the model every time, resulting in high computational resource consumption, slow processing speed, and high cost. When processing a large number of candidate entities, a complete enumeration list is required, which significantly increases the length of the input text and further exacerbates the computational complexity; (4) The simple combination of rules and models lacks deep integration and intelligent decision-making mechanisms, making it impossible to achieve a balance between cost and accuracy, and lacking the ability to identify complete entities from incomplete information; (5) Existing methods generally lack optimization for the special needs of the quality inspection field, such as unified processing of full-width and half-width characters, recognition of number + Chinese combination, and intent phrase filtering, especially lacking the ability to identify complete standard entity information from incomplete entity information. Addressing the shortcomings of existing technologies in named entity recognition (MID) applications in the quality inspection field, the following technical issues urgently need to be resolved: how to reduce the cost of MID and avoid the high computational costs caused by frequent LLM calls; how to improve the processing speed of MID to achieve millisecond-level response and improve user experience; how to improve the accuracy of MID while ensuring speed; how to identify complete standard entity information (such as "36.6 FB", "45 degrees", "UD") from incomplete entity information (such as "36.6", "45 degrees", "UD"); how to achieve deep integration of rule matching and LLM, rather than a simple pre- and post-processing relationship; how to optimize for the special needs of the quality inspection field; and how to reduce the length of input text and reduce computational complexity when LLM calls are required.

[0044] To address the shortcomings of the prior art, this disclosure provides an identification method with an intelligent decision-making mechanism that effectively combines rule matching and the advantages of large models to select the optimal entity recognition path. It also takes into account accuracy, speed, and cost, significantly reducing the number of LLM calls, lowering overall costs while ensuring high accuracy. This method optimizes cost, improves speed, and enhances accuracy.

[0045] like Figure 1As shown, the recognition method provided in this embodiment can be applied to various scenarios, including but not limited to any one or more of the following: SQL (Structured Query Language) generation scenario, information extraction scenario, knowledge graph construction scenario, intelligent question answering scenario, and text analysis scenario. Taking the SQL generation scenario as an example, a specific application scenario is the yield ChatBI (Conversational Business Intelligence) system in an industrial knowledge question answering system. Users input natural language questions through a web interface (e.g., "Query the situation of dark spot defects in product model 36.6 in the first quarter of 2024"). That is, the user inputs text through the web interface. Based on the recognition method of this disclosure, the entity recognition path of the input text is determined, and the entity recognition result of the text is output based on the entity recognition path. Then, the SQL generation module maps the entity recognition result into an SQL statement, the SQL executor executes the query, the business database returns the result, and the result is displayed through the web interface. The business data flow is: Web interface → Algorithm server (recognition method of this disclosure) → SQL generation module → SQL executor → Business database → Return result → Web interface. The algorithm server is used to execute the recognition method provided by this disclosure. In information extraction scenarios: Replace the "SQL generation module" and "SQL executor" with the "information extraction module" and "knowledge base storage," while keeping the algorithm server unchanged. In knowledge graph construction scenarios: Replace the "SQL generation module" and "SQL executor" with the "relationship extraction module" and "graph storage," while keeping the algorithm server unchanged. In intelligent question answering scenarios: Replace the "SQL generation module" and "SQL executor" with the "answer generation module" and "knowledge base retrieval," while keeping the algorithm server unchanged. In text analysis scenarios: Replace the "SQL generation module" and "SQL executor" with the "text analysis module" and "result storage," while keeping the algorithm server unchanged. Regardless of the application scenario, the recognition method executed in the algorithm server is universal, possessing versatility and scalability.

[0046] like Figure 1 As shown in the figure, this embodiment provides an identification method, and the specific implementation process of the method is as follows: Step 100: Obtain the input text; Optionally, the user's input text can be obtained via a web interface (web page). Optionally, the input text can be natural language text. In one example, the user can input text (such as a question) via a web interface, or input voice information via a web interface, convert the voice information into text, and use this text as the input text. Users can also input images via a web interface, extract text from the images, or input audio / video via a web interface, convert the audio / video into text, and use this text as the input text. This embodiment does not impose excessive limitations on the specific form of user input. The recognition method provided in this embodiment has the ability to convert non-text input data into text. When the user inputs non-text data, the non-text data is converted into text data, and the converted text data is used as the input text.

[0047] As one possible implementation, after obtaining the input text, the input text can be preprocessed, including but not limited to any one or more of the following: word segmentation, noise reduction, encoding standardization, etc., to provide a unified, standard, and normalized input for any subsequent entity recognition path.

[0048] Step 101: Determine the entity recognition path of the input text, wherein the entity recognition path includes the path for performing entity recognition on the input text based on rule matching algorithms and / or large models; This embodiment provides a strategy for dynamically determining entity recognition paths. Different entity recognition paths are selected based on different strategies. Entity recognition can be performed solely based on rule-matching algorithms, or it can combine rule-matching algorithms with a large model. The rule-matching algorithm and the large model can be implemented sequentially or in parallel, or entity recognition can be performed solely based on the large model, allowing for flexible selection of different entity recognition paths. Optionally, the entity recognition in this embodiment includes any or more of the following: Named Entity Recognition (NER), attribute entity recognition, numerical entity recognition, relational entity recognition, event entity recognition, etc.

[0049] The entity recognition path provided in this application is a complete execution link and processing flow formed by sequentially or in combination employing a rule-matching algorithm and a large model to perform entity recognition tasks on input text. This entity recognition path starts with input text and completes the entity recognition process by individually calling the rule-matching algorithm, individually calling the large model, or jointly calling both the rule-matching algorithm and the large model. The entity recognition path includes a first node and a second node, as well as the order in which the first and second nodes are called. The first node is the rule-matching algorithm, and the second node is the large model; either the first or second node is used to perform the entity recognition task. Optionally, the entity recognition path provided in the exemplary embodiments of this application includes: Path 1, including a first node, inputting input text into the first node and outputting entity recognition results; Path 2, including a second node, inputting input text into the second node and outputting entity recognition results; Path 3, including a first node and a second node, inputting input text into the first node first, connecting the output interface of the first node to the input interface of the second node, and outputting entity recognition results by the second node; Path 4, including a first node and a second node, inputting input text into the second node first, connecting the output interface of the second node to the input interface of the first node, and outputting entity recognition results by the first node; Path 5, including a first node, a second node, and a fusion node, connecting the output interface of the first node to the input interface of the fusion node, connecting the output interface of the second node to the input interface of the fusion node, inputting input text into the first node and the second node respectively, inputting the output results of the first node and the second node into the fusion node, fusing the output results of the first node and the second node using the fusion node, and outputting entity recognition results by the fusion node.

[0050] In one example, the entity recognition path for the current round can also be dynamically adjusted based on the entity recognition results output from the previous round of entity recognition path.

[0051] Optionally, the large model in this embodiment includes, but is not limited to, LLM, and / or, multimodal large models (large models integrating text, images, audio, and video). Examples of large models include Wenxin Yiyan, Tongyi Qianwen, the GPT series, Claude, Mistral, etc. The large models in this embodiment include, but are not limited to, lightweight LLM, full-scale LLM, etc.

[0052] Step 102: Output the entity recognition result of the input text based on the entity recognition path.

[0053] In practice, the processing flow for entity recognition of input text differs based on different entity recognition paths, resulting in different final entity recognition results. Different entity recognition paths reflect different business needs, aiming to select the optimal entity recognition path for the corresponding business needs, thereby satisfying those needs.

[0054] Optionally, the business requirements in this embodiment include, but are not limited to, prioritizing at least one of the following for entity recognition paths when performing entity recognition on the input text: accuracy, cost, and response latency. For example, if the business requirement is accuracy first, accuracy is considered over cost and response latency, and the most suitable entity recognition path is selected based on this business requirement. Similarly, if the business requirement is cost first, cost is considered first, and the most suitable entity recognition path is selected based on this business requirement. If the business requirement is response latency, response latency is considered first, and the most suitable entity recognition path is selected based on this business requirement.

[0055] This embodiment provides the following five entity recognition paths: Path 1, only the first path, is the complete execution link and processing flow of entity recognition of input text based on rule matching algorithm; that is, the entity recognition path is the path of entity recognition of input text based on rule matching algorithm. The entity recognition path outputs one or more candidate entities in the candidate entity list, and the one or more candidate entities in the candidate entity list are used as the entity recognition result output by the entity recognition path.

[0056] Optionally, if the candidate entity list includes multiple candidate entities, the candidate entity with the highest matching score among the multiple candidate entities is output as the entity recognition result of the first path.

[0057] Path 2, only the second path, is the complete execution link and processing flow of entity recognition of input text based on a large model; that is, this entity recognition path is the path of entity recognition of input text based on a large model, and the output of this entity recognition path is the output result of the large model, and the output result of the large model is used as the entity recognition result output by this entity recognition path.

[0058] Path 3, First Path → Second Path: This entity recognition path first performs entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then inputs the candidate entity list into a large model for entity recognition. The output of this entity recognition path is the output result of the large model, which is used as the entity recognition result output by this entity recognition path.

[0059] In one possible example, the number of candidate entities in the candidate entity list is within a specified range, and the large model is used to filter entities from the candidate entity list that match the input text, the specified range being used to limit the length of the text input to the large model.

[0060] For example, a rule-based matching algorithm is used to filter matching candidate entities from the full entity list. The candidate entities are then sorted according to their matching scores, from highest to lowest, ensuring the most matching candidate entity is at the top. The top 3-5 candidate entities are selected to generate a candidate entity list. Limiting the number of candidate entities to 3-5 ensures sufficient selection space while avoiding excessively long text input to the large model. Optionally, prompts are constructed based on the candidate entity list instead of based on a full enumeration of entities. Currently, large models typically provide a full enumeration of entities for entity recognition. For example, with 1000 product models, the prompt text input to the large model would be approximately 50,000 characters long. However, after optimization using this method, only 3-5 candidate entities are provided, and the prompt text input to the large model is approximately 500 characters long, reducing the input text length by about 99%. This significantly reduces response latency while maintaining accuracy, and the use of a lightweight large model saves costs.

[0061] In one possible example, the response latency of the entity recognition path in performing entity recognition on the input text is related to the number of candidate entities in the candidate entity list. The fewer the number of candidate entities, the shorter the length of the prompt text input to the large model, the less inference time the large model takes, and the lower the response latency.

[0062] Taking the large model LLM as an example, assuming the full list of product models contains 1000 items, and the user's input text is "Query product model 36.6". Before optimization (using full enumeration): the input prompt text provided to LLM contains a complete list of 1000 product models, and the length of the input prompt text is approximately 50,000 characters. LLM processing complexity is high, and computational resource consumption is large. Using the first path → second path scheme provided in this embodiment, prompts are generated using a candidate entity list, and the number of candidate entities is limited for rule pre-screening: from the 1000 product models, the 3 most matching candidate entities ["36.6 FB", "36.6 HD", "36.8 FB"] are selected. The input prompt text provided to LLM only contains 3 candidate entities, and the length of the input prompt text is approximately 500 characters, reducing the length of the input prompt text by approximately 99%. LLM only needs to select from 3 candidate entities instead of from the full list of 1000 entities. LLM processing complexity is low, computational resource consumption is small, and recognition accuracy is improved. Through the above implementation methods, the embodiments of this disclosure realize a hybrid architecture of rule matching algorithm and large model intelligent decision-making, which significantly reduces costs and increases speed while ensuring recognition accuracy, and realizes the ability to identify complete entities from incomplete information.

[0063] Path 4, Second Path → First Path, that is, this entity recognition path first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm; the output of this entity recognition path is the corrected result, that is, the corrected result is used as the entity recognition result output by this entity recognition path.

[0064] Path 5, First Path || Second Path, refers to the entity recognition path where the first path and the second path are executed in parallel. The output of this entity recognition path is a fusion of the first recognition result from the first path and the second recognition result from the second path. The first path and the second path perform entity recognition in parallel. Then, the first recognition result from the first path and the second recognition result from the second path are fused together, and the fused result is used as the entity recognition result output by this entity recognition path.

[0065] Optionally, the fusion in this embodiment includes, but is not limited to, any one or more of the following: ① Merging fusion, that is, merging the first recognition result and the second recognition result to obtain the fusion result; ② Priority fusion, that is, based on the priority order of the first path and the second path, the recognition result with higher priority directly covers the recognition result with lower priority. If the priority of the first path is higher than that of the second path, then the first recognition result is used as the fusion result; ③ Weighted fusion, assigning weights to the first path and the second path, and performing weighted summation based on the respective weights of the first path and the second path to obtain the fusion result; ④ Learning fusion, automatically learning the optimal weights / policies through data training, performing weighted fusion based on the optimal weights, and performing fusion based on the optimal policy. The types of learning fusion include, but are not limited to, any one or more of the following: attention fusion, MLP (Multi-Layer Perceptron) fusion, gating fusion, feature alignment fusion, etc.

[0066] In one possible implementation, an exemplary embodiment of this disclosure provides a strategy for intelligently selecting entity recognition paths based on quality scores, specifically as follows: The input text is subjected to entity recognition using a rule-based matching algorithm to obtain a candidate entity list; the entity recognition path is determined based on the quality score of the candidate entity list.

[0067] The candidate entity list in this embodiment includes one or more candidate entities. This candidate entity list is a summary list of candidate entities output after entity recognition of the input text using a rule-matching algorithm. Each candidate entity in the list can be associated with attributes such as entity content (entity value), matching type, matching score, and matching position, representing the entity identified from the input text. That is, the candidate entity list can also serve as the entity recognition result of the input text. For example, if the input text is "Querying for dark spot defects in product model 36.6", the rule-matching algorithm performs entity recognition on the input text, finding matching product models. For example, 36.6 might match "36.6FB" or "36.6HD". Therefore, "36.6FB" and "36.6HD" are both considered candidate entities, and the candidate entity list includes "36.6FB" and "36.6HD".

[0068] In this embodiment, after obtaining the input text, a rule-based matching algorithm is first used to perform entity recognition on the input text. Then, the quality of the candidate entity list is evaluated. Based on the quality score of the candidate entity list, the following options are selected: whether to use the candidate entity list output by the rule-based matching algorithm, whether to call LLM, and the combination of the rule-based matching algorithm and LLM. The quality score of the candidate entity list reflects the accuracy of the rule-based matching algorithm in recognizing entities in the input text. If the quality score is high, it means that the candidate entity list has a high degree of matching / relevance with the entities in the input text, and the candidate entity list is highly reliable. In this case, it is not necessary to call LLM to directly output the candidate entity with the highest matching degree as the final entity recognition result. If the quality score is low, it means that the candidate entity list has a low degree of matching / relevance with the entities in the input text, and the candidate entity list is less reliable. In this case, it is possible to call LLM to further identify the entity that best matches the input text based on the candidate entity list as the final entity recognition result. If the quality score is low enough to indicate that the candidate entity list is unreliable, entity recognition can also be performed using only LLM.

[0069] In some embodiments, the entity recognition path is selected based on the quality score in the following manner: The entity recognition path is determined based on the quality interval in which the quality score is located; the quality interval may include one or more, and one quality interval corresponds to one or more entity recognition paths.

[0070] For example, multiple quality ranges can be set to select different entity recognition paths. One quality range is greater than or equal to 0.9, and another is less than 0.9. When the quality score is ≥0.9 (high quality), path one is used, which uses a rule-based matching algorithm to output the candidate entity with the highest matching score from the candidate entity list as the entity recognition result of path one. In this scenario, the large model is completely avoided, achieving millisecond-level response (typically <100ms) and reducing processing costs by 60-80%. When the quality score is <0.9 (medium to low quality), path three is used, i.e., path one → path two. Based on the candidate entity list output by path one, prompt words are constructed and input into the large model for entity recognition. The input to the large model is a rule-pre-filtered candidate entity list (typically 3-5 candidate entities), rather than full enumeration, reducing the input prompt word text length by 80-90%. The large model is called to precisely select from the candidate entity list. The large model only needs to select from a small number of candidate entities, effectively reducing the error rate, improving recognition accuracy, and reducing response latency.

[0071] In one possible example, the response latency of the entity recognition path for entity recognition of input text is related to the quality score. The quality score determines whether a large model is needed for entity recognition. When a large model is not needed, entity recognition based solely on the rule-matching algorithm can respond quickly, significantly reducing response latency and cost. When both the rule-matching algorithm and a large model are needed for entity recognition, the candidate entity list output by the rule-matching algorithm can be used as input to the large model, reducing the length of the input text, improving the response speed of the large model, and reducing the overall response latency of entity recognition. It should be noted that the selection of the entity recognition path in this disclosure is real-time. When a user's input text is obtained, the corresponding entity recognition path is selected for that input text. When the next input text arrives, the corresponding entity recognition path is selected for that next input text. This ensures that the most suitable entity recognition path is selected for each input text. By evaluating the quality of rule-matching based on different input texts and flexibly selecting the entity recognition path, the accuracy of recognition is guaranteed while reducing response latency and cost. This application uses a rule-based matching algorithm for entity recognition, which can achieve millisecond-level response (typically <100ms), greatly reducing response latency and being much faster than the second-level response of pure LLM solutions.

[0072] In one example, under a high-quality scenario, the input text is: "Query products with model number 36.6 FB". The output of the rule matching algorithm is: number of candidate entities = 1, highest matching score = 100 points. The quality score of this candidate entity list is: quantity factor of candidate entity number = 1.0, score factor of matching score = 1.0, difference factor of difference = 1.0, and the weight of each factor is summed to obtain the comprehensive quality score = 1.0. At this time, the quality score is ≥ 0.9, and the output of the rule matching algorithm "36.6 FB" is used directly, with zero large model calls, achieving millisecond-level response.

[0073] In another example, in a low-to-medium quality scenario, the input text is: "Query products of model 36.6". The output of the rule matching algorithm is: number of candidate entities = 3, candidate entity list = ["36.6 FB" (matching score 70 points), "36.6HD" (matching score 65 points), "36.8 FB" (matching score 60 points)], and the quality score of the candidate entities is: quantity factor of candidate entity number = 0.9, score factor of matching score = 0.7, difference factor of difference = 0.8, and the weighted sum of each factor yields a comprehensive quality score of 0.79. At this point, the quality score is <0.9. The large model is invoked, selecting path three (path one → path two). The candidate entity list is used as input to the large model, providing only three candidate entities ["36.6FB", "36.6 HD", "36.8 FB"] instead of a full enumeration (assuming a total of 1000 product models). This reduces the input text length by approximately 99.7%, improving inference speed and reducing response latency while maintaining accuracy. This embodiment can be designed with two levels (directly using rule results in high-quality scenarios, and invoking LLM in medium-to-low-quality scenarios) or more. This embodiment does not impose excessive limitations on this.

[0074] In one possible implementation, whether the entity recognition path in this embodiment of the disclosure contains a large model is related to the quality score; the quality score is used to select whether the entity recognition path contains a large model and whether it is necessary to call the large model for entity recognition.

[0075] In one possible implementation, the weight parameters of the large model in the entity recognition path are related to the quality score. Optionally, the parameters of the large model include, but are not limited to, any one or more of the following: storage precision (quantization bits) of the weight parameters, number of weight parameters, model compression method, deployment environment, inference speed, model accuracy, running speed, and memory usage. For example, based on the parameters of the large model, the large model can be divided into a lightweight model and a full model. The size of the quality score can determine the selected parameters of the large model. For example, in practical applications, a multi-level decision mechanism can be designed according to the needs of specific scenarios. For example, a three-level decision mechanism can be designed: when the quality score is ≥0.95, the result obtained by the rule matching algorithm is directly used; when the quality score is in the quality interval [0.85, 0.95], path three (first path → second path) can be selected, where the large model in the second path can be a lightweight LLM; when the quality score is <0.85, path three (first path → second path) can be selected, where the large model in the second path can be a full LLM.

[0076] In some embodiments, exemplary embodiments of this disclosure may also update the maximum and / or minimum values ​​of the quality range; the update of the quality range is related to business requirements, which include a priority ranking of at least one of the accuracy, cost, and response latency of entity recognition paths for entity recognition of input text.

[0077] In implementation, 0.9 is used as the dividing threshold between high quality and medium-low quality, but the embodiments disclosed herein are not limited to this dividing threshold. In practical applications, the maximum and / or minimum values ​​of the quality range can be flexibly adjusted according to the specific needs of the scenario, cost control requirements, accuracy requirements, and other factors. For example, for cost-sensitive scenarios, the dividing threshold can be increased to 0.95. When the quality score is greater than or equal to 0.95, Path 1 (the first path) is used for entity recognition; when the quality score is less than 0.95, Path 3 (the first path → the second path) is used for entity recognition, making more use of rule matching results. For scenarios with high accuracy requirements, the dividing threshold can be decreased to 0.85. When the quality score is greater than or equal to 0.85, Path 1 (the first path) is used for entity recognition; when the quality score is less than 0.85, Path 3 (the first path → the second path) is used for entity recognition, making more use of large models for precise selection.

[0078] In one example, the quality range is adjusted based on different business requirements. After the adjustment, the entity recognition path selection strategy remains unchanged, but the baseline of the quality range will shift. The specific adjustment method is as follows: When the business requirement prioritizes accuracy, the quality range is updated from a quality score ≥ 0.9 to a quality score ≥ 0.95 (improvement). When the quality score is between (0.90 and 0.95), the entity recognition path selected before the update is path one. After the quality range is updated, path three is selected. After using the rule matching algorithm for entity recognition, a large model is used for verification.

[0079] When the business requirement is cost-first, the quality range is updated from quality score ≥ 0.9 to quality score ≥ 0.80 (reduced). When the quality score is (0.80, 0.90), the entity recognition path selected before the update is path three. After the quality range is updated, path one is selected, and the large model is skipped.

[0080] When business needs are balanced, the quality range is the default range. For example, the quality range is a quality score ≥ 0.9 and a quality score < 0.90.

[0081] In one possible example, when the business requirement prioritizes latency, and the quality range is a quality score ≥ 0.90, the quality range is not updated. Instead, the response latency constraint is updated from the maximum allowable latency T_max_user = 1~5s to 0.8~1.2s. When the business requirement prioritizes latency and the quality score < 0.90, the current path three selected based on the quality score can be executed asynchronously. That is, the result of the rule matching algorithm (first path) is returned first, and then the entity recognition of the large model (second path) is executed in the background.

[0082] As an optional implementation, a three-level strategy mechanism can be set up, configuring three quality intervals. When the quality score is ≥0.90, path one is selected, meaning the first path only involves the rule matching algorithm for entity recognition, with zero calls to the large model. In this scenario, it indicates that the rule matching result is reliable, and there is no need to waste the resources of the large model. When the quality score is in the range of [0.70, 0.90], path three is selected, i.e., the first path → the second path. The input of the large model is optimized, only inputting a limited number of candidate entity lists output by the first path. On the one hand, the rule matching algorithm provides a valuable list of candidate entities (3-5 candidate entities), and the large model only needs to select the most matching entity from the candidate entity list, reducing the cost by 80-90%. When the quality score is <0.70, path two, path three, or path four can be selected. If the candidate entity list still has reference value, path three is selected, and the number of candidate entities in the candidate entity list can be limited to 5-8. If the candidate entity list is not empty, and the highest matching score S_max ≥ 60 points, it indicates that the candidate entity list has no reference value at this time, and path two is selected, using only the large model for entity recognition without relying on the rule matching algorithm. If the candidate entity list is empty, or the highest matching score S_max in the candidate entity list is less than 60, it means that all rule matching has failed. Alternatively, the highest matching score is lower than the preset lower limit. In this case, the quality of the candidate entity list is poor, but the rules are expected to provide accurate verification. In this case, path four can be selected, i.e., the second path → the first path. The large model is used for coarse recognition first, and then the rule matching algorithm is used to refine / complete / standardize the entity recognition results of the large model.

[0083] As an optional implementation, an exemplary embodiment of this disclosure updates the entity recognition path based on the constraints of the cost and / or response latency of performing entity recognition on the input text using the entity recognition path.

[0084] In one example, cost and latency can be used as hard constraints to cover the entity recognition path selected based on the quality score. In practice, after selecting the recognition path based on the quality score, the cost constraint is used to re-evaluate whether the already selected entity recognition path needs to be updated.

[0085] In one possible example, based on the constraint of the cost of entity recognition on the input text according to the entity recognition path, if the entity recognition path does not contain a rule matching algorithm, then the entity recognition path is updated. The updated entity recognition path contains a rule matching algorithm, minimizing the use of a large model and thus reducing costs. If the entity recognition path contains a rule matching algorithm, then the entity recognition path is updated. The rule matching algorithm in the updated entity recognition path is called before the large model, so that the input of the large model is the candidate entity list output by the rule matching algorithm, thereby reducing the length of the input text of the large model. A small-scale, low-cost large model can be used, thus reducing the cost of the large model.

[0086] In one possible example, the response latency constraint for entity recognition of input text based on the entity recognition path is as follows: if the entity recognition path does not contain a rule matching algorithm, then the entity recognition path is updated, and the updated entity recognition path contains a rule matching algorithm, minimizing the use of a large model and thus reducing response latency; if the entity recognition path contains a rule matching algorithm, then the entity recognition path is updated, and the rule matching algorithm in the updated entity recognition path is called before the large model, so that the input of the large model is the candidate entity list output by the rule matching algorithm, thereby reducing the length of the input text of the large model, improving the response speed of the large model, and reducing response latency.

[0087] If the constraint is that the cost of a single entity recognition operation does not exceed the single-operation limit, then when the cost of a single entity recognition operation exceeds the single-operation limit, the current entity recognition path will be forcibly updated to Path 1, and the number of candidate entities in the candidate entity list output by the rule matching algorithm will be greater than or equal to 1. If the constraint is that the daily cost of calling the large model does not exceed the daily large model budget, then when the daily large model budget is exhausted, the current entity recognition path will be forcibly updated to Path 1, ignoring the entity recognition path selected based on the quality score. If the constraint is that using Path 3 can reduce the cost of the large model by more than 5 times, then when Path 2 is selected based on the quality score, if it is determined that using Path 3 can reduce the cost of the large model using Path 2 by more than 5 times, Path 2 will be forcibly updated to Path 3. In other words, if the input of the large model can be optimized using the rule matching algorithm, the use of the large model for full entity recognition should be avoided as much as possible.

[0088] In implementation, after selecting an entity recognition path based on the quality score, the system re-evaluates whether the selected entity recognition path needs to be updated based on the response latency constraint. If the constraint is that the response latency does not exceed the maximum allowable latency, then if the response latency of the currently selected entity recognition path exceeds the maximum allowable latency, the currently selected entity recognition path is forcibly updated to path one. Alternatively, if path three (path one → path two) is currently being used, an asynchronous mode can be adopted, first returning the output of path one (rule matching algorithm) while continuing to execute path two in the background.

[0089] In one possible example, if the rule engine implementing the rule matching algorithm malfunctions (e.g., a single rule engine execution takes more than 150ms), the rule matching algorithm (first path) is skipped, and path two or path four is selected. When the service of a large model malfunctions, the entity recognition path is forcibly updated to path one.

[0090] As an optional implementation, an exemplary embodiment of this disclosure determines the quality score of the candidate entity list in the following manner: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score of the candidate entity with the rule, and the degree of difference in matching scores between different candidate entities.

[0091] Optionally, the difference in matching scores among different candidate entities includes the difference between the highest matching score and the second highest matching score; the matching score of a candidate entity with a rule includes the highest matching score between the candidate entity and the rule. Optionally, the quality score of the candidate entity list is determined based on the number of candidate entities in the candidate entity list, the highest matching score between the candidate entity and the rule, and the difference between the highest and second highest matching scores.

[0092] In practice, when calculating the quality score of the candidate entity list, the number of candidate entities can be considered. The fewer the number of candidate entities, the higher the quality of the candidate entity list. The higher the matching score of the candidate entities, the higher the quality of the candidate entity list. The larger the difference between the highest matching score and the second highest matching score among the candidate entities, the higher the quality of the candidate entity list.

[0093] In one possible example, the quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest and second-highest matching scores; the quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score. In implementation, the quantity factor is calculated based on the quantity, the score factor is calculated based on the highest matching score between the candidate entity and the rule, and the difference factor is calculated based on the difference between the highest and second-highest matching scores. Optionally, these factors (quantity factor, score factor, difference factor, etc.) can be configured and adjusted individually, and the factors in the next entity recognition path can be adjusted in real time based on the entity recognition results of the current entity recognition path. Optionally, the type and / or weight of the above factors can be adjusted according to actual scenarios / business needs (cost-sensitive scenarios / medical scenarios / financial scenarios / cold start scenarios / intelligent customer service / online Q&A / multilingual hybrid scenarios, etc.), for example, by adding context factors, adding language factors, adding coding consistency factors, and reducing the weight of the score factor. The factors in the exemplary embodiments of this disclosure can be defined and configured with corresponding weights according to actual business needs.

[0094] Optionally, a quantity factor for the number of candidate entities is included. The fewer the number of candidate entities, the higher this quantity factor, resulting in a higher final quality score. The quantity factor measures the concentration of the candidate entity list; fewer candidate entities mean higher matching confidence. For example, when the number is 1, the quantity factor is 1.0; when the number is 2-3, the quantity factor is 0.9-0.95; when the number is 4-5, the quantity factor is 0.8-0.9; and when the number of candidate entities > 5, the quantity factor < 0.8. For example, the quantity factor C1 = 1 - 0.05 × (n-1), with a lower limit of 0.50, where n = the number of candidate entities. Among them, n=1, C1=1; n=2, C1=0.95; n=3, C1=0.90; n=4, C1=0.85; n=5, C1=0.80; n=6, C1=0.75; n=10, C1=0.55; n≥11, C1=0.50.

[0095] Optionally, a score factor is set for the highest matching score among the candidate entities. The higher the highest matching score, the higher the score factor, resulting in a higher final quality score. For example, when the highest matching score is 100, the score factor is 1.0; when the highest matching score is 90-99, the score factor is 0.9-0.99; when the highest matching score is 80-89, the score factor is 0.8-0.9; and when the highest matching score is <80, the score factor is <0.8. For example, score factor C2 is used to normalize the highest matching score (out of 60-100) to the 0-1 range. C2 = S_max / 100, where S_max is the highest matching score. S_max = 100, C2 = 1; S_max = 95, C2 = 0.95; S_max = 90, C2 = 0.90; S_max = 85, C2 = 0.85; S_max = 80, C2 = 0.80; S_max = 70, C2 = 0.70; S_max = 60, C2 = 0.60.

[0096] Optionally, a difference factor is used to measure the difference between the highest and second-highest matching scores among candidate entities. The larger the difference, the higher the difference factor, resulting in a higher final quality score. For example, when the difference is ≥20 points, the difference factor is 1.0; when the difference is 10-19 points, the difference factor is 0.8-0.9; and when the difference is <10 points, the difference factor is <0.8. For example, the difference factor C3 measures the difference between the highest and second-highest matching scores. The larger the difference, the higher the quality of the candidate entity corresponding to the highest matching score, and the higher the quality score. C3 = min(1.0, 0.6 + ΔS × 0.02), where ΔS = S_max - S_second (the difference between the highest and second-highest matching scores), where S_second is the second-highest matching score. Optionally, when n=1 (only 1 candidate entity), there is no second-highest matching score, and C3 can be directly set to 1.0. For example, when ΔS=1 and n=1, C3=1.0; when ΔS=0, C3=0.60; when ΔS=5, C3=0.70; when ΔS=10, C3=0.80; when ΔS=15, C3=0.90; when ΔS≥20, C3=1.0.

[0097] In one possible example, a quantity factor is calculated based on the quantity, a score factor is calculated based on the highest matching score of the candidate entity matching the rule, and a difference factor is calculated based on the difference between the highest matching score and the second highest matching score; the quantity factor, the score factor, and the difference factor are weighted and summed based on their respective weights to obtain the quality score.

[0098] In one example, the formula for this quality score is as follows: The quality score is calculated as follows: Quantity Factor × 0.3 + Score Factor × 0.4 + Difference Factor × 0.3. The weights of the quantity factor, score factor, and difference factor are configurable. The weights of 0.3, 0.4, and 0.3 for each factor are merely examples; in practical applications, the weights can be adjusted according to the specific needs of the scenario. For instance, in some scenarios where matching scores are more important, the weight of the score factor (with the highest matching score) can be increased (from 0.4 to 0.5), while the weight of the quantity factor can be decreased (from 0.3 to 0.2). In other scenarios where the uniqueness of the number of candidate entities is more important, the weight of the quantity factor can be increased.

[0099] In one possible example, the quantity factor, score factor, and difference factor mentioned above are merely examples, and this disclosure is not limited to these three factors. In practical applications, relevant factors can be added or reduced according to the needs of specific scenarios. For example, factors related to candidate entities (considering the matching type, matching position, etc. of candidate entities), candidate entity type factors (different entity types have different importance), and context factors (considering the impact of contextual information on confidence) can be added. This embodiment does not impose excessive limitations on this. For instance, the weight of the score factor can be reduced while adding a contextual factor (considering the impact of contextual information on confidence).

[0100] In one example, assuming the input text is "Query for product model 36.6", the rule matching algorithm returns a list of candidate entities, which includes three candidate entities: ["36.6 FB" (70 points), "36.6 HD" (65 points), "36.8 FB" (60 points)]. Based on the quantity factor C1 of the number of candidate entities in the candidate entity list, the score factor C2 of the highest matching score between the candidate entity and the rule, and the difference factor C3 of the difference between the highest and second-highest matching scores, a quality score is calculated as follows: C1=1-0.05×(3-1)=0.90; C2=70 / 100=0.70; C3=min(1.0,0.6+(70-65)×0.02)=0.70; Quality score = 0.90×0.3+0.70×0.4+0.70×0.3=0.76; the quality range of 0.76 is (0.70-0.90), and the entity recognition path corresponding to this quality range is path three (first path → second path). At this time, the output of the first path is the list of candidate entities that has been calculated. According to the matching score from high to low, select 3 to 5 candidate entities as the input of the second path. For example, the input is ["36.6 FB", "36.6 HD", "36.8FB"].

[0101] For example, after selecting path three, a cost constraint check can be performed: Assuming that Cost_L_max (maximum amount per transaction) and Cost_budget_daily (daily budget for the large model) are not set (no limit by default), the cost of a single entity recognition operation, calculated using the first path of path three to optimize the number of input candidate entities in the large model, is Cost_L' ≈ Cost_L (cost of entity recognition for the entire large model) × 15%. That is, with 3 candidate entities compared to the total of 1000 entities, the number of tokens in the large model is reduced by 99.7%, satisfying the cost constraint. Therefore, no update to the entity recognition path is triggered, and path three is maintained. If the cost of a single large model call exceeds Cost_L_max, the selected path three is updated to path one. If the cumulative cost of large model calls for the day exceeds Cost_L_max, the selected path three is updated to path one.

[0102] For example, after selecting path three, a response latency constraint check can be performed: Assuming the current scenario is the default interaction scenario, the maximum allowable latency T_max_user=2s, the estimated response latency of path three is T_S=T_R (response latency of the first path)+T_L' (response latency of the second path)<100ms+2s≈2s. Within T_max_user, the response latency constraint condition is met, the update of the entity recognition path is not triggered, and path three is maintained.

[0103] For example, after selecting path three, business requirements can still be checked: Assuming the current business requirement is balanced mode (default): the minimum value of the quality range of high quality score is 0.90. If the quality score of the candidate entity list calculated at this time is 0.76, then since 0.76 < 0.90, path three is maintained.

[0104] The final entity recognition path is path three, and the data flow is as follows: preprocessing → rule matching algorithm outputs 3 candidate entities → large model selects the most matching candidate entity from the candidate entities → output. Since the large model inputs only 3 candidate entities, approximately 500 characters, compared to enumerating approximately 50,000 characters in the full dataset, the input data volume is reduced by 99%, the expected response latency is <2 seconds, and the expected cost is approximately 15% of that of the full large model.

[0105] The relationship between different entity recognition paths and costs in the exemplary embodiments of this disclosure is as follows: If path one is used, i.e., entity recognition is performed using only rule matching algorithms, the cost is approximately 0, which is purely CPU (Central Processing Unit) and incurs no cost for calling the large model API (Application Programming Interface). If path two is used, i.e., entity recognition is performed using only the large model, which is a full-scale model, requiring complete input text and containing a full list of entity enumerations, resulting in a high token consumption and cost. If path three is used, i.e., path one → path two, the length of the input text for the large model in path two is only 3 to 5 candidate entities, reducing the token count by 80-90%, and the cost is approximately 10% to 20% of that of the full-scale model.

[0106] The relationship between different entity recognition paths and response latency in the exemplary embodiments of this disclosure is as follows: The response latency of path one is typically <100ms, purely CPU-based local computation; the response latency of path two is typically 1s~5s, path two input contains a complete entity enumeration list, has many tokens, and slow inference. The response latency of the second path of path three for the large model is typically 0.5s~2s, the input contains only 3~5 candidate entities, tokens are reduced by 80-90%, inference is significantly accelerated, and the overall response latency of path three is ≈ T_R + T_L' rule pre-screening, the large model input has been optimized. The overall response latency of path five is the longer response latency of the first and second paths, at this time the latency of the first path is masked by the second path.

[0107] The exemplary embodiments disclosed herein can also update the minimum values ​​of different quality intervals according to business needs. For example, the default quality intervals include a first quality interval: quality score ≥ 0.9, and a second quality interval: quality score < 0.9. When the business need prioritizes accuracy, 0.90 is updated to 0.95, allowing more input text to enter the second path of the large model; when the business need prioritizes cost, 0.9 is adjusted to 0.8, allowing more input text to skip the second path of the large model; when the business need prioritizes response latency, the maximum allowable latency is tightened to 0.8~1.2s; when the business need is balanced (default), the default quality interval is used without adjustment.

[0108] In the exemplary embodiments of this disclosure, a quality score is used to determine whether a large model is required for entity recognition processing. Optionally, the quality score intervals are set as follows: a first quality interval (quality score ≥ 0.9), a second quality interval (0.7 ≤ quality score < 0.9), and a third quality interval (quality score < 0.7). Cost constraints are used to determine whether a large model is allowed in the entity recognition path, constrained by Cost_L_max (maximum amount per transaction) and Cost_budget_daily (daily budget for the large model). Response latency constraints are used to determine whether waiting for the large model to process is permissible, using a maximum allowed latency (T_max_user) and an immediate response latency limit T_prefer = 0.5s. The maximum and minimum values ​​of each quality interval can be adjusted based on business requirements.

[0109] The specific latency and cost of calling different paths are given below: Path 1: Preprocessing → Rule matching algorithm → Output. Large models are not involved, response latency is <100ms, and the cost is extremely low. Path 2: Preprocessing → LLM → Output, using a full large model, with a response latency of about 1~5s and high cost; Path 3: Preprocessing → Rule matching algorithm (candidate entity list) → LLM (selection) → Output, with a response time of approximately 1-5 seconds, and the cost can be reduced by 80-90% compared to Path 2; Path 4: Preprocessing → LLM (coarse identification) → rule matching algorithm (refinement / validation) → output, response time is about 1~5s, uses full large model, high cost; Path 5: Preprocessing → [Rule matching algorithm || LLM parallel] → Fusion → Output, response latency is about 1~5s, uses a full large model, high cost.

[0110] This embodiment can provide a comprehensive decision table as shown below:

[0111] In the exemplary embodiments of this disclosure, business requirements can adjust all global quality ranges. Cost and response latency, as hard constraints, can reject the entity recognition path selected by the quality score, directly overriding the recommended path of the quality score when limits are exceeded. The quality score, as a soft recommendation, is the basis for the default selection of the entity recognition path. Optionally, the entity recognition path can also be selected based on the system state. Optionally, the system state includes, but is not limited to, any one or more of the following: running state (running, idle, occupied, standby, restarted, etc.), process / service state (started, started, starting, stopped, disabled, abnormally terminated), and hardware state (normal, abnormal, faulty).

[0112] As an optional implementation, the recognition method provided by the embodiments of the present disclosure is a general entity recognition method, which can recognize various types of named entities, including but not limited to the entity recognition of product models, bad names, and time. The rule matching algorithm in the embodiments of the present disclosure can be applied to any scenario that requires named entity recognition. The large model in the embodiments of the present disclosure can be applied to any scenario that requires named entity recognition. The entity recognition path in the embodiments of the present disclosure can be applied to any scenario that requires named entity recognition.

[0113] In an optional embodiment, the time recognition of the present disclosure adopts a general time pattern matching method, which can recognize various common time expression formats from natural language texts. The content of time recognition includes but is not limited to: ① Date format recognition; Standard date format: such as "2025-01-01", "2025 / 01 / 01", "2025.01.01", etc.; Chinese date format: such as "January 1, 2025", "January 2025", "2025", etc.

[0114] ② Quarter recognition: such as "the first quarter of 2025", "2025 Q1", "Q1 of 2025", etc.

[0115] ③ Time period recognition: such as "January to March 2025", "2025-01-01 to 2025-03-31", etc.

[0116] ④ Relative time recognition: such as "today", "yesterday", "last week", "this month", "this year", etc.

[0117] ⑤ Time range recognition: such as "the last month", "the past three months", "the last year", etc.

[0118] Optionally, the time recognition in the exemplary embodiments of the present disclosure adopts a rule matching algorithm, which is matched through a predefined time pattern regular expression. The recognition speed is fast and the accuracy is high. There is no need to call a large model (such as an LLM), and it can be merged with other entity recognition results for output.

[0119] In one possible example, the rule matching algorithm of this exemplary embodiment can provide a multi-level progressive fuzzy matching algorithm for identifying various types of named entities. The rule matching algorithm includes N levels of rules, where the accuracy of rule matching decreases as the level increases, and N is a positive integer greater than or equal to 2. Optionally, the accuracy of rule matching at each level can be represented by a matching score for that level; a higher matching score indicates greater accuracy at that level. By using this rule matching algorithm to match level by level, starting with higher-accuracy matching and directly outputting the matching result upon successful matching, and then proceeding to lower-accuracy matching if the matching fails, a full-process matching process from precise to fuzzy is achieved.

[0120] In one possible example, in an exemplary embodiment of this disclosure, when the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i, the matching result including the matched entity and the matching score; when the input text does not match a rule at level i, the input text is matched using a rule at level i+1, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0121] Optionally, the matching results include, but are not limited to, any one or more of the following: entity type, entity (value), matching position, matching score, etc. In implementation, rule matching is performed using an entity enumeration value list. For each entity in the entity enumeration value list, if it matches the input text at level 1 (exact match), and the entity in the entity enumeration value list successfully matches the input text based on the level 1 rule, then the matching score and the matched entity are output. If the matching fails at level 1, matching is performed at level 2. If the entity in the entity enumeration value list successfully matches the input text based on the level 2 rule, then the matching score and the matched entity are output. When the level is greater than 2, subsequent levels of rule matching are performed in this manner.

[0122] In one example, the matching score ranges for different levels of the rule matching algorithm in this embodiment are different, and the minimum value of the matching score range decreases as the level increases. Specifically, the first level is an exact match with the highest score range, such as a score of 100. Afterward, as the level increases, the minimum value of the matching score range for the entities matched by the rules at each level gradually decreases. Optionally, different matching score ranges can be set from the j-th level to the last level, and the minimum value of these matching score ranges is the same, all set to a threshold score such as 60. The maximum values ​​can be the same or different. Optionally, j ≥ m, where m takes values ​​in the range [5, 9], and j and m are both integers. Optionally, the matching score range can have only one value.

[0123] Optionally, embodiments of this disclosure may also adjust the parameters in the next entity recognition path based on one or more of the accuracy of the entity recognition result output by the current entity recognition path, the required cost, and the response latency. These parameters include, but are not limited to, any one or more of the following: the maximum / minimum value of the quality score range, the level in the rule matching algorithm, the matching type of each level, the matching score range, the scale of the large model, the deployment method, cost constraints, response latency constraints, the candidate entity list, and decision conditions (two-level quality range / three-level quality range), etc.

[0124] In one possible example, the rule matching algorithm in this embodiment of the disclosure includes any or more of the following: prefix matching, subsequence matching, partial keyword matching, and longest match priority. Optionally, the rule matching algorithm is also used to perform any or more of the following: unifying full-width and half-width characters, recognizing combinations of data and text, filtering intent phrases, filtering template keywords, and supporting multilingual mixed recognition.

[0125] For example, taking a rule matching algorithm with 10 levels of rules as an example, each level corresponds to a different matching score range, which is within the range of [60, 100], where: Level 1: Exact match (100 points); Level 2: Ignore case matching (95 points); Level 3: Standardized bracket matching, full-width / half-width character conversion (90 points); Level 4: Normalized matching, unifying spaces, parentheses, and capitalization (88 points); Level 5: Remove whitespace matching (85 points); Level 6: Remove delimiter matching (82 points); Level 7: Matching parentheses and spaces (75 points); Level 8: Keyword matching (60-70 points); Level 9: Word matching (60-75 points); Level 10: Prefix / subsequence matching (60-75 points).

[0126] like Figure 2 The flowchart shown in this embodiment illustrates a rule-matching algorithm, using the identification of product model and defective name as an example. However, this rule-matching algorithm is not limited to this application scenario. The matching process of the rule-matching algorithm is as follows: Step 200: Obtain the input text and the list of entity enumeration values; The rule-based matching algorithm matches the input text against each entity in a predefined list of entity enumeration values. For example, if the input text is "Query products of model 36.6", and the entity enumeration value list contains entities such as "36.6 FB" and "36.6 HD", the algorithm will match the input text against each entity in the list in turn.

[0127] Step 201: For each entity in the entity enumeration value list, match it sequentially according to the 10-level matching strategy, gradually reducing the matching threshold from exact matching to fuzzy matching. Level 1: Exact Match (100 points); Checks whether the entities in the entity enumeration value list are exactly matched in the input text; for example: "36.6 FB" in the entity enumeration value list is an exact match in the text "Query products of model 36.6 FB", returning 100 points.

[0128] Level 2: Case-insensitive matching (95 points); if an exact match fails, perform case-insensitive matching; for example, "36.6 FB" matches "36.6 fb", returning 95 points.

[0129] Level 3: Standardized bracket matching (90 points); converts full-width brackets to half-width brackets before matching; for example, "PTGap(G)" matches "PT Gap(G)", returning 90 points.

[0130] Level 4: Normalized matching (88 points); matches after unifying spaces, parentheses, and capitalization; for example, "36.6 FB" matches "36.6 FB", returning 88 points.

[0131] Level 5: Remove whitespace matching (85 points); Match after removing all whitespace; for example, "36.6FB" matches "36.6FB", returning 85 points.

[0132] Level 6: Match without separators (82 points); Matches after removing separators (such as "-", "_", etc.). For example, "36-6-FB" matches "36.6 FB", returning 82 points.

[0133] Level 7: Matching after removing parentheses and spaces (75 points); Matching after removing parentheses and spaces; For example: "PTGap(G)" matches "PT Gap (G)", returning 75 points.

[0134] Level 8: Keyword matching (60-70 points); Extract keywords for partial matching; for example, "36.6" in the question can match the entity "36.6 FB" (the numeric part is the same), returning 60-70 points.

[0135] Level 9: Word matching (60-75 points); supports word-level matching and tolerates spelling errors; for example, "dark spot" can match "dark spot (MDL)" and return 60-75 points.

[0136] Level 10: Prefix / Subsequence Matching (60-75 points); Supports both prefix and subsequence matching; For example, "UD" can match "UHD" (subsequence matching), and "De" can match "Dell" (prefix matching), returning 60-75 points.

[0137] Step 202: If no match is found at any level, return 0 points, indicating no match.

[0138] Step 203: Following the order from level 1 to level 10, if a level matches successfully, output the matching result.

[0139] The matching results include, but are not limited to, whether a match was made, the matching score, the matching type, and the matched entities. The matching type refers to the type of matching rule at the corresponding level, including but not limited to any one or more of the following: exact match, case-insensitive match, normalized bracket match, normalized match, whitespace-removed match, separator-removed match, bracket and whitespace-removed match, keyword match, word match, prefix / subsequence match, etc. Optionally, the matching types at each level are only examples; in actual use, the matching types and number of levels can be flexibly configured according to business needs.

[0140] Optionally, this disclosure provides an example of a scenario for identifying a complete entity from incomplete information, as shown below: The input text is: "Query products with model number 36.6"; the entity enumeration list includes entities such as "36.6 FB" and "36.6HD". The rule matching process involves the algorithm matching the input text against each entity in the entity enumeration list. Through 8th-level keyword matching, the "36.6" in the question matches the "36.6" part of the entity "36.6 FB", returning a score of 60-70. The result is a successful identification of the complete "36.6 FB" from the incomplete "36.6".

[0141] Optionally, this disclosure provides a prefix matching example, as shown below: Input text: "Query products starting with De"; Entity enumeration value list: contains entities such as "Dell" and "Dell UHD". Rule matching process: The rule matching algorithm matches the input text with each entity in the entity enumeration value list. Through the 10th level prefix matching, "De" matches the prefix of "Dell", returning a score of 60-75. Result: Successfully identified the complete "Dell" from "De".

[0142] Optionally, this disclosure provides a subsequence matching example, as shown below: Input text: "Query UD products"; Entity enumeration value list: contains entities such as "UHD" and "UHD 4K". Rule matching process: The rule matching algorithm matches the input text with each entity in the entity enumeration value list. Through the 10th level subsequence matching, "UD" is a subsequence of "UHD", returning 60-75 points. Result: Successfully identified the complete "UHD" from "UD".

[0143] The implementation method of the rule-matching algorithm for multi-level progressive fuzzy matching provided in this disclosure is described below: 1) Process Sequence: The rule-based matching algorithm performs matching sequentially from exact matching to fuzzy matching, starting from level 1 (exact matching, 100 points) and gradually decreasing the matching threshold until level 10 (prefix / subsequence matching, 60-75 points). If a match is successful at a certain level, the matching score for that level is immediately returned, and matching at subsequent levels is not continued. If no match is found at any level, a score of 0 is returned to indicate no match.

[0144] 2) Number of Matching Levels: The above explanation uses a 10-level matching strategy as an example, but the rule matching algorithm of this embodiment is not limited to 10 levels. In practical applications, the number and type of matching levels can be flexibly adjusted according to the needs of specific scenarios. For example, for some simple scenarios, only the first 5 levels of the matching strategy can be used; for complex scenarios, more matching levels can be added, such as adding matching levels for synonym matching, matching levels for pinyin matching, etc.

[0145] 3) Scoring Mechanism: The matching scores for each level (100, 95, 90, etc.) are only examples. In actual applications, the matching scores for each level can be adjusted according to the specific needs of the scenario. For example, the score weights for each level can be adjusted based on factors such as the importance of different entity types and the reliability of different matching levels. Furthermore, the matching score range within the same level (e.g., the range of 60-70 points for level 8) can also be further subdivided based on matching quality to more accurately reflect the similarity of the matches.

[0146] like Figure 3 As shown in the exemplary embodiments of this disclosure, an implementation flow of a named entity recognition method is also provided, which is illustrated using an SQL generation scenario as an example, as follows: Step 300: The user inputs a natural language question through the user interface; For example, "Query the cases of ACF missing (PCB) defects in product model 98 4K 60Hz 0+4 in the first quarter of 2025".

[0147] Step 301: Obtain the natural language question input by the user; In this embodiment, the natural language question input by the user serves as the input text for an exemplary embodiment of this disclosure.

[0148] Step 302: Use a rule matching algorithm to perform entity recognition on the input text to obtain a candidate entity list; In implementation, a rule-matching algorithm is used to extract a list of candidate entities (such as product model, defective name, etc.) from the input text, and a general time recognition method is used to extract time information (such as "first quarter of 2025") from the input text. Step 303: Calculate the quality score based on the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the difference in matching scores between different candidate entities; Step 304: Determine the entity recognition path based on the quality score; For example, when the quality score is ≥0.9, the candidate entity list output by the rule matching algorithm is used directly as the candidate entity result; when the confidence score is <0.9, LLM is called to make an exact selection from the candidate entity list.

[0149] Step 305: In the SQL generation scenario, pass the entity recognition result of the entity recognition path to the SQL generation module.

[0150] Optional entity recognition results include entity information such as product model, defect name, and time.

[0151] Step 306: The SQL generation module generates the final SQL query statement based on the entity recognition results.

[0152] Step 307: The database query module executes an SQL query and retrieves the query results from the database.

[0153] Step 308: The result processing module processes and formats the query results, and then returns them to the user interface for display.

[0154] The recognition method provided in this disclosure can be implemented by a processor. The recognition method executed by the processor can be independently applied to various scenarios requiring named entity recognition, and is not limited to SQL generation scenarios. For example, it can also be applied to other scenarios such as information extraction, knowledge graph construction, and intelligent question answering.

[0155] like Figure 4 As shown, this disclosure provides an implementation flowchart for a named entity recognition scenario, using the identification of product models and inappropriate names from natural language processing as an example, as detailed below: Step 400: Receive input text; For example, the input text could be "Query for dark spot defects in product model 36.6 during the first quarter of 2025".

[0156] Step 401: Extract candidate entities using a rule-matching algorithm.

[0157] The rule-based matching algorithm performs matching according to a 10-layer matching strategy, with each layer corresponding to a different matching score range. The matching score is limited to a range of 60-100 points, including 60 and 100 points.

[0158] During implementation, different types of entities are identified separately: 1) Product Model Rule Matching: Uses a rule matching algorithm to find matching product models from the input text. For example, "36.6" may match candidate entities such as "36.6 FB" and "36.6 HD". 2) Bad Name Rule Matching: Use rule matching algorithms to find matching bad names in the input text. For example, "dark spot" may match candidate entities such as "dark spot" and "dark spot (MDL)". 3) Time recognition: Use general time recognition methods to extract time information from the input text, such as candidate entities like "first quarter of 2025", "2025-01-01", and "January 2025"; It should be noted that the rule-matching algorithm provided in this disclosure can identify various types of named entities, not limited to product models, bad names, and times. It can also be applied to identify other types of entities such as personal names, place names, organization names, and quantities. For example, time recognition can use a general time pattern matching method, which can identify various common time expression formats.

[0159] Step 402: Output the matching results; The matching results include the matched entity value, the matching type, and the matching score.

[0160] Step 403: Generate a candidate entity list based on the matching results; Optionally, 3 to 5 candidate entities are retained for each entity type, sorted from high to low according to matching score.

[0161] Step 404: Calculate the quality score of the candidate entity list; Step 405: Decision on entity identification path based on quality score; 1) If the quality score is ≥0.9 (high quality), the candidate entity with the highest score is directly matched from the candidate entity list output by the rule matching algorithm, completely avoiding large model calls, achieving millisecond-level response, and reducing costs by 60-80%.

[0162] 2) If the quality score is <0.9 (medium to low quality), construct LLM prompts based on the candidate entity list, and only provide a pre-filtered list of candidate entities for rule matching (3 to 5 candidate entities) instead of full enumeration, reducing the length of the input text by 80-90%.

[0163] When an LLM needs to be invoked, select path three. The LLM in the second path will make an exact selection from the list of candidate entities and select the most matching entity.

[0164] Step 406: Output the entity recognition results of the entity recognition path.

[0165] In one example, the candidate entity list output by the rule matching algorithm, the matching entity output by the LLM from the candidate entity list, and the time recognition result can be merged to finally output the identified entity information. In the SQL generation scenario, the output identified entities include product model, defect name, time, etc.; in other application scenarios, entity information of corresponding types (such as person name, place name, organization name, quantity, etc.) can be output. The above implementation uses the SQL generation scenario as an example to illustrate the specific application of this disclosure embodiment in identifying entities such as product model and defect name from natural language questions. However, the identification method provided by this disclosure embodiment is a general entity recognition method that can be applied to various scenarios that require named entity recognition, including but not limited to any one or more of the following: information extraction scenario: extracting entities such as person name, place name, and organization name from documents; knowledge graph construction scenario: identifying entities from text and constructing a knowledge graph; intelligent question answering scenario: identifying key entity information from user questions; text analysis scenario: identifying various types of named entities from text; and other application scenarios that require named entity recognition.

[0166] For example, the rule matching algorithms in the embodiments of this disclosure include, but are not limited to, any one or more of the following: regular expression-based matching, precise / fuzzy matching based on a predefined entity dictionary, search matching based on an enumerated value list, structured matching based on a pattern template, etc.

[0167] In one possible example, see Figure 5This disclosure provides an entity recognition system architecture, including an input module 500, a rule recognition module 501, a model recognition module 502, a path control module 503, and an output module 504. The input module 500 receives user input text. The rule recognition module 501 performs entity recognition based on a rule-based algorithm, which includes, but is not limited to, matching based on rules, dictionaries, and enumerated values. The model recognition module 502 performs entity recognition based on a large model. The path control module 503 determines the entity recognition path to use based on metrics (quality score, cost, response latency, business requirements, etc.), such as whether to call the rule matching algorithm or the large model, and in what order / combination (serial, parallel, sequential). The output module 504 fuses the entity recognition results from the rule matching algorithm and the large model and outputs them to downstream applications.

[0168] Optionally, after receiving the raw natural language text input by the user, the input module 500 can also preprocess the input text, including but not limited to any one or more of the following: word segmentation, noise reduction, encoding unification, etc., to provide a unified input for any subsequent entity recognition path.

[0169] In implementation, the rule recognition module 501 performs entity matching on the input text based on preset entity knowledge (entity enumeration list, dictionary, rules, pattern templates, etc.), and outputs candidate entities and their matching related information (such as matching position, matching score, etc.). This disclosure does not limit the rule recognition module 501 to exact matching or fuzzy matching; any "explicitly configurable rule / table-driven" approach falls within the protection scope of this disclosure. For example, rule matching algorithms include, but are not limited to: entity enumeration value lists maintained through configuration files or database tables (such as product model tables, defective name tables), regular expression rules configured through a rule engine, entity dictionaries maintained through dictionary files, and structured patterns defined through template files. These can all adjust the rule recognition behavior by updating configuration / table data without modifying the code. Optionally, this application can update at least one of the following in the rule matching algorithm: the number of levels, the matching score range corresponding to each level, and the matching type corresponding to each level. Specifically, this can be achieved by updating the entity enumeration value list maintained in the configuration file or database table, which updates at least one of the following in the rule matching algorithm: the number of levels, the matching score range for each level, and the matching type for each level. Alternatively, it can be achieved by updating the entity dictionary maintained in the dictionary file, which updates at least one of the following in the rule matching algorithm: the number of levels, the matching score range for each level, and the matching type for each level.

[0170] For example, the rule recognition module 501 adopts a 10-layer matching strategy with a matching score range of 60-100 points. The threshold is gradually reduced from exact matching to fuzzy matching. It supports mechanisms such as prefix matching, subsequence matching, partial keyword matching, and longest matching priority. It can identify complete standard entity information from incomplete entity information and output a candidate entity list (usually 3-5 candidate entities) containing information such as matching score and matching type.

[0171] The model recognition module 502 utilizes a large model to perform semantic understanding and entity recognition on the input text. It can directly identify entities from the input text, or, under the control of the path control module 503, select or correct entities based on the candidate entities output by the rule recognition module 501. This disclosure does not limit the model size, deployment method (local / cloud), or specific vendor; any subsystem that "completes entity recognition tasks based on a large model" is acceptable.

[0172] The path control module 503, acting as the global "scheduling and decision-making center," uniformly manages the calling order, calling conditions, and combination methods of the rule recognition module 501 and the model recognition module 502, and evaluates intermediate results. The path control module 503 possesses at least some or all of the following functions: a) Evaluation and Measurement: Evaluate the quality of the recognition (matching) results output by the rule matching algorithm, the recognition results output by the large model, or a combination of the two. Evaluation metrics include, but are not limited to, any one or more of the following: quality score, matching score, number of candidate entities, cost estimation, etc. b) Strategy Decision: Based on the evaluation results and business needs (such as cost priority, accuracy priority, response latency priority, etc.), the following paths are selected: ① Only call the rule recognition module 501 and output directly; i.e., select path one. ② Only call the model recognition module 502 and output directly; i.e., select path two. ③ First call the rule recognition module 501, and then decide whether to call the large model based on the quality score of the candidate entity list output by the model recognition module 502; if it is decided to call, select path three; otherwise, select path one and directly output the entity recognition results of the rule recognition module 501. ④ First call the model recognition module 502, and then call the rule recognition module 501 for correction or supplementation; i.e., select path four. ⑤ The rule recognition module 501 and the model recognition module 502 are called in parallel, and then the output module 504 performs fusion processing before output.

[0173] The path control module 503 of this disclosure is located at the center of the system architecture and has bidirectional information paths with both the rule recognition module 501 and the model recognition module 502. Example of intelligent decision-making mechanism: High quality (quality score ≥ 0.9): Directly uses the output of the rule recognition module 501, completely avoiding large model calls, achieving millisecond-level response, and reducing costs by 60-80%. Medium to low quality (quality score < 0.9): Calls the model recognition module 502, i.e., executes path three, and makes precise selections from the candidate entity list output by the rule recognition module 501. In one example, this disclosure can also dynamically adjust subsequent calling strategies based on the entity recognition results of the previous round, achieving deep integration of the rule matching algorithm and the large model rather than simple concatenation. Compared to full enumeration, it can significantly reduce the length of the input text, reduce the computational complexity and resource consumption of large model inference, and the large model only needs to select from a small number of candidate entities, improving recognition accuracy.

[0174] Output module 504 receives one or more candidate entities from rule recognition module 501 and model recognition module 502, generates the final entity recognition result according to preset or learned fusion rules, and outputs it to the downstream system in a unified format. This disclosure does not limit the fusion algorithm (simple priority, weighted fusion, learned fusion, etc. are all acceptable), as long as there is an "explicit, downstream-oriented unified output interface". Optionally, when only one module participates in recognition, output module 504 is simply a direct output path; when two modules participate in recognition, output module 504 performs the function of fusion or simple merging. Output module 504 provides a unified output interface to output the final entity recognition result to the downstream system.

[0175] The rule recognition module 501 provided in this disclosure is an independent module specifically designed for recognizing and extracting named entities from text. Its core function is "entity recognition" rather than "constraint verification" or "information query." This rule recognition module 501 can actively search, match, and extract candidate entities (such as product models, defective names, personal names, place names, etc.) from input text; output structured entity recognition results (including at least: entity type, entity value (i.e., matched entity), matching position, matching score, etc.); and perform rule matching based on explicitly configurable rule / dictionary / enumeration value tables (rather than implicit model parameters). The model recognition module 502 provided in this disclosure is an independent module specifically designed for recognizing and extracting named entities from text. Its core function is "entity recognition" rather than "information provision" or "text generation." This model recognition module 502 can actively recognize and extract candidate entities from input text (either directly or by selecting from a candidate entity list); output structured entity recognition results (including at least: entity type, entity value, confidence score, etc.); and complete the recognition task based on the semantic understanding capabilities of a large model. The path control module 503 provided in this disclosure is an independent control module capable of dynamically determining the calling order, calling conditions, and combination method of the rule recognition module 501 and the model recognition module 502 based on evaluation indicators. The path control module 503 possesses evaluation capabilities: it can measure the quality of the output results of the rule recognition module 501, the model recognition module 502, or a combination of the two (evaluation indicators such as quality score, matching score, number of candidate entities, etc.); the path control module 503 possesses decision-making capabilities: it can dynamically select the entity recognition path to be called based on the evaluation results and preset strategies; and it also possesses data flow control capabilities: it can control the flow direction, timing, and content of the input text between the rule recognition module 501 and the model recognition module 502. The data flow characteristics are as follows: Input data flow: Preprocessed input text → Path control module 503 (decision) → Call rule recognition module 501 and / or model recognition module 502. Example intermediate data flow: Rule recognition module 501 → Path control module 503 → May flow to model recognition module 502 or output directly. Example intermediate data flow: Output of model recognition module 502 → Path control module 503 (decision) → Flow back to rule recognition module 501. Output data flow: Rule recognition module 501 and / or model recognition module 502 → Output module 504 (may be fused).

[0176] Output module 504 is an independent module that receives entity recognition results from rule recognition module 501 and model recognition module 502, integrates them, and outputs the results. It can perform fusion processing on the results of rule recognition module 501 and model recognition module 502 (such as priority selection, weighted fusion, conflict resolution, etc.); c) it provides a unified output interface to output the final entity recognition result to downstream systems. The inputs to output module 504 are: the entity recognition result of rule recognition module 501 (i.e., the first recognition result) and the entity recognition result of model recognition module 502 (i.e., the second recognition result) (there may be only one, or both). Output module 504 can also use fusion algorithms (priority, weighted, learned, etc.) to output a unified entity recognition result format for easy reception by downstream application interfaces.

[0177] The input data flow in the exemplary embodiments of this disclosure is: text → input module 500 → path control module 503, whereby path control module 503 distributes the text to rule recognition module 501 and / or model recognition module 502 based on a decision. The intermediate data flow (bidirectional) is: path control module 503 → rule recognition module 501 → path control module 503 → possibly flowing to model recognition module 502; the output of model recognition module 502 → path control module 503 → possibly flowing to rule recognition module 501. This bidirectional data flow embodies a "deep fusion" of rule matching and large models rather than a "simple concatenation." The output data flow in the exemplary embodiments of this disclosure is: rule recognition module 501 → output module 504; or, model recognition module 502 → output module 504; the output of output module 504 → downstream application interface. The path control module 503 is used to evaluate the output result (candidate entity list) of the rule recognition module 501 and decide whether to call the large model (i.e. whether to call the model recognition module 502) based on the evaluation result. The path control module 503 is also used to evaluate the output result of the model recognition module 502 and decide whether to call the rule recognition module 501 based on the evaluation result. The path control module 503 is also used to dynamically select the entity recognition path based on factors such as system status, cost, and response latency.

[0178] like Figure 6As shown, this disclosure provides a schematic diagram of entity recognition data flow transmission in an SQL generation scenario. Taking an SQL generation scenario (the yield ChatBI application in an industrial knowledge question-answering system) as an example, it illustrates the complete information flow between the web interface, database, processor (entity recognition), and dynamic adjustment module, particularly the multi-point data sources required for quality score calculation. It should be noted that although this data flow transmission schematic diagram uses an SQL generation scenario as an example, the processor, dynamic adjustment module, and multi-point data flow in the recognition method provided in this disclosure are universal and can be applied to other scenarios requiring named entity recognition (such as information extraction, knowledge graph construction, intelligent question answering, etc.). Simply replace the "SQL generation module" and "SQL executor" with the corresponding downstream processing modules. Optionally, the processor is used to execute the steps of any of the recognition methods provided in this disclosure. Optionally, the dynamic adjustment module in this disclosure is used to adjust specific parameters in the entity recognition method executed by the processor, such as the level N in the rule matching algorithm, the matching type of each level, the matching score range configured for each level, and the range (maximum / minimum) of the quality score range.

[0179] See Figure 6 The main business data flow is: WEB interface → processor (entity recognition processing) → SQL generation module → SQL executor → business database → return results → WEB interface. The data acquisition flow (solid arrow, multiple points converging) is as follows: ① Processor data collection: Matching score of the rule matching algorithm, number of candidate entities, confidence level of entity recognition results output by the large model, and entity recognition path for routing decisions. ② SQL executor data collection: SQL execution time, success rate, number of returned data rows, etc. ③ Web interface data collection: User feedback, manual correction annotations, etc. Optionally, all data is aggregated into a log and monitoring database to provide a data foundation for quality score calculation and entity recognition path selection optimization.

[0180] During implementation, the closed-loop flow can be dynamically adjusted (solid arrows indicate dynamic optimization), and the adjustment methods are as follows: ① Log and Monitoring Library → Dynamic Adjustment Module: Statistically analyzes multi-source data, calculates the distribution of quality scores, the confidence distribution of entity recognition results output by large models, accuracy curves, costs, response latency, and the relationship between accuracy, cost, and response latency. Based on the results of statistical analysis, it determines whether to dynamically adjust the specific configuration of the recognition method executed by the processor, such as adjusting specific parameters in the rule matching algorithm and adjusting the quality range of the quality score.

[0181] ② Dynamic Adjustment Module → Configuration Center: Outputs the adjusted strategy parameters (such as the level in the rule matching algorithm, the matching type of each level, the matching score range configured for each level, the quality range of the quality score, etc.).

[0182] ③ Configuration Center → Processor: Loads the latest configuration at runtime for dynamic adjustment.

[0183] ④ Configuration Center → SQL Generation Module (dashed line): Optionally restrict SQL generation or execution strategies.

[0184] The quality score of the candidate entity list in this embodiment relies on multiple data sources: matching score (from the processor's rule matching algorithm), number of candidate entities (from the rule matching algorithm), SQL execution performance (from the SQL executor, indirectly reflecting entity recognition accuracy), and user feedback (from the web interface, directly reflecting recognition accuracy). These multiple data sources are aggregated through logs and a monitoring database, and then comprehensively analyzed by a dynamic adjustment module to optimize the quality range of the quality score and the parameters in the rule matching algorithm. The dynamic adjustment module dynamically adjusts the strategy parameters based on actual operating results (accuracy, cost, response latency), realizing the transformation from "fixed code implementation" to "dynamic configuration optimization," supporting scenario-based combination configurations, and automatically or manually adjusting strategies according to different scenario requirements. Data from multiple data sources is collected through logs and a monitoring database, the dynamic adjustment module optimizes strategy parameters, and the configuration center issues the latest configuration.

[0185] Aside from SQL generation scenarios, the core architecture of this embodiment is general and can be applied to other scenarios, such as information extraction scenarios: replace the "SQL generation module" and "SQL executor" with the "information extraction module" and "knowledge base storage," while keeping other components unchanged. For knowledge graph construction scenarios: replace the "SQL generation module" and "SQL executor" with the "relationship extraction module" and "graph storage," while keeping other components unchanged. For intelligent question answering scenarios: replace the "SQL generation module" and "SQL executor" with the "answer generation module" and "knowledge base retrieval," while keeping other components unchanged. For text analysis scenarios: replace the "SQL generation module" and "SQL executor" with the "text analysis module" and "result storage," while keeping other components unchanged. Regardless of the scenario, the core modules such as the processor (rule matching algorithm + large model), dynamic adjustment module (multi-source data optimization), and configuration center (scenario-based combination configuration) are universal, demonstrating the versatility and scalability of this embodiment.

[0186] Existing technologies typically employ hard-coding, directly embedding matching strategies, thresholds, and decision rules into the code. When adapting to different scenarios, this requires code modification and redeployment, lacking dynamic adjustment, scenario-based configuration management, and the ability to flexibly combine features according to different scenario needs or progressively optimize based on actual performance. Implementation relies solely on code modification. Compared to the existing "fixed code implementation, no dynamic configuration" approach, the scenario-based combination configuration provided in this disclosure offers the following advantages: (1) Dynamic principle setting: Supports dynamic setting of parameters (such as level), quality range of quality score, and judgment conditions of path decision (such as the number of quality ranges and the entity recognition path corresponding to the quality range) involved in the rule matching algorithm through configuration files, management interface or API interface, without modifying the code. (2) Scenario-based combination: For different scenario needs, various components can be flexibly combined and adjusted to achieve the best recognition effect. (3) Real-time adjustment: The configuration parameters can be adjusted in real time according to the actual running effect (such as cost constraints, response delay constraints, etc.) to achieve continuous optimization. (4) Template-based management: Supports preset configuration templates for different scenarios, quick switching and reuse. When making dynamic adjustments, only the specific parameters in the template need to be changed, without changing the entire code.

[0187] This disclosure separates configuration from code: matching strategies, quality score calculations, decision rules, etc., are abstracted into configurable parameters. Runtime loading: Configuration is dynamically loaded at system startup or runtime without requiring a restart. Configuration verification: The legality of configuration parameters is verified to ensure correctness. Configuration version management: Configuration version management is supported, allowing rollback to historical configurations. Optionally, configuration parameters include, but are not limited to, any one or more of the following: the maximum / minimum value of the quality score range, the level in the rule matching algorithm, the matching type of each level, the matching score range, the scale of the large model, deployment method, cost constraints, response latency constraints, etc. The recognition method provided by this disclosure is highly adaptable: configuration can be dynamically adjusted according to the needs of different scenarios without modifying the code; cost is controllable: in cost-sensitive scenarios, the number of LLM calls can be significantly reduced through configuration; accuracy is adjustable: in accuracy-priority scenarios, recognition accuracy can be improved through configuration; rapid adaptation: new scenarios can be quickly adapted through configuration without redevelopment; incremental optimization: configuration parameters can be gradually adjusted based on actual operating results to achieve continuous optimization.

[0188] The configuration management method disclosed herein supports dynamic configuration via configuration files, management interfaces, or API interfaces. Configuration parameters include, but are not limited to, any one or more of the following: (1) Rule matching level N configuration: Enable / disable the matching strategy of each level and adjust the matching score range; (2) Quality score configuration: Adjust the factors required to calculate the quality score (such as quantity factor, score factor, difference factor), the weight of each factor, and the maximum / minimum value of the quality range; (3) Decision rule configuration: Select the decision level (two-level / three-level quality range) and set the range (maximum / minimum value) of the corresponding quality range for each level; (4) Candidate entity list configuration: Set the upper limit of the number of candidate entities, sorting rules, etc.; (5) Scene template configuration: Preset configuration templates for different scenes and switch quickly. The above scenario-based combination configuration capability enables this embodiment to adapt to various different application scenarios and achieve the best recognition effect.

[0189] like Figures 7A-7C As shown, this disclosure provides an interface diagram of a question-and-answer interaction system. Any of the above-described identification methods provided in this disclosure can be applied to this question-and-answer interaction system. (See also...) Figure 7A Users enter the "Industrial Knowledge Q&A Assistant" homepage, click "Yield Analysis Assistant," and input a natural language question: "GLS Concentration Analysis of Dark Spot Defects at 554K 60Hz from September 30, 2022 to September 30, 2025." The system backend performs entity recognition on the input text based on the recognition method disclosed herein, selects the corresponding entity recognition path, outputs the entity recognition result of the input text, and inputs the entity recognition result to the downstream processing module for processing. Simultaneously, the display interface redirects to... Figure 7B This displays the system's thought process, outputs key information for entity recognition, and jumps to [the next step] when processing is complete. Figure 7C The results of the question are displayed and presented to the user in the form of charts.

[0190] This disclosure uses an SQL generation scenario as an example to test the recognition method of this disclosure. The test report of SQL generation rate is as follows: Total number of tests: 4518; Correct number: 4427; Error number: 91; Accuracy: 97.99%.

[0191] The identification method disclosed herein employs a 10-layer matching strategy (adjustable according to specific scenarios), with a scoring range of 60-100 points. It progressively lowers the matching threshold layer by layer, from precise matching to fuzzy matching, achieving a gradual matching process from precise to fuzzy. This rule-based matching algorithm can handle various format variations, spelling errors, incomplete information, and other situations, supporting mechanisms such as prefix matching, subsequence matching, partial keyword matching, and longest match priority. A comprehensive confidence score is calculated based on multiple factors (adjustable according to specific scenarios), including the number of candidate entities, the highest matching score, and the difference between the highest and second-highest matching scores. When the confidence score is higher than the threshold (≥0.9, adjustable according to specific scenarios), the result of the rule-based matching algorithm is directly used, completely avoiding LLM calls; when the confidence score is low to medium, LLM is called to precisely select from the pre-filtered candidate entity list. This mechanism achieves a deep integration of the rule-based matching algorithm and LLM, rather than a simple pre- and post-processing relationship. When LLM needs to be called, only a list of candidate entities pre-screened by LLM rules (usually 3-5, which can be adjusted according to the specific scenario) is provided instead of full enumeration, which can greatly reduce the length of input text (reduced by 80-90%) and reduce the computational complexity and resource consumption of LLM inference. The rule matching module and the LLM module are deeply integrated at the architecture level. The result of the rule matching algorithm (candidate entity list) is directly used as the input of LLM. LLM makes precise selection from the candidate entity list, rather than performing entity recognition independently. The result of the rule matching algorithm and the result of LLM selection are fused to form the final entity recognition result. This architecture design realizes the organic unity of rule matching and LLM, rather than a simple pre- and post-processing relationship. Through a multi-level progressive fuzzy matching algorithm, the ability to identify complete standard entity information from incomplete entity information is realized. This capability includes the following key technical features: (1) Prefix matching mechanism: able to identify complete entities (such as "Dell") from partial prefixes (such as "De"). (2) Subsequence matching mechanism: able to identify complete entities (such as "UHD") from subsequences (such as "UD"). (3) Partial keyword matching mechanism: It can identify complete entities (such as "36.6 FB") from partial keywords (such as "36.6"). (4) Longest matching priority mechanism: When multiple entities match, the entity with the longest matching length is selected first to improve the recognition accuracy. It has been deeply optimized for specific technical fields such as quality inspection and Chinese environment, including the following key technical features: (1) Unified processing of full-width and half-width characters: Unified processing of full-width and half-width characters (such as "()" and "()") to improve the matching accuracy. (2) Number + Chinese combination recognition: Supports the overall recognition of number + Chinese combination (such as "45 degrees" and "10.36 process") to avoid split recognition. (3) Intent phrase filtering: Identifies and filters intent expression phrases (such as "poor situation analysis") to avoid misidentification.(4) Template keyword filtering: Filter template keywords to avoid misidentifying them as entities. For example, keywords used to identify bad types are not product models or bad names, but they can affect identification and need to be filtered out in advance. (5) Multilingual mixed processing: Supports the recognition of mixed Chinese and English entities, such as "dark spot", "Bad / Shift Via DE", "Broken(MDL)-SQA", etc. Provides a configurable and flexible architecture. Each module / component in this embodiment is configurable and can be adjusted according to the needs of specific scenarios: (1) Adjustable number of rule matching algorithm levels: The number and type of matching layers can be flexibly adjusted according to the needs of specific scenarios. (2) Adjustable matching score: The matching score range of each level can be adjusted according to the needs of specific scenarios. (3) Adjustable factors: The factors required for quality scoring can be increased or decreased according to the needs of specific scenarios. (4) Adjustable quality range: The quality range can be flexibly adjusted according to the needs of specific scenarios, cost control requirements, accuracy requirements, etc. (5) Adjustable decision rules: Multi-level decision rules can be designed according to the needs of specific scenarios (such as multiple quality ranges corresponding to different entity recognition paths).

[0192] The core of the recognition method provided in this disclosure lies in its ability to dynamically combine and adjust various components according to the needs of different scenarios to achieve optimal recognition results. The following examples illustrate how to perform this combination configuration: Scenario Example 1: Cost-sensitive scenarios (e.g., large-scale batch queries); Scenario characteristics: large query volume, high cost control requirements, relatively standardized entity formats, high rule matching accuracy, high response speed requirements, and tolerance for a small number of false positives. The combined configuration strategy is as follows: (11) Hierarchical configuration of rule matching algorithm: reduce the number of levels of fuzzy matching, retain only the first 5 levels (precise matching to keyword matching), and remove low-accuracy levels such as prefix / subsequence matching.

[0193] (12) Quality range adjustment: Increase the quality range corresponding to high quality to ≥0.95, make more use of rule matching results, and reduce LLM calls.

[0194] (13) Weighting of factors required for quality scoring: Increase the weight of the score factor with the highest matching score (from 0.4 to 0.5), and decrease the weight of the number factor with the number of candidates (from 0.3 to 0.2).

[0195] (14) Decision rules: Two-level decision is adopted. For high-quality (quality score ≥ 0.95), the entity recognition result of the rule matching algorithm (path one) is directly used (first recognition result). For low confidence (quality score < 0.95), LLM is called and path three is selected.

[0196] (15) Number of candidate entities in the candidate entity list: limited to 3 to reduce the length of LLM input text.

[0197] Expected results: LLM call count reduced by 80-90%, overall cost reduced by 70-85%, and response time maintained in the millisecond range.

[0198] Scenario Example 2: Accuracy-Priority Scenarios (e.g., medical, financial fields); Scenario Characteristics: Extremely high accuracy requirements, high cost of misidentification, complex and diverse entity formats, numerous variations and incomplete information; cost can be appropriately relaxed, but accuracy must be guaranteed. The combined configuration strategy is as follows: (21) Hierarchical configuration of rule matching algorithm: Enable all 10-level matching strategies, and add extended levels such as synonym matching level and pinyin matching level.

[0199] (22) Quality range adjustment: Reduce the quality range corresponding to high quality to ≥0.85, and call LLM more for precise selection.

[0200] (23) Weight adjustment of factors required for quality scoring: increase the quality factor of candidate quality (considering matching type and matching position), and increase the type factor of entity type (different entity types have different importance).

[0201] (24) Decision rules: A three-level decision is adopted. For high quality (quality score ≥ 0.95), path one is used directly. For medium quality (quality score between 0.85 and 0.95, including 0.85 but not including 0.95), a lightweight LLM is called and path three is selected. For low quality (quality score < 0.85), a full LLM is called and path three is selected.

[0202] (25) The number of candidate entities in the candidate entity list: increased to 5, giving LLM more options.

[0203] Expected results: Recognition accuracy will be improved to over 95%, the number of LLM calls will be moderate (reduced by 40-50%), and costs will be reduced while maintaining high accuracy.

[0204] Scenario Example 3: Cold Start Scenario (e.g., new domain, new entity type). Scenario characteristics: lack of training data, inability to use deep learning methods, incomplete entity enumeration list, large number of unknown entities, requiring rapid deployment and gradual optimization. Combined configuration strategy: (31) Hierarchical configuration of rule matching algorithm: The fuzzy matching layer set (layers 8-10) is mainly used, and prefix matching, subsequence matching, partial keyword matching, etc. are supported.

[0205] (32) Quality range adjustment: Set the quality range corresponding to high quality to ≥0.80, and rely more on LLM for entity recognition and expansion.

[0206] (33) Weight adjustment of factors required for quality scoring: increase the context factor (consider the impact of contextual information on confidence), and decrease the weight of the score factor of the matching score (because the accuracy of rule matching is low). (34) Decision rules: Two-level decision-making is adopted. For high quality, the quality score is ≥0.80, and path one is selected; for low quality, the quality score is <0.80, path three is selected; more calls are made to LLM.

[0207] (35) Number of candidate entities in the candidate entity list: Increased to 5-7, allowing LLM to select or generate new entities from more candidates.

[0208] Expected results: Rapid deployment, no training data required, and gradual improvement of the entity list by supplementing the deficiencies in rule matching through LLM.

[0209] Scenario Example 4: Real-time interactive scenarios (such as intelligent customer service, online Q&A). Scenario characteristics: extremely high requirements for response speed, sensitive to user experience, diverse query types, including both standardized and complex queries requiring a balance between speed and accuracy. Combined configuration strategy: (41) Hierarchical configuration of rule matching algorithm: Use the first 7 layers of matching strategy, N=7, and remove the most ambiguous matching layer; (42) Quality range adjustment: Set the quality range corresponding to high quality to ≥0.90 to balance cost and accuracy; (43) Adjustment of the weights of factors required for quality scoring: balance the weights of each factor and keep the default configuration.

[0210] (44) Decision rules: Two-level decision is adopted. For high quality, the quality score is ≥0.90, and path one is selected; for low quality, the quality score is <0.90, path three is selected; but a fast path is added: when the number of candidates is 1 and the matching score is ≥95, the result is returned directly and the calculation of the quality score is skipped.

[0211] (45) Number of candidate entities in the candidate entity list: limited to 3-4 to ensure fast LLM response.

[0212] Expected results: High-quality queries will achieve millisecond-level response times, while low-quality queries will complete within 1-2 seconds, resulting in a good user experience.

[0213] Scenario Example 5: Multilingual Mixed Scenarios (e.g., internationalized products, cross-language entity recognition). Scenario characteristics: Entities contain multiple languages ​​(Chinese, English, Japanese, etc.), requiring handling of full-width and half-width characters, encoding conversion, etc., with significant differences in entity formats across different languages. Combined Configuration Strategy: (51) Hierarchical configuration of rule matching algorithm: add a unified processing level for full-width and half-width characters, an encoding conversion level, and a multilingual synonym matching level; (52) Quality range adjustment: Set the quality range corresponding to high quality to ≥0.88, taking into account the uncertainty brought about by multiple languages; (53) Weight adjustment of factors required for quality scoring: increase language factor (different languages ​​have different recognition difficulties), increase coding consistency factor; (54) Decision-making rules: A three-level decision-making system is adopted. For high quality (quality score ≥ 0.88), Path 1 is used directly; for medium quality (quality score between 0.7 and 0.88, including 0.7 but excluding 0.88), Path 3 is selected; for low quality (quality score < 0.7), Path 3 is selected. A language detection step is added, and different matching strategies are selected according to the language type.

[0214] (55) Number of candidate entities in the candidate entity list: limited to 4-5, taking into account the diversity of multilingual matching.

[0215] Expected results: Supports multilingual entity recognition, maintains an accuracy rate of over 90%, and has a moderate cost.

[0216] The identification method disclosed herein utilizes an intelligent decision-making mechanism based on quality scores. When the quality score of rule matching exceeds a threshold (≥0.9), it directly uses the result of the rule matching algorithm and completely avoids LLM calls, reducing overall costs by 60-80%. Existing pure LLM methods require model calls every time, resulting in high costs; existing simple rule-model combinations lack intelligent decision-making mechanisms and cannot achieve cost optimization. The rule matching path (Path 1) of this disclosure achieves millisecond-level response, far faster than the second-level response of pure LLM solutions. When the quality score is high, identification results are quickly returned without waiting for LLM inference, significantly improving user experience. Existing pure LLM methods require model inference every time, resulting in slow processing speed. This disclosure employs a dual guarantee mechanism of rule matching + LLM. Rule matching provides rapid filtering, while LLM performs precise selection from the candidate entity list, resulting in higher accuracy. Existing pure rule matching methods lack flexible matching mechanisms and have limited accuracy; while existing pure LLM methods have strong generalization capabilities, they lack rule pre-screening and are prone to misidentification. This disclosure employs a multi-level, progressive fuzzy rule-based matching algorithm, including prefix matching (e.g., "De" matches "Dell"), subsequence matching (e.g., "UD" matches "UHD"), partial keyword matching (e.g., "36.6" matches "36.6 FB"), and longest match priority mechanisms. This algorithm can accurately identify complete standard entity information (e.g., "36.6FB", "45-degree diagonal stripes Mura", "UHD") from incomplete entity information (e.g., "36.6", "45 degrees", "UD") in user questions. Existing technologies typically require complete entity information for accurate identification, often failing to identify or producing misidentifications when users provide only partial information. The rule-based matching algorithm of this disclosure is clear and interpretable; the matching process, scoring mechanism, and decision logic are all traceable, facilitating problem localization and optimization in industrial applications. In contrast, existing deep learning and pure LLM methods are black-box models, making the identification results difficult to interpret and unsuitable for industrial applications. This rule-based matching algorithm requires no training data, making it suitable for cold-start scenarios and reducing implementation costs. Existing deep learning methods require large amounts of labeled training data, making them unsuitable for new domains or cold-start scenarios. This disclosure offers deep optimizations for the quality inspection domain and the Chinese environment, including unified processing of full-width and half-width characters, recognition of combinations of numbers and Chinese characters, template keyword filtering, and intent phrase recognition, solving practical problems in the quality inspection field. Existing methods are mostly general-purpose and lack in-depth optimization for specific domains. When LLM needs to be invoked, this disclosure provides only a pre-filtered list of candidate entities (usually 3-5) in path three, instead of a full enumeration, significantly reducing the input text length (by 80-90%) and lowering the computational complexity and resource consumption of LLM inference. Existing pure LLM methods require a complete enumeration list, significantly increasing the input text length and computational complexity.This disclosure achieves a deep integration of rule matching and LLM. Through an intelligent decision-making mechanism based on quality scoring, it dynamically selects between rule matching and LLM according to specific circumstances, forming a complete hybrid architecture. Existing methods that simply combine rules with large models only involve basic pre- and post-processing relationships and lack deep integration. Therefore, this disclosure effectively combines the advantages of rule matching and LLM, possesses an intelligent decision-making mechanism, and can be optimized for the specific needs of the quality inspection field, while balancing accuracy, speed, and cost.

[0217] One of the core features of the recognition method provided in this disclosure is the adoption of an intelligent decision-making mechanism based on quality scores. When the quality score is high, the result of the rule matching algorithm is directly used (millisecond-level response), and when the quality score is low, LLM (second-level response) is invoked. The response time of entity recognition is observed by testing different question inputs. The recognition method provided in this disclosure has a response time in the millisecond range (typically <100ms) for some questions, while the response time for others may be in the second range (typically >1s), such as when a fusion mechanism of rule matching algorithm and LLM is used. The response time of entity recognition in this disclosure shows significant differences. By systematically testing questions with different matching qualities, since this disclosure uses an entity recognition path selection mechanism based on quality scores, there is a significant correlation between the response time and the quality score (questions with high quality scores have short response times, and questions with low quality scores have long response times). This correlation is not a simple linear relationship, but rather has a clear threshold boundary (e.g., a short response time when the quality score is above a certain threshold, and a long response time when it is below that threshold). If it were just a simple rule matching algorithm + LLM fusion mechanism (e.g., fixed rule preprocessing + LLM postprocessing), the response time and quality score might have a linear relationship or no obvious pattern. However, if the intelligent decision-making mechanism is based on quality scoring, when the quality score is higher than a threshold (e.g., ≥0.9), the system directly uses the result of the rule matching algorithm (millisecond-level response). When the quality score is lower than the threshold, the system calls LLM (second-level response). This results in a clear dividing point in response time near the confidence threshold: issues above the threshold are all millisecond-level, and issues below the threshold are all second-level, forming a "bimodal distribution" rather than a continuous distribution. This threshold dividing characteristic is unique to the quality scoring decision-making mechanism, and other fusion mechanisms (such as fixed rules + LLM, random selection, etc.) do not produce this characteristic. For example, if a user inputs "query product model 36.6 FB" (complete information, high match score, quality score ≥0.9), the response time is in milliseconds; if the user inputs "query product model 36.6" (incomplete information, low match score, quality score <0.9), the response time is in seconds, and this difference shows a clear dividing point near the threshold (e.g., 0.9).

[0218] The response time disclosed herein is related to the number of candidate entities: Testing with different numbers of candidate entities revealed that problems with fewer candidate entities (e.g., 1-2) have shorter response times (milliseconds), while problems with more candidate entities (e.g., 3-5) have longer response times (seconds). This difference is related to the quantity factor of candidate entities in the quality score calculation. For example, inputting "query products with model number 36.6 FB" (complete information), rule matching only finds one candidate "36.6 FB" (matching score 100 points). With a high candidate quantity factor and a comprehensive confidence level ≥ 0.9, directly using the result of the rule matching algorithm results in a response time in milliseconds (<100ms). Inputting "Query for product model 36.6" (incomplete information), the rule matching algorithm matches 3 candidate entities ["36.6 FB" (matching score 70), "36.6 HD" (matching score 65), "36.8 FB" (matching score 60)]. The quantity factor decreases, and the quality score is <0.9. The LLM is invoked to select from the 3 candidate entities, with a response time in the order of seconds (>1 second). Inputting "Query for product model 36" (even more incomplete information), the rule matching algorithm matches 5 candidate entities ["36.6 FB", "36.6 HD", "36.8 FB", "36.5 FB", "36.7 FB"]. The quantity factor decreases further, and the quality score is even lower. The LLM is invoked to select from the 5 candidate entities, with a response time also in the order of seconds. By comparison, it can be found that the fewer the number of candidate entities, the higher the quality score and the shorter the response time; the more candidate entities, the lower the quality score and the longer the response time. This correlation demonstrates that the number of candidate entities is a crucial factor in calculating the quality score. This disclosure is validated through batch testing: by testing a large number of problems of varying complexity, the response time distribution is statistically analyzed. The response time of this disclosure exhibits a clear bimodal distribution (two peaks at the millisecond and second levels), rather than a single or continuous distribution, and this bimodal distribution is correlated with factors such as the matching quality of the input problem (i.e., the quality score) and the number of candidate entities. The identification method provided in this disclosure has the following characteristics on the front-end page: (1) The front-end page displays the result immediately after some questions are entered (without loading animation or with a very short loading time), while it displays obvious loading animation or waiting prompts after some questions are entered (usually 1-3 seconds). In the pure LLM scheme, all questions will have obvious loading status, while in the pure rule matching scheme, all questions should respond instantly. (2) Result display method: The system can identify complete entities from incomplete information (such as identifying "36.6 FB" after entering "36.6") and display the complete entity information on the front end. In some cases, this identification is completed instantly (millisecond level), while in others it requires waiting (second level). (3) Error prompts: The system can return the result immediately for high-quality matching questions, and will display prompts such as "analyzing" and "identifying" for low-quality matching questions.The API call characteristics of the identification method provided in this disclosure are as follows: (1) API call frequency difference: The LLM API is not called in the processing of some problems (by monitoring network requests, log analysis, etc.), while the LLM API is called in the processing of some problems, which can prove that a confidence-based intelligent decision-making mechanism is used. (2) API call delay: The response time of some problems is consistent with the LLM API call time (second level), while the response time of some problems is much faster than the LLM API call time (millisecond level). (3) Input text length: When calling the LLM API, the length of the input text is significantly smaller than the length of the full entity list (for example, the full entity list has 1000 entities, but only 3-5 candidates are passed when the API is called). (4) API call pattern: The API call does not occur every time an entity is identified, but is selectively called according to certain conditions (such as the quality interval where the quality score is located). The entity identification result characteristics of the embodiment of this disclosure are as follows: (1) Incomplete information identification capability: The system can identify complete entities (such as "36.6", "UD", "De") from partial information (such as "36.6"). FB", "UHD", "Dell"), and this recognition capability is significantly better than pure rule matching (pure rule matching usually requires complete information). (2) Matching quality score: The recognition result returned by the system includes matching score or quality score information, and the matching score range is between 60 and 100 points, and different matching types correspond to different matching score ranges / ranges. (3) Candidate entity list return: In some cases, the system returns multiple candidate entities (usually 3-5), for example, these candidate entities are sorted by matching score. (4) Recognition accuracy mode: The system has a very high recognition accuracy (close to 100%) for high-quality matching (high quality score), and a relatively low recognition accuracy for low-quality matching, but the accuracy is improved after LLM selection. Embodiments of this disclosure The system logs and performance monitoring characteristics of the system using entity recognition methods are as follows: (1) Log recording differences: The system logs show that some requests (user issues) use identifiers such as "rule matching" and "direct return", while some requests show identifiers such as "call LLM" and "LLM selection". (2) Performance indicator differences: The system performance monitoring shows that some requests have low CPU utilization, small memory usage, and short response time (milliseconds), while some requests have high CPU utilization, large memory usage, and long response time (seconds), and this difference is related to the complexity of the problem. (3) Resource consumption pattern: The resource consumption related to LLM (such as GPU utilization and API call count) is not proportional to the number of requests (the number of input texts), but is proportional to the number of low-quality matching requests.The network traffic characteristics of the entity recognition method used in this embodiment are as follows: (1) Request size difference: The API request data volume of some questions is small (containing only a small number of candidate entities), while the API request data volume of some questions is large (containing the full list of entities). (2) Response time (response latency) distribution: Network monitoring shows that the response latency shows a clear bimodal distribution (milliseconds and seconds), rather than a single distribution. (3) API call pattern: Some requests do not call the external LLM API (by monitoring network traffic), while some requests call the external LLM API. The user experience characteristics of the entity recognition method used in this embodiment are as follows: (1) Response speed experience: Users report that the response speed of some questions is extremely fast (almost imperceptible), while the response speed of some questions is slow (requiring waiting), and this difference is related to the complexity of the question. (2) Recognition accuracy experience: Users report that the system can identify complete entities from incomplete information, and this recognition is instantaneous in some cases and requires waiting in others. (3) Error handling pattern: The system rarely makes errors for high-quality matching questions, and the error rate is significantly reduced for low-quality matching questions after LLM selection.

[0219] Based on the same inventive concept, this disclosure also provides another identification method. The principle of this method in solving the problem is similar to any of the above identification methods. Therefore, the implementation of this identification method can refer to the implementation of the above identification methods, and the repeated parts will not be described again.

[0220] The specific implementation process of another identification method provided in this embodiment is as follows: 1) Obtain the input text; 2) The input text is subjected to entity recognition using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the level increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0221] In practice, a rule-matching algorithm can be used to perform entity recognition on the input text. The implementation principle of this rule-matching algorithm is the same as that of the rule-matching algorithms in any of the recognition methods mentioned above, and will not be repeated here.

[0222] As one possible implementation, this embodiment can also utilize a rule-matching algorithm for entity recognition, with the specific process as follows: Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches the rule at level i, the entity recognition result is determined based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0223] Optionally, when the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i; the entity recognition result is then determined based on this candidate entity list. For example, the candidate entity with the highest matching score in the candidate entity list can be used as the entity recognition result, or a candidate entity can be selected based on a comprehensive judgment of matching score, matching type, etc. For instance, candidate entities with a specified matching type and a matching score higher than a threshold can be selected as the entity recognition result. Alternatively, the entire candidate entity list can be used as the entity recognition result.

[0224] In practice, the specific matching process of the rule matching algorithm can be found in the description of the above embodiments, and will not be repeated here.

[0225] Optionally, the candidate entity list output by the rule matching algorithm can be used as input to a large model to reduce the length of the input text of the large model. For the scheme of combining the rule matching algorithm with a large model for entity recognition, please refer to the above content, which will not be repeated here.

[0226] As an optional implementation, at least one of the following in the rule matching algorithm is updated: the number of levels, the matching score range corresponding to each level, and the matching type corresponding to each level.

[0227] Based on the same inventive concept, this disclosure also provides an identification device. Since this device is the same as the device in the method of this disclosure, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0228] The identification device includes a processing unit for performing the steps of any of the above-described identification methods.

[0229] As one possible implementation, the processing unit is specifically configured to perform: Get the input text; Determine the entity recognition path of the input text, wherein the entity recognition path includes the path of performing entity recognition on the input text based on rule matching algorithms and / or large models; The entity recognition result of the input text is output based on the entity recognition path.

[0230] As one possible implementation, the processing unit is specifically configured to perform: The input text is used to perform entity recognition using a rule-matching algorithm to obtain a candidate entity list; The entity recognition path is determined based on the quality score of the candidate entity list.

[0231] As one possible implementation, the response latency of the entity recognition path to entity recognition of the input text is related to the quality score.

[0232] As one possible implementation, the processing unit is specifically configured to determine the quality score of the candidate entity list in the following manner: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the degree of difference in matching scores between different candidate entities.

[0233] As one possible implementation, the processing unit is specifically configured to perform: The quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest matching score and the second highest matching score. The quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score.

[0234] As one possible implementation method, The entity recognition path is a path for performing entity recognition on the input text based on a rule matching algorithm, and the entity recognition result output by the entity recognition path is one or more candidate entities in the candidate entity list.

[0235] As one possible implementation, the entity recognition path is to first perform entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then input the candidate entity list into a large model for entity recognition. The entity recognition result output by the entity recognition path is the output result of the large model.

[0236] As one possible implementation, the number of candidate entities in the candidate entity list is within a specified range, and the large model is used to filter entities from the candidate entity list that match the input text. The specified range is used to limit the length of the text input to the large model; and / or, The response latency of the entity recognition path to entity recognition of the input text is related to the number of candidate entities in the candidate entity list.

[0237] As one possible implementation, the entity recognition path is a path for performing entity recognition on the input text based on a large model, and the entity recognition result output by the entity recognition path is the output result of the large model; or, The entity recognition path is a path that first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm. The entity recognition result output by the entity recognition path is the corrected result.

[0238] As one possible implementation, the entity recognition path is a first path and a second path executed in parallel. The first path is the path obtained by performing entity recognition on the input text based on a rule matching algorithm, and the second path is the path obtained by performing entity recognition on the input text based on a large model. The entity recognition result output by the entity recognition path is a fusion result of the first recognition result output by the first path and the second recognition result output by the second path.

[0239] As one possible implementation, the processing unit is specifically configured to perform: The entity recognition path is updated based on the constraints of cost and / or response latency for performing entity recognition on the input text using the entity recognition path.

[0240] As one possible implementation, the rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0241] As one possible implementation method, When the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels. As the level increases, the minimum value of the matching score range decreases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0242] As one possible implementation, the processing unit is specifically configured to perform: Update at least one of the following in the rule matching algorithm: number of levels, matching score range for each level, and matching type for each level.

[0243] As one possible implementation, the processing unit is further configured to execute

[0244] Get the input text; The input text is subjected to entity recognition using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0245] As one possible implementation, the processing unit is further configured to execute

[0246] Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches the rule at level i, the entity recognition result is determined based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0247] Based on the same inventive concept, this disclosure also provides an interactive system. Since this system is the same as the system in the method of this disclosure, and the principle of solving the problem by this system is similar to that of this method, the implementation of this system can refer to the implementation of the method, and the repeated parts will not be described again.

[0248] like Figure 8 As shown, the system includes the aforementioned identification device 800, processing device 801, and display device 802, wherein: The processing device 801 is configured to perform corresponding processing using the entity recognition result output by the recognition device 800, and transmit the processing result to the display device 802 for display.

[0249] Optionally, the display device 802 is used to acquire input text; for example, a user inputs a question through the display device, and the user-input question is identified as input text. The recognition device 800 is used to perform the steps of any of the recognition methods described above.

[0250] As an optional implementation, the identification device 800 is configured to perform: Obtain input text; determine the entity recognition path of the input text, the entity recognition path including the path of entity recognition of the input text based on rule matching algorithm and / or large model; output the entity recognition result of the input text based on the entity recognition path.

[0251] As an optional implementation, the identification device 800 is specifically configured to perform: The input text is subjected to entity recognition using a rule-based matching algorithm to obtain a candidate entity list; the entity recognition path is determined based on the quality score of the candidate entity list.

[0252] As an optional implementation, the response latency of the entity recognition path to entity recognition of the input text is related to the quality score.

[0253] As an optional implementation, the identification device 800 is specifically configured to determine the quality score of the candidate entity list in the following manner: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the degree of difference in matching scores between different candidate entities.

[0254] As an optional implementation, the identification device 800 is specifically configured to perform: The quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest matching score and the second highest matching score. The quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score.

[0255] As an optional implementation, the identification device 800 is specifically configured to perform: The entity recognition path is a path for performing entity recognition on the input text based on a rule matching algorithm, and the entity recognition result output by the entity recognition path is one or more candidate entities in the candidate entity list.

[0256] As an optional implementation method, The entity recognition path is to first perform entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then input the candidate entity list into a large model for entity recognition. The entity recognition result output by the entity recognition path is the output result of the large model.

[0257] As an optional implementation method, The number of candidate entities in the candidate entity list is within a specified range. The large model is used to filter entities from the candidate entity list that match the input text. The specified range is used to limit the length of the text input to the large model; and / or, The response latency of the entity recognition path to entity recognition of the input text is related to the number of candidate entities in the candidate entity list.

[0258] As an optional implementation method, The entity recognition path is a path for performing entity recognition on the input text based on a large model, and the entity recognition result output by the entity recognition path is the output result of the large model; or, The entity recognition path is a path that first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm. The entity recognition result output by the entity recognition path is the corrected result.

[0259] As an optional implementation method, The entity recognition path consists of a first path and a second path executed in parallel. The first path is the path obtained by performing entity recognition on the input text based on a rule matching algorithm, and the second path is the path obtained by performing entity recognition on the input text based on a large model. The entity recognition result output by the entity recognition path is a fusion result of the first recognition result output by the first path and the second recognition result output by the second path.

[0260] As an optional implementation, the identification device 800 is further configured to perform: The entity recognition path is updated based on the constraints of cost and / or response latency for performing entity recognition on the input text using the entity recognition path.

[0261] As an optional implementation, the rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0262] As an optional implementation method, When the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels. As the level increases, the minimum value of the matching score range decreases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0263] As an optional implementation, the identification device 800 is further configured to perform: Update at least one of the following in the rule matching algorithm: number of levels, matching score range for each level, and matching type for each level.

[0264] Based on the same inventive concept, this disclosure also provides an interactive system. Since this system is the same as the system in the method of this disclosure, and the principle of solving the problem by this system is similar to that of this method, the implementation of this system can refer to the implementation of the method, and the repeated parts will not be described again.

[0265] The system includes the aforementioned identification device, processing device, and display device, wherein: The processing device is configured to perform corresponding processing using the entity recognition result output by the recognition device, and transmit the processing result to the display device for display.

[0266] Optionally, a display device is used to acquire input text; for example, a user inputs a question through the display device, and the user-input question is identified as input text. This recognition device is used to perform the steps of any of the recognition methods described above.

[0267] As an optional implementation, the identification device is configured to perform: Obtain input text; perform entity recognition on the input text using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0268] As an optional implementation, the identification device is specifically configured to perform: Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches the rule at level i, the entity recognition result is determined based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0269] Based on the same inventive concept, this disclosure also provides an electronic device. Since this electronic device is the same as the electronic device in the method of this disclosure, and the principle by which this electronic device solves the problem is similar to that of the method, the implementation of this electronic device can refer to the implementation of the method, and repeated details will not be described again. This electronic device includes the aforementioned identification device and executes the steps of any of the aforementioned identification methods.

[0270] Based on the same inventive concept, this disclosure also provides an identification device. Since this device is the same as the device in the method of this disclosure, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0271] like Figure 9 As shown, the device includes: Input module 900 is used to acquire input text; The path control module 901 is used to determine the entity recognition path of the input text, wherein the entity recognition path includes the path for entity recognition of the input text based on rule matching algorithms and / or large models; Output module 902 is used to output the entity recognition result of the input text based on the entity recognition path.

[0272] As an optional implementation, the path control module 901 is specifically used for: The input text is used to perform entity recognition using a rule-matching algorithm to obtain a candidate entity list; The entity recognition path is determined based on the quality score of the candidate entity list.

[0273] As an optional implementation, the response latency of the entity recognition path to entity recognition of the input text is related to the quality score.

[0274] As an optional implementation, the path control module 901 is specifically used to determine the quality score of the candidate entity list in the following manner: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the degree of difference in matching scores between different candidate entities.

[0275] As an optional implementation, the path control module 901 is specifically used for: The quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest matching score and the second highest matching score. The quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score.

[0276] As an optional implementation method, The entity recognition path is a path for performing entity recognition on the input text based on a rule matching algorithm, and the entity recognition result output by the entity recognition path is one or more candidate entities in the candidate entity list.

[0277] As an optional implementation method, The entity recognition path is to first perform entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then input the candidate entity list into a large model for entity recognition. The entity recognition result output by the entity recognition path is the output result of the large model.

[0278] As an optional implementation method, The number of candidate entities in the candidate entity list is within a specified range. The large model is used to filter entities from the candidate entity list that match the input text. The specified range is used to limit the length of the text input to the large model; and / or, The response latency of the entity recognition path to entity recognition of the input text is related to the number of candidate entities in the candidate entity list.

[0279] As an optional implementation method, The entity recognition path is a path for performing entity recognition on the input text based on a large model, and the entity recognition result output by the entity recognition path is the output result of the large model; or, The entity recognition path is a path that first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm. The entity recognition result output by the entity recognition path is the corrected result.

[0280] As an optional implementation method, The entity recognition path consists of a first path and a second path executed in parallel. The first path is the path obtained by performing entity recognition on the input text based on a rule matching algorithm, and the second path is the path obtained by performing entity recognition on the input text based on a large model. The entity recognition result output by the entity recognition path is a fusion result of the first recognition result output by the first path and the second recognition result output by the second path.

[0281] As an optional implementation, a path update module is also included, specifically used for: The entity recognition path is updated based on the constraints of cost and / or response latency for performing entity recognition on the input text using the entity recognition path.

[0282] As an optional implementation, the rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

[0283] As an optional implementation method, When the input text matches a rule at level i, a candidate entity list is generated based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels. As the level increases, the minimum value of the matching score range decreases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0284] As an optional implementation, an update module is also included, specifically for: Update at least one of the following in the rule matching algorithm: number of levels, matching score range for each level, and matching type for each level.

[0285] Based on the same inventive concept, this disclosure also provides an identification device. Since this device is the same as the device in the method of this disclosure, and the principle by which this device solves the problem is similar to that of the method, the implementation of this device can be referred to the implementation of the method, and repeated details will not be described again. The identification device includes: The input module is used to obtain the input text; The recognition module is used to perform entity recognition on the input text using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of the rule matching decreases. N is a positive integer greater than or equal to 2.

[0286] As an optional implementation, the identification module is specifically used for: Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches the rule at level i, the entity recognition result is determined based on the matching result at level i. The matching result includes the matched entity and the matching score. The matching score ranges are different at different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

[0287] Based on the same inventive concept, this disclosure provides a computer storage medium comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the identification methods described above. Since the principle by which the computer storage medium solves the problem is similar to that of the identification method, the implementation of the computer storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0288] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0289] Based on the same inventive concept, this disclosure also provides a computer program product, which includes computer program code that, when executed on a computer, causes the computer to perform any of the identification methods described above. Since the principle by which the above computer program product solves the problem is similar to that of the identification method, the implementation of the above computer program product can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0290] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0291] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0292] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0293] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0294] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0295] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A recognition method, wherein, The method includes: Get the input text; Determine the entity recognition path of the input text, wherein the entity recognition path includes the path of performing entity recognition on the input text based on rule matching algorithms and / or large models; The entity recognition result of the input text is output based on the entity recognition path.

2. The method according to claim 1, wherein, Determining the entity recognition path of the input text includes: The input text is used to perform entity recognition using a rule-matching algorithm to obtain a candidate entity list; The entity recognition path is determined based on the quality score of the candidate entity list.

3. The method according to claim 2, wherein, The response latency of the entity recognition path to entity recognition of the input text is related to the quality score.

4. The method according to claim 2, wherein, The quality score of the candidate entity list is determined as follows: The quality score of the candidate entity list is determined based on at least one of the following: the number of candidate entities in the candidate entity list, the matching score between the candidate entities and the rules, and the degree of difference in matching scores between different candidate entities.

5. The method according to claim 4, wherein, Determining the quality score of the candidate entity list includes: The quality score is determined based on the quantity factor, the score factor of the highest matching score between the candidate entity and the rule, and the difference factor of the difference between the highest matching score and the second highest matching score. The quantity factor is negatively correlated with the quality score, the score factor is positively correlated with the quality score, and the difference factor is positively correlated with the quality score.

6. The method according to any one of claims 1-5, wherein, The entity recognition path is a path for performing entity recognition on the input text based on a rule matching algorithm, and the entity recognition result output by the entity recognition path is one or more candidate entities in the candidate entity list.

7. The method according to any one of claims 1-5, wherein, The entity recognition path is to first perform entity recognition on the input text based on a rule matching algorithm to obtain a candidate entity list, and then input the candidate entity list into a large model for entity recognition. The entity recognition result output by the entity recognition path is the output result of the large model.

8. The method according to claim 7, wherein, The number of candidate entities in the candidate entity list is within a specified range. The large model is used to filter entities from the candidate entity list that match the input text. The specified range is used to limit the length of the text input to the large model; and / or, The response latency of the entity recognition path to entity recognition of the input text is related to the number of candidate entities in the candidate entity list.

9. The method according to any one of claims 1-5, wherein, The entity recognition path is a path for performing entity recognition on the input text based on a large model, and the entity recognition result output by the entity recognition path is the output result of the large model; or, The entity recognition path is a path that first performs entity recognition on the input text based on a large model to obtain candidate entities, and then corrects the candidate entities based on a rule matching algorithm. The entity recognition result output by the entity recognition path is the corrected result.

10. The method according to any one of claims 1-5, wherein, The entity recognition path consists of a first path and a second path executed in parallel. The first path is the path obtained by performing entity recognition on the input text based on a rule matching algorithm, and the second path is the path obtained by performing entity recognition on the input text based on a large model. The entity recognition result output by the entity recognition path is a fusion result of the first recognition result output by the first path and the second recognition result output by the second path.

11. The method according to any one of claims 1-5, wherein, The method also includes: The entity recognition path is updated based on the constraints of cost and / or response latency for performing entity recognition on the input text using the entity recognition path.

12. The method according to claim 1, wherein, The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

13. The method according to claim 12, wherein, When the input text matches the rule at level i, a candidate entity list is generated based on the matching result at level i. The matching result includes the matched entities and the matching score. The matching score ranges differ for different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

14. The method according to claim 12, wherein, Also includes: Update at least one of the following in the rule matching algorithm: number of levels, matching score range for each level, and matching type for each level.

15. An identification method, wherein, The method includes: Get the input text; The input text is subjected to entity recognition using a rule matching algorithm to determine the entity recognition result of the input text. The rule matching algorithm includes N levels of rules. As the number of levels increases, the accuracy of rule matching decreases. N is a positive integer greater than or equal to 2.

16. The method according to claim 15, wherein, Entity recognition is performed on the input text sequentially according to the order of each level in the rule matching algorithm; When the input text matches a rule at level i, the entity recognition result is determined based on the matching result at level i; the matching result includes the matched entity and the matching score. The matching score ranges differ for different levels, and the minimum value of the matching score range decreases as the level increases. When the input text does not match the rule at level i, the rule at level i+1 is used to match the input text, where i is an integer greater than or equal to 1 and less than or equal to N-1.

17. An identification device, wherein, The apparatus includes a processing unit for performing the steps of the method as described in any one of claims 1 to 16.

18. An interactive system, wherein, Includes the identification device, processing device, and display device as described in claim 17, wherein: The processing device is configured to perform corresponding processing using the entity recognition result output by the recognition device, and transmit the processing result to the display device for display.

19. The system according to claim 18, wherein, The identification device is specifically configured to perform: Get the input text; Determine the entity recognition path of the input text, wherein the entity recognition path includes the path of performing entity recognition on the input text based on rule matching algorithms and / or large models; The entity recognition result of the input text is output based on the entity recognition path.

20. The system according to claim 19, wherein, The identification device is specifically configured to perform: The input text is used to perform entity recognition using a rule-matching algorithm to obtain a candidate entity list; The entity recognition path is determined based on the quality score of the candidate entity list.

Images