Entity identification method, system and equipment for operator business performance index based on large model, and medium

Abstract

The invention discloses an entity identification method, system and equipment for operator business performance indexes based on a large model, and a medium, belongs to the technical field of artificial intelligence, and aims to solve the technical problems of inaccurate identification, frequent ambiguity, high efficiency and the like generally existing in the existing entity identification technology. According to the technical scheme, efficient, accurate and comprehensive named entities and conversion services are provided by combining a large model, AC keyword matching, similarity search and a natural language processing technology; the method specifically comprises the following steps: constructing an entity library and a vector library; constructing an entity identification fine tuning data set; finely adjusting the large model; identifying a large model entity; performing AC accurate retrieval; performing vector similarity retrieval; operator entity conversion; outputting an index list; and integration: sorting AC accurate retrieval results, vector similarity retrieval results, operator entity conversion results and index list output results according to similarity relevancy from high to low, integrating the results into a unified JSON format and outputting the unified JSON format, and providing standardized data for service calling.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, system, device and medium for entity recognition of operator service performance indicators based on a large model. Background Art

[0002] Named entity recognition and conversion technology aims to accurately capture entity information required by the business from natural language and standardize it. This technology is further combined with the imported operator business performance indicators (covering indicator name, ID, default network element and granularity) to provide complete indicator information through vector similarity matching.

[0003] At present, the mainstream methods of named entity recognition include rule-based, machine learning and deep learning. Although these methods rely on dictionaries, templates and regular expressions to a certain extent, they are incapable of dealing with ambiguous problems. Especially in actual business scenarios, due to the incompleteness of entity representation, these traditional methods often face the dilemma of recognition failure, which brings great challenges to entity extraction and interface calls in user dialogues.

[0004] Therefore, how to provide efficient, accurate and comprehensive named entity recognition and conversion services is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical task of the present invention is to provide an entity recognition method, system, device and medium based on a large model of operator business performance indicators to solve the problems of inaccurate recognition and frequent ambiguity that are common in existing entity recognition technologies.

[0006] The technical task of the present invention is achieved in the following way: a method for entity recognition of operator service performance indicators based on a large model, which combines large models, AC keyword matching, similarity search and natural language processing (NPL) technology to provide efficient, accurate and comprehensive named entity and conversion services; specifically as follows:

[0007] Build entity library and vector library: collect entities and corresponding values ​​required by operators' services, build ontology table and entity table, and store them in database; convert entity values ​​in entity table into vectors through vector embedding service (Embedding model), and store them in vector library for similarity retrieval;

[0008] Constructing entity recognition fine-tuning datasets: Based on questions and entities in business scenarios, constructing datasets in Alpaca format. The Alpaca format datasets will serve as the basis for subsequent model fine-tuning to ensure that the model can identify key entities in the business.

[0009] Fine-tuning large models: Using domestic open source large models, only some parameters are trained through the LoRA (Low-Rank Adaptation) fine-tuning method, so as to quickly adapt to business data. This not only reduces training parameters, but also reduces GPU memory requirements, while improving the model's understanding of business knowledge;

[0010] Large model entity recognition: Convert the command prompt words into prompt word templates that the large model can understand, merge the user input with the prompt word template, transmit the fine-tuned large model for entity recognition, obtain the large model output, and then obtain accurate entity information;

[0011] AC Precision Retrieval: Use AC precision retrieval technology to segment and retrieve entities output by large models;

[0012] Vector similarity retrieval: Convert the unidentified entity values ​​in AC precise retrieval into vectors, and perform cosine similarity retrieval in the vector library to complete the missing entity information;

[0013] Operator entity conversion: For special entities with indicator names in operator services, the most relevant indicator information is output through vector similarity matching and sorting;

[0014] Index list output: The acquired JSON format data is sorted into an index list according to the agreed fields (such as NENAME, INDICATE) and similarity matching results;

[0015] Integration: The results of AC precision search, vector similarity search, operator entity conversion, and indicator list output are sorted from high to low according to similarity relevance, and integrated into a unified JSON format output to provide standardized data for business calls.

[0016] Preferably, the Alpaca format dataset refers to a dataset containing instruction (instruction prompt words), input (input text) and output (expected output). The dataset is used to simulate various business scenarios to train the model to accurately identify entities.

[0017] As a preferred method, fine-tuning the large model is as follows:

[0018] Fine-tuning freezes the pre-trained model weights and injects the trainable rank decomposition matrix into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks and lowering GPU memory requirements. At the same time, through fine-tuning of business scenario data, the large model's ability to understand business knowledge is improved.

[0019] As a preferred method, AC precise search is as follows:

[0020] For each entity, a corresponding AC retrieval tree is constructed, and efficient and fast retrieval is performed in the AC retrieval tree;

[0021] After the retrieval is completed, the entity ID is used to search and obtain the corresponding conversion value of the entity in the entity library.

[0022] Preferably, the large model uses chatglm or Tongyi Qianwen.

[0023] Preferably, the database uses MySQL or PostgreSQL;

[0024] The vector library uses pgvector or milvus.

[0025] An entity recognition system for operator service performance indicators based on a large model, the system comprising:

[0026] Construction module 1 is used to determine the ontology and entity required according to the business scenario, use MySQL to build the entity table, and convert the entity value into a vector and store it in the vector database through the Embedding model;

[0027] Building module 2 is used to build Alpaca format datasets based on questions and entities in business scenarios;

[0028] The fine-tuning module is used to inject fine-tuning knowledge into the large model using the domestic open source large model and LoRA fine-tuning method;

[0029] The recognition module is used to convert the instruction prompt words into the large model prompt word template, merge the user input with the template, pass it to the fine-tuned large model, and obtain the large model output;

[0030] Retrieval module 1 is used to segment the output entities of the large model. For each entity, an AC retrieval tree is constructed and the entity value is searched in the corresponding retrieval tree. After the retrieval is completed, the corresponding entity conversion value is searched in the entity library through the retrieved entity ID.

[0031] Retrieval module 2 is used to convert entity values ​​into vectors when the recognized entities are incomplete and cannot be accurately retrieved through AC, and to perform cosine similarity search in the vector library to complete the entities that cannot be recognized by AC accurate retrieval;

[0032] The conversion module is used to output the most relevant indicator information for the special entities of the indicator names in the operator's business through vector similarity matching and sorting;

[0033] The output module is used to organize the acquired JSON format data into an indicator list according to the agreed fields (such as NENAME, INDICATE) and similarity matching results;

[0034] The integration module is used to sort the results of AC precision search, vector similarity search, operator entity conversion and indicator list output from high to low according to similarity relevance, and integrate them into a unified JSON format output to provide standardized data for business calls.

[0035] As a preferred embodiment, the working process of the system is as follows:

[0036] (1) The user inputs the conversation content;

[0037] (2) The large model preliminarily identifies elements and outputs them in the form of “element name: element value”;

[0038] (3) AC precise search, precise search for each element;

[0039] (4) Perform similarity retrieval on the element values ​​of each entity;

[0040] (5) Merge the recognition results of AC precise retrieval and similarity retrieval and remove duplicates;

[0041] (6) Vector matching of operator service-related indicators;

[0042] (7) Output the element recognition results and the JSON output sorted by similarity value.

[0043] An electronic device comprising: a memory and at least one processor;

[0044] Wherein, the memory stores a computer program;

[0045] The at least one processor executes the computer program stored in the memory, so that the at least one processor performs the entity recognition method of operator business performance indicators based on the large model as described above.

[0046] A computer-readable storage medium stores a computer program, which can be executed by a processor to implement the entity identification method of operator business performance indicators based on a large model as described above.

[0047] The entity identification method, system, device and medium of the operator service performance indicator based on the large model of the present invention have the following advantages:

[0048] (i) The present invention fine-tunes the large model based on data from specific business scenarios, enabling it to accurately identify various entities in vertical fields;

[0049] (ii) The large model of the present invention not only has excellent recognition ability, but also exhibits strong reasoning ability, and can accurately identify entities even when faced with entities that do not appear in the training set;

[0050] (III) In order to further improve the accuracy and efficiency of recognition, the present invention combines the technologies of AC precise retrieval and vector similarity retrieval, carefully adjusts and optimizes the results output by the large model, and converts them into the standardized data format required by the business; in particular, for specific entities such as time, advanced NLP algorithms are used for conversion to ensure the accuracy and consistency of the data, which not only realizes the effective application of large models in vertical fields, but also can more accurately identify and convert entities in user dialogue and conversational data query scenarios, providing high-quality entity recognition services for downstream tasks, greatly promoting the automation level of business processes, and helping users accurately identify key indicators, place names and other elements in natural language, so as to more effectively meet business needs;

[0051] (IV) The present invention aims to solve the problems of inaccurate identification and frequent ambiguity that are common in existing entity identification technologies. In particular, when processing the names of operator service-related indicators, the present invention can significantly improve the accuracy of identification and provide more reliable and efficient support for interface calls and data processing of related services;

[0052] (V) The present invention combines an efficient keyword search AC algorithm to achieve accurate identification and conversion of complex entities in business scenarios, thereby ensuring efficiency and accuracy from data preparation to final output. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention is further described below in conjunction with the accompanying drawings.

[0054] Attached Figure 1 A flowchart of an entity identification method for operator business performance indicators based on a large model;

[0055] Attached Figure 2 The flowchart of the working process of the entity recognition system of operator business performance indicators based on the big model. DETAILED DESCRIPTION

[0056] The entity identification method, system, device and medium of the operator service performance indicator based on the big model of the present invention are described in detail below with reference to the drawings and specific embodiments of the specification.

[0057] Embodiment 1:

[0058] As attached Figure 1 As shown, this embodiment provides an entity recognition method for operator service performance indicators based on a large model, which combines a large model, AC keyword matching, similarity search, and natural language processing (NPL) technology to provide efficient, accurate, and comprehensive named entity and conversion services; the details are as follows:

[0059] S1. Build entity library and vector library: collect entities and corresponding values ​​of entities required by operators' services, build ontology table and entity table, and store them in database; convert entity values ​​in entity table into vectors through vector embedding service (Embedding model), and store them in vector library for similarity retrieval;

[0060] S2. Build an entity recognition fine-tuning dataset: Based on the questions and entities in the business scenario, build an Alpaca-format dataset. The Alpaca-format dataset will serve as the basis for subsequent model fine-tuning to ensure that the model can identify key entities in the business.

[0061] S3. Fine-tune the large model: Using the domestic open source large model, only some parameters are trained through the LoRA (Low-Rank Adaptation) fine-tuning method, so as to quickly adapt to business data. This not only reduces the training parameters, but also reduces the GPU memory requirements, while improving the model's understanding of business knowledge;

[0062] S4, large model entity recognition: convert the instruction prompt word into a prompt word template that the large model can understand, merge the user input with the prompt word template, transmit the fine-tuned large model for entity recognition, obtain the large model output, and then obtain accurate entity information;

[0063] S5, AC precise retrieval: Use AC precise retrieval technology to segment and retrieve entities output by large models;

[0064] S6, vector similarity retrieval: convert the entity values ​​not identified in the AC precise retrieval into vectors, and perform cosine similarity retrieval in the vector library to complete the missing entity information;

[0065] S7, operator entity conversion: For special entities of indicator names in operator services, the most relevant indicator information is output through vector similarity matching and sorting;

[0066] S8, indicator list output: the obtained JSON format data is sorted into an indicator list according to the agreed fields (such as NENAME, INDICATE) and similarity matching results;

[0067] S9. Integration: Sort the results of AC precision search, vector similarity search, operator entity conversion, and indicator list output from high to low according to similarity relevance, and integrate them into a unified JSON format output to provide standardized data for business calls.

[0068] The Alpaca format data set in step S2 of this embodiment refers to a data set containing instruction (instruction prompt words), input (input text) and output (expected output). The data set is used to simulate various business scenarios to train the model to accurately identify entities.

[0069] Questions should be as diverse as possible, generating large model instruction prompts - instruction, for example: "Please identify the place, location, and time entities in the following sentence. If an entity is not identified, please answer no"; generate input - input, for example: "What is the weather like in a certain place today"; generate corresponding output - output, for example: "Place: somewhere; Location: somewhere; Time: today".

[0070] The fine-tuning of the large model in step S3 of this embodiment is specifically as follows:

[0071] Fine-tuning freezes the pre-trained model weights and injects the trainable rank decomposition matrix into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks and lowering GPU memory requirements. At the same time, through fine-tuning of business scenario data, the large model's ability to understand business knowledge is improved.

[0072] The AC accurate search in step S5 of this embodiment is specifically as follows:

[0073] S501, for each entity, construct a corresponding AC search tree, and perform efficient and fast search in the AC search tree;

[0074] S502: After the search is completed, the conversion value corresponding to the entity is searched in the entity library according to the entity ID and obtained.

[0075] The large model in step S3 of this embodiment adopts chatglm or Tongyi Qianwen.

[0076] In this embodiment, the database in step S1 adopts MySQL or PostgreSQL.

[0077] The vector library in step S1 of this embodiment adopts pgvector or milvus.

[0078] Embodiment 2:

[0079] This embodiment provides an entity recognition system for operator service performance indicators based on a large model, the system comprising:

[0080] Construction module 1 is used to determine the ontology and entity required according to the business scenario, use MySQL to build the entity table, and convert the entity value into a vector and store it in the vector database through the Embedding model;

[0081] Building module 2 is used to build Alpaca format datasets based on questions and entities in business scenarios;

[0082] The fine-tuning module is used to inject fine-tuning knowledge into the large model using the domestic open source large model and LoRA fine-tuning method;

[0083] The recognition module is used to convert the instruction prompt words into the large model prompt word template, merge the user input with the template, pass it to the fine-tuned large model, and obtain the large model output;

[0084] Retrieval module 1 is used to segment the output entities of the large model. For each entity, an AC retrieval tree is constructed and the entity value is searched in the corresponding retrieval tree. After the retrieval is completed, the corresponding entity conversion value is searched in the entity library through the retrieved entity ID.

[0085] Retrieval module 2 is used to convert entity values ​​into vectors when the recognized entities are incomplete and cannot be accurately retrieved through AC, and to perform cosine similarity search in the vector library to complete the entities that cannot be recognized by AC accurate retrieval;

[0086] The conversion module is used to output the most relevant indicator information for the special entities of the indicator names in the operator's business through vector similarity matching and sorting;

[0087] The output module is used to organize the acquired JSON format data into an indicator list according to the agreed fields (such as NENAME, INDICATE) and similarity matching results;

[0088] The integration module is used to sort the results of AC precision search, vector similarity search, operator entity conversion and indicator list output from high to low according to similarity relevance, and integrate them into a unified JSON format output to provide standardized data for business calls.

[0089] As attached Figure 2 As shown in the figure, the working process of the system is as follows:

[0090] (1) The user inputs the conversation content;

[0091] (2) The large model preliminarily identifies elements and outputs them in the form of “element name: element value”;

[0092] (3) AC precise search, precise search for each element;

[0093] (4) Perform similarity retrieval on the element values ​​of each entity;

[0094] (5) Merge the recognition results of AC precise retrieval and similarity retrieval and remove duplicates;

[0095] (6) Vector matching of operator service-related indicators;

[0096] (7) Output the element recognition results and the JSON output sorted by similarity value.

[0097] Embodiment 3:

[0098] This embodiment also provides an electronic device, including: a memory and a processor;

[0099] Wherein, the memory stores computer-executable instructions;

[0100] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the entity identification method for operator business performance indicators based on a large model in any embodiment of the present invention.

[0101] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0102] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0103] Embodiment 4:

[0104] This embodiment also provides a computer-readable storage medium, which stores a plurality of instructions, which are loaded by a processor to enable the processor to execute the entity identification method of the operator service performance indicator based on the large model in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0105] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0106] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0107] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0108] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for entity identification of operator service performance indicators based on a large model, characterized in that: This method combines large models, AC keyword matching, similarity search and natural language processing technology to provide efficient, accurate and comprehensive named entity and conversion services; the details are as follows: Build entity library and vector library: collect entities and corresponding values ​​required by operators' services, build ontology table and entity table, and store them in database; convert entity values ​​in entity table into vectors through vector embedding service, and store them in vector library for similarity retrieval; Constructing entity recognition fine-tuning datasets: Based on questions and entities in business scenarios, constructing datasets in Alpaca format. The Alpaca format datasets will serve as the basis for subsequent model fine-tuning to ensure that the model can identify key entities in the business. Fine-tune the large model: Using the domestic open source large model, only some parameters are trained through the LoRA fine-tuning method to quickly adapt to business data; Large model entity recognition: Convert the command prompt words into prompt word templates that the large model can understand, merge the user input with the prompt word template, transmit the fine-tuned large model for entity recognition, obtain the large model output, and then obtain accurate entity information; AC Precision Retrieval: Use AC precision retrieval technology to segment and retrieve entities output by large models; Vector similarity retrieval: Convert the unidentified entity values ​​in AC precise retrieval into vectors, and perform cosine similarity retrieval in the vector library to complete the missing entity information; Operator entity conversion: For special entities with indicator names in operator services, the most relevant indicator information is output through vector similarity matching and sorting; Indicator list output: The acquired JSON format data is sorted into an indicator list according to the agreed fields and similarity matching results; Integration: The results of AC precision search, vector similarity search, operator entity conversion, and indicator list output are sorted from high to low according to similarity relevance, and integrated into a unified JSON format output to provide standardized data for business calls.

2. The entity identification method of operator business performance indicators based on a large model according to claim 1 is characterized in that: Alpaca format datasets refer to datasets that contain instructions, input, and output. The datasets are used to simulate various business scenarios to train models to accurately identify entities.

3. The entity identification method of operator business performance indicators based on a large model according to claim 1 is characterized in that: The specific steps of fine-tuning the large model are: Fine-tuning freezes the pre-trained model weights and injects trainable rank factorization matrices into each layer of the Transformer architecture, reducing the number of trainable parameters for downstream tasks and lowering GPU memory requirements.

4. The entity identification method of operator business performance indicators based on a large model according to claim 1 is characterized in that: AC precise search is as follows: For each entity, a corresponding AC retrieval tree is constructed, and efficient and fast retrieval is performed in the AC retrieval tree; After the retrieval is completed, the entity ID is used to search and obtain the corresponding conversion value of the entity in the entity library.

5. The entity identification method of operator business performance indicators based on a large model according to claim 1 is characterized in that: The large model uses chatglm or Tongyi Qianwen.

6. The entity identification method of operator service performance indicators based on a large model according to any one of claims 1 to 5, characterized in that: The database uses MySQL or PostgreSQL; The vector library uses pgvector or milvus.

7. An entity recognition system for operator business performance indicators based on a large model, characterized in that: The system includes: Construction module 1 is used to determine the ontology and entity required according to the business scenario, use MySQL to build the entity table, and convert the entity value into a vector and store it in the vector database through the Embedding model; Building module 2 is used to build Alpaca format datasets based on questions and entities in business scenarios; The fine-tuning module is used to inject fine-tuning knowledge into the large model using the domestic open source large model and LoRA fine-tuning method; The recognition module is used to convert the instruction prompt words into the large model prompt word template, merge the user input with the template, pass it to the fine-tuned large model, and obtain the large model output; Retrieval module 1 is used to segment the output entities of the large model. For each entity, an AC retrieval tree is constructed and the entity value is searched in the corresponding retrieval tree. After the retrieval is completed, the corresponding entity conversion value is searched in the entity library through the retrieved entity ID. Retrieval module 2 is used to convert entity values ​​into vectors when the recognized entities are incomplete and cannot be accurately retrieved through AC, and to perform cosine similarity search in the vector library to complete the entities that cannot be recognized by AC accurate retrieval; The conversion module is used to output the most relevant indicator information for the special entities of the indicator names in the operator's business through vector similarity matching and sorting; The output module is used to organize the acquired JSON format data into an indicator list according to the agreed fields and similarity matching results; The integration module is used to sort the results of AC precision search, vector similarity search, operator entity conversion and indicator list output from high to low according to similarity relevance, and integrate them into a unified JSON format output to provide standardized data for business calls.

8. The entity recognition system for operator business performance indicators based on a large model according to claim 7, characterized in that: The working process of the system is as follows: (1) The user inputs the conversation content; (2) The large model preliminarily identifies elements and outputs them in the form of "element name: element value"; (3) AC precise search, precise search for each element; (4) Perform similarity retrieval on the element values ​​of each entity; (5) Merge the recognition results of AC precise retrieval and similarity retrieval and remove duplicates; (6) Vector matching of operator service-related indicators; (7) Output the element recognition results and the JSON output sorted by similarity value.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the entity identification method for operator business performance indicators based on a large model as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the entity identification method for operator business performance indicators based on a large model as described in any one of claims 1 to 6.

Images