LLM-based semi-closed domain intelligent instruction analysis method and apparatus, and medium

Through the instruction analysis method of large language model and inference model based on LLM, the problems of inaccurate command matching and inflexible parameter management in traditional methods are solved, and higher accuracy and robustness are achieved, and suitable for semi-closed domains Intelligent instruction analysis reduces system complexity and maintenance costs.

CN118939316APending Publication Date: 2024-11-12INSPUR WORLDWIDE SERVICES LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410588674.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-13
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional natural language instruction analysis methods have problems such as low robustness, high cost, poor generalization ability, low efficiency and inflexible parameter management in identifying instructions, extracting parameters and parameter alignment, resulting in insufficient in-depth understanding of user intentions and instruction execution. Inaccurate.

Method used

Using a large language model and inference model based on LLM, we use calculating the matching distance between user problems and instruction description sets and keyword sets, extract parameters, and realize business parameters alignment through dynamic parameter domains, and construct a registered parameter domain to improve instruction analysis. accuracy and flexibility.

Benefits of technology

Improve the accuracy and efficiency of instruction matching, a deep understanding of user intentions, ensure that parameters match requirements, and are suitable for semi-closed domains, reducing system complexity and maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939316A_ABST
    Figure CN118939316A_ABST
Patent Text Reader

Abstract

The invention provides an LLM-based semi-closed domain intelligent instruction analysis method and apparatus, and a medium, and belongs to the technical field of artificial intelligence. According to the technical scheme, the method comprises the steps of obtaining a user question, an instruction description set and an instruction keyword set; calculating a direct matching distance with the instruction description set and the instruction keyword set, and selecting the most matched instruction; calculating an instruction prompt word set according to the user question based on a large language model, and extracting a parameter set required by instruction execution from the user question; constructing a registrable dynamic parameter domain based on a push mechanism, and realizing parameter alignment of a problem parameter domain service party through the parameter domain; and directly extracting an extraction value without the parameters of parameter alignment. The technology is realized based on a pre-trained large language model technology, and meanwhile, a parameter alignment process based on a registrable parameter domain is introduced, so that the method has higher accuracy, robustness and generalization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a semi-closed domain intelligent instruction parsing method, device and medium based on LLM, belonging to the technical field of artificial intelligence. Background Art

[0002] With the continuous development of natural language processing technology and the continuous change of human-computer interaction, intelligent system control based on natural language instructions has become increasingly important. To achieve a natural language instruction parsing process with high accuracy, three main problems need to be solved: the first is how to identify an instruction, the second is how to extract the parameters in the instruction, and the third is how to align the extracted parameters with the business system to improve the robustness of the system.

[0003] Traditional solutions focus more on the first and second problems and ignore the third problem, resulting in low robustness and poor usability of the instruction system. At the same time, traditional instruction parsing methods require training of proprietary models, which have high construction costs and poor generalization capabilities. In addition, instruction matching in traditional methods is inaccurate and inefficient, the understanding of user intent is not deep enough, and the parsing of user questions is inaccurate. The automatic extraction of parameters in traditional methods does not match the requirements of instruction execution, and parameter management is inflexible and difficult to adapt to business changes. Summary of the invention

[0004] The purpose of the present invention is to provide a semi-closed domain intelligent instruction parsing method, device and medium based on LLM, which solves the above problems by constructing an instruction parsing method through LLM technology and PLM technology, and has higher accuracy, robustness and generalization than traditional methods.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions: Obtain user questions, instruction description sets, and instruction keyword sets; Calculate the direct matching distance between the instruction description set and the instruction keyword set, and select the best matching instruction; Based on the large language model, the user's question and the instruction prompt word set are calculated, and the parameter set required for instruction execution is extracted from the user's question; A registrable dynamic parameter domain is constructed based on the push mechanism, and parameter alignment of the problem parameter domain business party is achieved through the parameter domain; for parameters that do not require parameter alignment, the extracted values ​​are directly taken.

[0006] Preferably, the specific method of calculating the direct matching distance between the instruction description set and the instruction keyword set is as follows: Vectorize the user question and instruction description set; Calculate the instruction similarity and select the most relevant instruction greater than a threshold; the instruction similarity The calculation formula is as follows: , in, Indicates the semantic similarity of instructions, Indicates the instruction keyword similarity.

[0007] Preferably, the instruction semantic similarity calculation formula is as follows: , in, represents the vectorized user problem, Indicates vectorized instruction description; Preferably, the instruction keyword similarity calculation formula is as follows: , in, Indicates a user problem. Indicates a command keyword.

[0008] Preferably, the parameter alignment process calculates the distance between the extracted parameter and the in-domain parameter by minimizing the distance between the extracted parameter and the in-domain parameter. , the specific formula is as follows: , in, Represents the extraction parameters, Indicates a domain parameter.

[0009] The advantage of the present invention is that the present invention can accurately select the most matching instruction by calculating the matching distance between the user question and the instruction description set and the instruction keyword set, which solves the problem of inaccurate instruction matching and low efficiency in traditional methods.

[0010] Based on a large language model, this method can deeply understand the intention of user questions and calculate their matching degree with the instruction prompt word set. This helps to parse user questions more accurately and improve the accuracy of instruction execution.

[0011] Automatically extracting the parameter set required for instruction execution from user questions eliminates the tedious process of manual parameter input and improves work efficiency. At the same time, this method can ensure that the extracted parameters match the requirements of instruction execution and improve the accuracy of instruction execution.

[0012] By constructing a registrable dynamic parameter domain through the push mechanism, this method achieves the alignment of the problem parameter domain with the business side parameters. This solves the problem of inflexible parameter management and difficulty in adapting to business changes in traditional methods. At the same time, for parameters that do not require parameter alignment, this method can directly obtain values, further improving work efficiency.

[0013] This method is suitable for intelligent instruction parsing in semi-closed domains, that is, limiting the diversity and complexity of instructions within a certain range while maintaining a certain degree of flexibility and scalability. This helps to reduce system complexity and maintenance costs while ensuring the accuracy of instruction parsing. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0015] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Example 1 like Figure 1 As shown, a semi-closed domain intelligent instruction parsing method based on LLM is implemented by the following technical solutions: S1: Obtain user questions, instruction description sets, and instruction keyword sets.

[0018] S2: Calculate the direct matching distance between the instruction description set and the instruction keyword set, and select the best matching instruction.

[0019] S3: Calculate the set of instruction prompt words based on the user question based on the large language model, and extract the parameter set required for instruction execution from the user question.

[0020] S4: Construct a registrable dynamic parameter domain based on the push mechanism, and use the parameter domain to align the parameters of the problem parameter domain business party; for parameters that do not require parameter alignment, directly extract the value.

[0021] As a refinement of step S2, calculating the direct matching distance between the instruction description set and the instruction keyword set includes the following steps: Vectorize user instructions.

[0022] Vectorized instruction description set.

[0023] Calculate the instruction semantic similarity. The calculation formula of instruction semantic similarity is as follows: , in, represents the vectorized user problem, Indicates vectorized instruction description.

[0024] Calculate the instruction keyword similarity. The instruction keyword similarity calculation formula is as follows: , in, Indicates a user problem. Indicates a command keyword.

[0025] Calculate instruction similarity, the instruction similarity The calculation formula is as follows: , in, Indicates the semantic similarity of instructions, Indicates the instruction keyword similarity.

[0026] The most relevant instruction greater than a threshold is selected.

[0027] As a refinement of step S3, the specific method of extracting the parameter set required for instruction execution from the user question is as follows: According to the hit instruction, obtain the LLM configuration item and construct the parameter extraction prompt.

[0028] Extracting parameters from instructions using a large language model , the specific formula is as follows: .

[0029] As a refinement of step S4, the specific implementation steps are as follows: Get the parameter domain according to the instruction and parameters.

[0030] Searches for the closest parameter that is greater than the threshold in the parameter domain and returns it.

[0031] If the parameter domain is not specified, the extracted parameters are returned directly.

[0032] The parameter alignment process calculates the distance from the extracted parameters to the parameters in the domain by minimizing the distance between the extracted parameters and the parameters in the domain. , the specific formula is as follows: , in, Represents the extraction parameters, Indicates a domain parameter.

[0033] Example 2 The embodiment of the present disclosure also provides a semi-closed domain intelligent instruction parsing device based on LLM, including a processor and a memory. Optionally, the device may also include a communication interface and a bus. Among them, the processor, the communication interface, and the memory can communicate with each other through the bus. The communication interface can be used for information transmission. The processor can call the logic instructions in the memory to execute the semi-closed domain intelligent instruction parsing method based on LLM of the above embodiment.

[0034] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0035] The memory, as a computer-readable storage medium, can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes the functional application and data processing by running the program instructions / modules stored in the memory, that is, the semi-closed domain intelligent instruction parsing method based on LLM in the above embodiment is implemented.

[0036] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory.

[0037] The embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned LLM-based semi-closed domain intelligent instruction parsing method.

[0038] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0039] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A semi-closed domain intelligent instruction parsing method based on LLM, characterized in that: include: Obtain user questions, instruction description sets, and instruction keyword sets; Calculate the direct matching distance between the instruction description set and the instruction keyword set, and select the best matching instruction; Based on the large language model, the user's question and the instruction prompt word set are calculated, and the parameter set required for instruction execution is extracted from the user's question; Based on the push mechanism, a registrable dynamic parameter domain is constructed, and parameter alignment of the business party of the problem parameter domain is achieved through the parameter domain; There is no need to align the parameters, just extract the values ​​directly.

2. The semi-closed domain intelligent instruction parsing method based on LLM according to claim 1 is characterized in that: The specific method for calculating the direct matching distance between the instruction description set and the instruction keyword set is as follows: Vectorize the user question and instruction description set; Calculate the instruction similarity and select the most relevant instruction greater than a threshold; the instruction similarity The calculation formula is as follows: , in, Indicates the semantic similarity of instructions, Indicates the instruction keyword similarity.

3. The semi-closed domain intelligent instruction parsing method based on LLM according to claim 2 is characterized in that: The formula for calculating instruction semantic similarity is as follows: , in, represents the vectorized user problem, Indicates vectorized instruction description.

4. The semi-closed domain intelligent instruction parsing method based on LLM according to claim 3 is characterized in that: The formula for calculating instruction keyword similarity is as follows: , in, Indicates a user problem. Indicates a command keyword.

5. The semi-closed domain intelligent instruction parsing method based on LLM according to claim 1 is characterized in that: The parameter alignment process calculates the distance from the extracted parameters to the parameters in the domain by minimizing the distance between the extracted parameters and the parameters in the domain. , the specific formula is as follows: , in, Represents the extraction parameters, Indicates a domain parameter.

6. A semi-closed domain intelligent instruction parsing device based on LLM, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the LLM-based semi-closed domain intelligent instruction parsing method as described in any one of claims 1-5 when running the program instructions.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.