Intelligent question answering system based on large language model

Abstract

The invention relates to the field of intelligent question answering, in particular to an intelligent question answering system based on a large language model. Comprising a first historical event information base, a second historical event information base, an input prompt generation module and a large language model, the input prompt generation module runs according to a first method and comprises the following steps: matching a to-be-judged event feature vector with event feature vectors in first event information and second event information respectively; and splicing the first sub-prompt information, the second sub-prompt information, the description information of the to-be-judged event by the user and the corresponding verdict text, and inputting the spliced information into a large language model. According to the method, the judgment texts and the judgment rules corresponding to the first few historical events with high similarity serve as the input prompts and are spliced with the user input, so that more accurate input convenient for model understanding can be generated, meanwhile, the input prompts can guide the model to generate expected output, and the illusion problem of a large language model is avoided.

Description

Background Art

[0002] Large Language Models (LLMs) have shown extensive application potential in many fields due to their powerful language understanding and generation capabilities. The "hallucination" problem of large language models refers to the phenomenon that these models may be inaccurate, self-contradictory, or inconsistent with the input information when generating text. Although large language models (LLMs) have demonstrated excellent capabilities in many natural language processing tasks, such as language understanding, text generation, question-answering systems, etc., they sometimes generate content that seems reasonable but is actually wrong or irrelevant. This phenomenon is called "hallucination."

[0003] Currently, large language models are used in some professional application scenarios to provide more accurate and diverse business services. Usually, in order to meet the ability to answer professional and non-professional questions, general large language models and special large language models are integrated in existing application scenarios. In some general scenarios with low answer accuracy requirements (such as story writing scenarios), the "hallucination" problem of large language models has little impact. However, in some vertical field scenarios with higher accuracy requirements (such as medical, financial, legal and other scenarios), users usually need the model to output more accurate and correct domain expertise to solve the corresponding problems. In the prior art, due to the "hallucination" problem of large language models, it will affect the user's accurate acquisition of professional knowledge, making it unable to be better applied to vertical fields with higher accuracy requirements. Summary of the invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is:

[0005] According to one aspect of the present invention, there is provided an intelligent question answering system based on a large language model, comprising:

[0006] A first historical event information database, a second historical event information database, an input prompt generation module and a large language model; the input prompt generation module is respectively connected to the first historical event information database, the second historical event information database and the large language model for communication;

[0007] The first historical event information database includes a plurality of first event information; the first event information is information corresponding to historical events whose determination results are favorable to the first category of users;

[0008] The second historical event information database includes a plurality of second event information, the second event information being information corresponding to historical events whose judgment results are favorable to the second category of users; the first category of users and the second category of users are the opposing parties in the historical events;

[0009] Each first event information or second event information includes a judgment text corresponding to the historical event, an event feature vector corresponding to the judgment text, and a rule list corresponding to the judgment rule used in the judgment text, wherein each judgment rule corresponds to a rule ID and an effectiveness level identifier, an implementation time identifier, and a special identifier;

[0010] The input prompt generation module operates according to a first method, which includes:

[0011] Obtaining user input information; the input information includes the user's description information of the event to be determined and the user type identifier, where the user type identifier is a first-class user identifier or a second-class user identifier;

[0012] The semantic acquisition model is used to convert the user's description information of the event to be determined into the corresponding feature vector of the event to be determined;

[0013] The feature vector of the event to be determined is used to match the event feature vectors in each of the first event information and the second event information, respectively, to obtain a first prompt rule set and a second prompt rule set; the first prompt rule set is a set of determination rules used in the top n historical events in the first event information that are ranked similar to the feature vector of the event to be determined; the second prompt rule set is a set of determination rules used in the top n historical events in the second event information that are ranked similar to the feature vector of the event to be determined;

[0014] If the user type identifier is a first type user identifier, the first prompt rule set is used as the first sub-prompt information, and the second prompt rule set is used as the second sub-prompt information;

[0015] The first sub-prompt information, the second sub-prompt information, the user's description of the event to be determined, and the judgment texts of the top n cases in the first event information and the second event information that are similar to the feature vector of the event to be determined are concatenated and input into the large language model.

[0016] Furthermore, after obtaining the first prompt rule set and the second prompt rule set, the first method further includes:

[0017] If the user type identifier is a second type user identifier, the first prompt rule set is used as the second sub-prompt information, and the second prompt rule set is used as the first sub-prompt information;

[0018] The first sub-prompt information, the second sub-prompt information, the user's description of the event to be determined, and the judgment texts of the top n cases in the first event information and the second event information that are similar to the feature vector of the event to be determined are concatenated and input into the large language model.

[0019] Furthermore, the event feature vector to be determined is used to match the event feature vectors in each of the first event information and the second event information, respectively, to obtain the first prompt rule set and the second prompt rule set, including:

[0020] Obtain a list of rules corresponding to the determination rules used in the determination text in the first event information that ranks top n in similarity to the feature vector of the event to be determined;

[0021] According to the rule ID corresponding to each determination rule, determine the applicable identifier of each determination rule; the applicable identifier is the number of repetitions of the same determination rule;

[0022] According to the implementation time mark corresponding to each of the repeated determination rules, only the determination rule with the latest implementation time is guaranteed;

[0023] Each of the remaining judgment rules is used as a rule in the first prompt rule set, wherein each judgment rule carries a corresponding effectiveness level identifier, an implementation time identifier, a special identifier, and an applicable identifier.

[0024] Furthermore, the semantic acquisition model includes the Doc2Vec model.

[0025] Furthermore, the semantic acquisition model is used to convert the user's description information of the event to be determined into a corresponding feature vector of the event to be determined, including:

[0026] Split the text corresponding to the description information into words or phrases;

[0027] Remove stop words from the text corresponding to the description information;

[0028] Input the processed text into the Doc2Vec model to generate the corresponding feature vector of the event to be determined.

[0029] Furthermore, the Doc2Vec model is obtained by retraining the pre-trained general Doc2Vec model using the corpus of the field to which the event to be determined belongs.

[0030] Furthermore, the large language model is a general large language model or a special large language model; the special large language model is obtained by retraining the general large language model using the corpus of the event field to be determined.

[0031] Furthermore, the field to which the event to be determined belongs includes the legal field.

[0032] Furthermore, the user's input information is obtained, including:

[0033] The user type identifier is determined based on the preset identity type label selected by the user.

[0034] Furthermore, the first event information is obtained according to the following steps:

[0035] Obtain the text information in the area corresponding to the judgment result in the judgment text corresponding to the historical event;

[0036] A preset regular expression is used to match and identify text information in an area corresponding to the determination result, so as to determine whether the information corresponding to the historical event is the first event information.

[0037] The present invention has at least the following beneficial effects:

[0038] The present invention sets up a first historical event information database and a second historical event information database, and the two historical event information databases respectively store case judgment information that is favorable to the plaintiff and the defendant in historical legal disputes. When there is a new event to be determined, the event feature vector extracted according to the event description information input by the user can be matched with the event feature vectors in the first historical event information database and the second historical event information database, respectively, and the judgment text corresponding to the first few historical events with high similarity and the rule list corresponding to the judgment rule are used as corresponding input prompts, and after splicing with the user's input information, a more accurate input that is easy for the model to understand can be generated to accurately capture user needs. At the same time, these input prompts can also guide the model to generate the desired output to avoid the "hallucination" problem of the large language model.

[0039] Since, in legal dispute cases, there may be applicable laws for both the plaintiff and the defendant, it is necessary to analyze the case from two directions before a more comprehensive and more meaningful answer can be generated. Therefore, the prompt information of the present invention contains high similarity information from both the first historical event information library and the second historical event information library. Correspondingly, when generating the corresponding answer, the large language model can analyze from the perspectives of both the plaintiff and the defendant, and finally generate a more comprehensive answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A structural block diagram of an intelligent question-answering system based on a large language model provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of the operation of an input prompt generation module in an intelligent question-answering system based on a large language model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0044] As a possible embodiment of the present invention, Figure 1 As shown, an intelligent question answering system based on a large language model is provided, including:

[0045] The first historical event information database, the second historical event information database, the input prompt generation module and the large language model. The input prompt generation module is respectively connected to the first historical event information database, the second historical event information database and the large language model.

[0046] The first historical event information database includes a plurality of first event information. The first event information is information corresponding to historical events whose judgment results are favorable to the first type of users. The second historical event information database includes a plurality of second event information, and the second event information is information corresponding to historical events whose judgment results are favorable to the second type of users. The first type of users and the second type of users are the opposing parties in the historical events.

[0047] The intelligent question-and-answer system in this embodiment can be an intelligent question-and-answer system in a legal consultation usage scenario. In this usage scenario, the information corresponding to the historical event can be information extracted from the text of the judgment corresponding to the actual historical dispute case. Correspondingly, the first event information can be a judgment result that the plaintiff (that is, the first type of user) wins or is favorable to the plaintiff. The second event information can be a judgment result that the defendant (that is, the second type of user) wins or is favorable to the defendant. In addition, the judgment rules in the rule list can be the legal provisions used in the judgment. The rule ID can be a combination of the number of the law to which the corresponding legal provision belongs and the number of the legal provision itself. Such as AAAbb, where AAA is the number of the law to which the corresponding legal provision belongs; bb is the legal provision's own sequential number in the law.

[0048] Among them, each first event information or second event information includes the judgment text corresponding to the historical event, the event feature vector corresponding to the judgment text, and a rule list corresponding to the judgment rule used in the judgment text, wherein each judgment rule corresponds to a rule ID and an effectiveness level identifier, an implementation time identifier, and a special identifier.

[0049] In addition, in this embodiment, three-dimensional identification is set for each legal provision to indicate the priority of the effectiveness of the legal provision in the actual use process. Specifically, the effectiveness level identification is used to indicate the priority of use of each rule, that is, to indicate the rank of the law to which it belongs (can be identified by numbers). In this embodiment, the rules may include legal provisions and implementation details. In actual use, the effectiveness of XXX law is usually greater than the implementation details of XXX law, so when reflected in the effectiveness level identification, the effectiveness level identification of XXX law is higher than the effectiveness level identification of the implementation details of XXX law. Through this effectiveness level identification, the principle of superior priority in the use of the law can be met, which is usually referred to as "superior law prevails over inferior law". In order to meet the situation where the inferior law conflicts (contradicts, contradicts) with the superior law, the superior law should be selected for application in the judiciary.

[0050] The implementation time mark is used to indicate the implementation time of each rule (it can be expressed as an implementation time string). Generally speaking, the law or implementation details whose implementation time is closest to the current time have higher accuracy of each rule. Through this implementation time mark, the principle of new law priority in the use of law can be met, that is, "new law is superior to old law". In order to meet the requirements that when there is a conflict between the general provisions or special provisions of the new and old legal documents formulated by the same agency and currently in force, the legal norms in the new legal document should be applied. Therefore, when the model makes a reference to the law, it needs to select the legal text closest to the current time according to the implementation time mark for calculation.

[0051] The special mark is used to indicate whether the corresponding law is a special law, and it can usually be marked with 0 and 1. Through this special mark, the principle of "special law prevails over general law" in the use of law can be met, so as to meet the requirement that when a special provision formulated by the same agency and both currently effective conflicts with a general provision, the special provision should be applied.

[0052] In this embodiment, by setting corresponding effectiveness level identifiers, implementation time identifiers and special identifiers for each judgment rule, the priority of use of the law can be roughly identified, and the comprehensive comparison of these identifiers can be more consistent with the rules of actual use of legal provisions. Therefore, when the large language model is used for reference, the corresponding laws and legal provisions can be more accurately and appropriately selected according to these identifiers to answer and analyze related events, so as to obtain more accurate and reasonable answers.

[0053] Specifically, the first event information and the second event information can be obtained according to the following steps:

[0054] S110: Acquire text information in an area corresponding to a judgment result in a judgment text corresponding to a historical event.

[0055] S120: Use a preset regular expression to match and identify text information in an area corresponding to the determination result, so as to determine whether the information corresponding to the historical event is the first event information.

[0056] Usually, the judicial decision will include the content corresponding to the decision result. In this part of the content, there will usually be corresponding text sentences to indicate which party (the plaintiff or the defendant) is the winner in the corresponding dispute event. For example, "support all the plaintiff's claims", "reject all the plaintiff's claims", "support some of the plaintiff's claims" or "the plaintiff / defendant shall bear the litigation costs", etc. These text expressions are usually relatively standard and consistent. Therefore, multiple regular expressions can be set in advance to make judgments, so as to determine whether the final judgment of the historical event is more favorable to the plaintiff or the defendant, and finally determine whether the corresponding information is the first event information or the second event information.

[0057] like Figure 2 As shown, the input prompt generation module operates according to the first method, and the first method includes:

[0058] S100: Obtaining user input information. The input information includes the user's description information of the event to be determined and the user type identifier, where the user type identifier is a first type user identifier or a second type user identifier.

[0059] Furthermore, S100 includes:

[0060] S101: Determine a user type identifier according to a preset identity type label selected by a user.

[0061] In this step, when the user inputs the consultation, the corresponding identity type selection label can be displayed for the user to indicate his / her identity. Of course, the existing neural network model can also be used to autonomously identify the user's identity based on the content input by the user, and the identified result can be displayed for the user to confirm.

[0062] S200: Using a semantic acquisition model, converting the user's description information of the event to be determined into a corresponding feature vector of the event to be determined.

[0063] Specifically, the semantic acquisition model may be a Doc2Vec model.

[0064] Doc2Vec (also known as Paragraph Vector) is an unsupervised learning algorithm used to generate fixed-length document vector representations. It is an extension of the Word2Vec word embedding model. Doc2Vec can capture document-level semantic information, not just individual word information. Because the Doc2Vec model has two main characteristics: regardless of the length of the input document, Doc2Vec can generate a fixed-length vector representation, and Doc2Vec not only considers the meaning of individual words but also uses the context of the entire document to enhance the representation of words. It is more suitable for converting long texts such as judgment documents in the legal field into a fixed-length vector.

[0065] Furthermore, the Doc2Vec model is obtained by retraining the pre-trained general Doc2Vec model using the corpus of the field to which the event to be judged belongs. When the field to which the event to be judged belongs is the legal field, the corresponding corpus can include texts such as judicial judgments, judicial interpretations, implementation rules of corresponding laws, operation standard guides, and interpretation information of legal provisions on some public authoritative websites. Through further training with the corresponding professional corpus in the legal field, the existing general Doc2Vec model can have a more accurate segmentation and vector conversion ability for various texts in the legal field, thereby improving the accuracy of the output vector.

[0066] Correspondingly, S200 includes:

[0067] S201: Segment the text corresponding to the description information into words or phrases.

[0068] S202: Remove stop words from the text corresponding to the description information.

[0069] S203: Input the processed text into the Doc2Vec model to generate the corresponding feature vector of the event to be judged.

[0070] The stop words in S202 refer to words that are usually filtered out in information retrieval and natural language processing. These words are usually very common and frequently appear in the text, but they contribute little to understanding the meaning of the text. For example, words such as "de" (的), "shi" (是), "zai" (在), "he" (和). The method of converting text information into a fixed-length vector in S200 can also be used in extracting the event feature vectors corresponding to the judgment texts in the first event information or the second event information.

[0071] S300: Use the feature vector of the event to be determined to match the event feature vectors in each of the first event information and the second event information to obtain the first prompt rule set and the second prompt rule set. The first prompt rule set is a set of determination rules used in the top n historical events in the first event information that are ranked similar to the feature vector of the event to be determined. The second prompt rule set is a set of determination rules used in the top n historical events in the second event information that are ranked similar to the feature vector of the event to be determined.

[0072] The vector matching method in this step can be calculated using an existing vector similarity calculation method, such as cosine distance. In this embodiment, n is a positive integer greater than or equal to 1, such as 2 or 3 in this embodiment.

[0073] Specifically, in another embodiment of the present invention, S300 may also be implemented by the following steps, including:

[0074] S301: Obtain a list of rules corresponding to the determination rules used to determine the text in the first event information that ranks top n in similarity to the feature vector of the event to be determined.

[0075] S302: Determine the applicable identifier of each determination rule according to the rule ID corresponding to each determination rule. The applicable identifier is the number of times the same determination rule appears repeatedly.

[0076] S303: According to the execution time identifier corresponding to each of the repeated determination rules, only the determination rule with the latest execution time is maintained.

[0077] S304: Each remaining judgment rule is used as a rule in the first prompt rule set, wherein each judgment rule carries a corresponding effectiveness level identifier, an implementation time identifier, a special identifier, and an applicable identifier.

[0078] Generally, if the first n events with high similarity all use the same rule when judging, it means that this rule has high applicability to the current event to be judged and should be given special consideration. Therefore, in the present embodiment, the applicable identification of each judgment rule is determined by the number of repeated occurrences of the same judgment rule during the deduplication process, so as to provide reference for the large language model later. This can further improve the guiding ability of the rules in the prompt rule set for the large language model.

[0079] S400: If the user type identifier is a first type user identifier, the first prompt rule set is used as the first sub-prompt information, and the second prompt rule set is used as the second sub-prompt information.

[0080] S500: After the first sub-prompt information, the second sub-prompt information, the user's description information of the event to be determined, and the judgment texts of the cases that rank top n in similarity to the feature vector of the event to be determined in the first event information and the second event information are concatenated, the concatenation is input into the large language model.

[0081] Specifically, the large language model is a general large language model or a special large language model. The special large language model is obtained by retraining the general large language model using the corpus in the field of the event to be determined. If the field to which the event to be determined belongs is the legal field, the corresponding corpus may include texts such as judicial decisions, judicial interpretations, implementation details of the corresponding laws, operating standard guidance, and interpretation information on legal provisions on some public authoritative websites. Through further training of the corresponding professional corpus in the above-mentioned legal field, the existing general large language model can have a more accurate understanding and analysis capability of each text in the legal field, thereby improving the accuracy of the output answer.

[0082] S600: If the user type identifier is a second type user identifier, the first prompt rule set is used as the second sub-prompt information, and the second prompt rule set is used as the first sub-prompt information.

[0083] S700: After the first sub-prompt information, the second sub-prompt information, the user's description information of the event to be determined, and the judgment texts of the cases that rank top n in similarity to the feature vector of the event to be determined in the first event information and the second event information are concatenated, the concatenation is input into the large language model.

[0084] Since, in legal dispute cases, there may usually be applicable laws for both the plaintiff and the defendant, it is necessary to analyze the case from two directions before a more comprehensive and more meaningful answer can be generated. For example, in a violent conflict, there are both springs for intentional injury and springs for legitimate defense. Therefore, the prompt information of this embodiment contains high-similarity information from both the first historical event information library and the second historical event information library. Correspondingly, when generating the corresponding answer, the large language model can analyze from the perspectives of both the plaintiff and the defendant, and finally generate a more comprehensive answer.

[0085] In this embodiment, the content of the first sub-prompt information and the second sub-prompt information is adjusted in real time according to the type of the user, so that no matter whether the user is a plaintiff or a defendant, the corresponding first sub-prompt information is information that is favorable to the user, and the second sub-prompt information is information that is unfavorable to the user. Therefore, no matter when the large language model is answering the plaintiff or the defendant's inquiry, the content at the corresponding position in the prompt information constructed by the large language model has the same guiding intention, which makes it easier for the large language model to understand and analyze.

[0086] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0087] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0088] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0089] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0090] The electronic device according to this embodiment of the present invention is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0091] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one storage device mentioned above, and a bus connecting different system components (including storage devices and processors).

[0092] The storage stores program codes, which can be executed by the processor, so that the processor executes the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0093] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0094] The storage may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0095] The bus may represent one or more of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0096] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device (e.g., routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0097] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0098] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0099] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0101] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0102] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0103] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0104] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0105] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An intelligent question-answering system based on a large language model, characterized in that: include: A first historical event information database, a second historical event information database, an input prompt generation module and a large language model; the input prompt generation module is respectively connected to the first historical event information database, the second historical event information database and the large language model in communication; The first historical event information database includes a plurality of first event information; the first event information is information corresponding to historical events whose determination results are favorable to the first category of users; The second historical event information database includes a plurality of second event information, wherein the second event information is information corresponding to historical events whose judgment results are favorable to the second type of users; the first type of users and the second type of users are the opposing parties in the historical events; Each first event information or second event information includes a judgment text corresponding to the historical event, an event feature vector corresponding to the judgment text, and a rule list corresponding to the judgment rule used in the judgment text, wherein each judgment rule corresponds to a rule ID and an effectiveness level identifier, an implementation time identifier, and a special identifier; The input prompt generation module operates according to a first method, which includes: Acquire user input information; the input information includes user description information of the event to be determined and a user type identifier, wherein the user type identifier is a first-class user identifier or a second-class user identifier; Using a semantic acquisition model, converting the user's description of the event to be determined into a corresponding feature vector of the event to be determined; The feature vector of the event to be determined is used to match the event feature vectors in each of the first event information and the second event information, respectively, to obtain a first prompt rule set and a second prompt rule set; the first prompt rule set is a set of determination rules used in the top n historical events in the first event information that are ranked similar to the feature vector of the event to be determined; the second prompt rule set is a set of determination rules used in the top n historical events in the second event information that are ranked similar to the feature vector of the event to be determined; If the user type identifier is a first type user identifier, the first prompt rule set is used as the first sub-prompt information, and the second prompt rule set is used as the second sub-prompt information; The first sub-prompt information, the second sub-prompt information, the user's description information of the event to be determined, and the judgment texts of the top n cases in the first event information and the second event information that are similar to the feature vector of the event to be determined are concatenated and input into the large language model.

2. The intelligent question-answering system based on a large language model according to claim 1, characterized in that: After obtaining the first prompt rule set and the second prompt rule set, the first method further includes: If the user type identifier is a second type user identifier, the first prompt rule set is used as the second sub-prompt information, and the second prompt rule set is used as the first sub-prompt information; The first sub-prompt information, the second sub-prompt information, the user's description information of the event to be determined, and the judgment texts of the top n cases in the first event information and the second event information that are similar to the feature vector of the event to be determined are concatenated and input into the large language model.

3. The intelligent question-answering system based on a large language model according to claim 1, characterized in that: The event feature vector to be determined is used to match the event feature vector in each first event information and the second event information respectively to obtain a first prompt rule set and a second prompt rule set, including: Obtain a list of rules corresponding to the determination rules used in the determination text in the first event information that are ranked top n in similarity to the feature vector of the event to be determined; According to the rule ID corresponding to each determination rule, determine the applicable identifier of each determination rule; the applicable identifier is the number of repetitions of the same determination rule; According to the implementation time mark corresponding to each of the repeated determination rules, only the determination rule with the latest implementation time is guaranteed; Each of the remaining judgment rules is used as a rule in the first prompt rule set, wherein each judgment rule carries a corresponding effectiveness level identifier, an implementation time identifier, a special identifier, and an applicable identifier.

4. The intelligent question-answering system based on a large language model according to claim 1, characterized in that: The semantic acquisition model includes a Doc2Vec model.

5. The intelligent question-answering system based on a large language model according to claim 4, characterized in that: The semantic acquisition model is used to convert the user's description information of the event to be determined into a corresponding feature vector of the event to be determined, including: Split the text corresponding to the description information into words or phrases; Removing stop words from the text corresponding to the description information; Input the processed text into the Doc2Vec model to generate the corresponding feature vector of the event to be determined.

6. The intelligent question-answering system based on a large language model according to claim 5, characterized in that: The Doc2Vec model is obtained by retraining a pre-trained general Doc2Vec model using the corpus of the field to which the event to be determined belongs.

7. The intelligent question-answering system based on a large language model according to claim 6, characterized in that: The large language model is a general large language model or a special large language model; the special large language model is obtained by retraining the general large language model using the corpus in the field of the event to be determined.

8. An intelligent question-answering system based on a large language model according to claim 6 or 7, characterized in that: The field to which the event to be determined belongs includes the legal field.

9. The intelligent question-answering system based on a large language model according to claim 7, characterized in that: Get user input information, including: The user type identifier is determined based on the preset identity type label selected by the user.

10. The intelligent question-answering system based on a large language model according to claim 1, characterized in that: The first event information is obtained according to the following steps: Obtain the text information in the area corresponding to the judgment result in the judgment text corresponding to the historical event; A preset regular expression is used to match and identify text information in the area corresponding to the determination result to determine whether the information corresponding to the historical event is the first event information.

Images