Natural language processing method and device

Abstract

The invention provides a natural language processing method and device. The method comprises the following steps of: reasoning description information according to an original cue word by utilizing a large model to obtain an original answer; verifying the original answer according to the judgment prompt word; and if the original answer does not pass the verification, reconstructing the original answer according to a reconstruction prompt word by using the large model to obtain a target answer.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and particularly to natural language processing methods and devices. Background Art

[0002] A large model refers to a machine learning model with a huge parameter scale and complexity. During the application process of a large model, prompt words can be constructed to make the model complete specified tasks. As the application of large models becomes more and more extensive, the application scenarios tend to be complex. In complex scenarios, it often means that the complexity of the prompt words increases, and the demand for the model's capabilities also rises accordingly. However, the cycle of improving the model's capabilities is relatively long. In actual applications, it often occurs that the demand for the model's capabilities in business scenarios reaches or exceeds the upper limit of the model in this field, thereby causing the quality of the model's answers to decline, the accuracy to decrease, and it is difficult to meet the requirements of business scenarios. Summary of the Invention

[0003] To overcome the problems existing in the related art, this specification provides natural language processing methods and devices.

[0004] According to the first aspect of the embodiments of this specification, a natural language processing method is provided. The method includes: using a large model to reason about description information according to an original prompt word to obtain an original answer; verifying the original answer according to a judgment prompt word; if the original answer fails the verification, using the large model to reconstruct the original answer according to a reconstruction prompt word to obtain a target answer.

[0005] According to the second aspect of the embodiments of this specification, a natural language processing device is provided. The device includes: an inference module for using a large model to reason about description information according to an original prompt word to obtain an original answer; a verification module for verifying the original answer according to a judgment prompt word; a reconstruction module for, if the original answer fails the verification, using the large model to reconstruct the original answer according to a reconstruction prompt word to obtain a target answer.

[0006] According to the third aspect of the embodiments of this specification, a natural language processing device is provided, including:

[0007] A processor;

[0008] A memory for storing instructions executable by the processor;

[0009] Wherein, the processor is configured to: use a large model to reason about description information according to an original prompt word to obtain an original answer; verify the original answer according to a judgment prompt word; if the original answer fails the verification, use the large model to reconstruct the original answer according to a reconstruction prompt word to obtain a target answer.

[0010] The technical solutions provided by the embodiments of this specification may include the following beneficial effects:

[0011] In the embodiments of this specification, by verifying the content output by the large model and reconstructing the incorrect content, a result that meets the requirements is finally output to the front end. Thereby, the quality and accuracy of model question answering can be improved. In addition, the large model can be enabled to complete some complex tasks that exceed the scope of the model's capabilities, meeting the requirements of more business scenarios.

[0012] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments that conform to this specification, and are used together with the specification to explain the principles of this specification.

[0014] Figure 1 is an example of the system architecture shown according to an exemplary embodiment of this specification.

[0015] Figure 2 is a flowchart of a natural language processing method shown according to an exemplary embodiment of this specification.

[0016] Figure 3 is a flowchart of a natural language processing method shown according to another exemplary embodiment of this specification.

[0017] Figure 4 is a hardware structure diagram of the computer device where the natural language processing device in the embodiments of this specification is located.

[0018] Figure 5 is a block diagram of a natural language processing device shown according to an exemplary embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods that are consistent with some aspects of this specification as detailed in the appended claims.

[0020] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0022] Next, the embodiments of this specification will be described in detail.

[0023] The following Figure 1 describes the system architecture to which the natural language processing method and apparatus according to the embodiments of this specification can be applied. It should be noted that Figure 1 only the examples of the system architecture to which the embodiments of this specification can be applied are shown to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments or scenarios.

[0024] Figure 1 is a schematic diagram of the system architecture shown according to an exemplary embodiment of this specification.

[0025] As Figure 1 shown, the system architecture may include, for example, a terminal device, a network, and a server. The network is a medium for providing a communication link between the terminal device and the server. The network may include various connection types, such as wired and / or wireless communication links, etc.

[0026] Users can use the terminal device to interact with the server through the network to receive or send messages, etc. Various communication client applications may be installed on the terminal device, such as a service access client, a web browser application, a search application, an instant messaging tool, an email client, and / or a social platform software, etc.

[0027] The terminal device may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0028] The server can be a server that provides various services, such as a background management server that supports the content browsed by users using terminal devices. The background management server can analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0029] Next, a natural language processing method provided by an embodiment of this specification will be described in detail. As Figure 2 shown, Figure 2 FIG. is a flowchart of a natural language processing method shown according to an exemplary embodiment of this specification. This method can be applied to a server, for example. The natural language processing method provided by the embodiment of this specification may include the following steps.

[0030] In step 210, using a large model, reason about the description information according to the original prompt word to obtain the original answer.

[0031] According to an embodiment of this specification, a user can input description information through a terminal device and then send it to the server. The server can input the original prompt word and the description information into the large model, and the large model outputs the original answer. The large model can be deployed locally on the server, or the large model can also be deployed in other servers. This server can call the large model in other servers through a preset inference interface to use the inference function of the large model.

[0032] According to an embodiment of this specification, the original prompt word can be the prompt word used in the original business implementation logic.

[0033] In step 220, verify the original answer according to the judgment prompt word.

[0034] According to an embodiment of this specification, the judgment prompt word can include a predetermined judgment criterion required for verifying the original answer.

[0035] For example, the original answer and the judgment prompt word can be input into the large model to use the large model to verify the original answer according to the judgment prompt word.

[0036] Optionally, in addition to the large model, other models other than the large model can also be used to perform the verification of the original answer. Or, an automated script can be used to replace the model to perform the verification of the original answer.

[0037] In step 230, if the original answer fails the verification, then use the large model to reconstruct the original answer according to the reconstruction prompt word to obtain the target answer.

[0038] According to an embodiment of this specification, the judgment prompt word can be used to instruct the large model to reconstruct the original answer that does not meet the predetermined judgment criterion.

[0039] According to the embodiments of this specification, if the original answer passes the verification, the original answer is output. If the original answer fails the verification, the large model is used to reconstruct the original answer according to the reconstruction prompt words to obtain the target answer, and then the target answer is output. It can be understood that before outputting the answer, preprocessing can also be performed on the answer, and the preprocessing can include, for example, data cleaning, format conversion, etc.

[0040] According to the embodiments of this specification, by verifying the content output by the large model and reconstructing the incorrect content therein, a result that meets the requirements is finally output to the front end. Thereby, the quality and accuracy of model question answering can be improved. In addition, the large model can be enabled to complete some complex tasks that exceed the scope of the model's capabilities and meet the requirements of more business scenarios.

[0041] For a large model, if the original prompt words contain too many instructions or steps, exceeding the model's understanding or processing capabilities, it will be difficult for the large model to effectively complete the tasks in the original prompt words. In the judgment prompt words, the large model is only required to judge a small number or even a single task objective or indicator in the original task requirements. This step is greatly simplified compared to the original task, and the requirements for the large model's capabilities are also greatly reduced. Similarly, in the reconstruction step, the large model is also only required to reconstruct the non-compliant items in the original output message, and the difficulty of this step for the large model is also greatly reduced compared to the original task. Therefore, the large model can be made to complete some things that originally exceeded its own capabilities.

[0042] Optionally, using the large model to reconstruct the original answer according to the reconstruction prompt words to obtain the target answer may include:

[0043] According to the verification result, determine the reason why the original answer fails the verification. Embed the original answer and the reason why the original answer fails the verification into the reconstruction prompt words to obtain the intermediate prompt words. Use the large model to reason about the intermediate prompt words to obtain the target answer. There are substitution symbols preset in the reconstruction prompt words.

[0044] Optionally, the judgment prompt words may include at least one judgment criterion. Using the large model to verify the original answer according to the judgment prompt words may include:

[0045] Use the large model to verify the original answer based on the judgment criteria in the judgment prompt words to obtain the verification result. The verification result may include the judgment criteria that the original answer conforms to and / or the judgment criteria that the original answer does not conform to. If the original answer conforms to all judgment criteria, it is determined that the original answer passes the verification; otherwise, it is determined that the original answer fails the verification.

[0046] Based on this, the judgment criteria for which the original answer in the verification result does not conform can be determined as the reason why the original answer fails the verification.

[0047] Optionally, for example, the requirement information and key metrics can be extracted from the original prompt. Then, based on the requirement information and key metrics, the evaluation criteria in the judgment prompt are determined.

[0048] According to the embodiments of this specification, for example, the specific requirements and quantifiable key metrics that the user is concerned about can be analyzed from the original prompt. And based on this, a "judgment criterion" that is simpler than the original task is formulated.

[0049] According to the embodiments of this specification, for example, the specific requirements and key metrics that the user is concerned about can be converted into specific judgment criteria. Based on the judgment criteria, a "message evaluation prompt" is generated.

[0050] According to the embodiments of this specification, for example, a series of prompts for reconstructing messages can be prefabricated, and one or more can be prefabricated according to the actual task requirements.

[0051] According to the embodiments of this specification, after the server receives the message generated by the original prompt, it does not directly output it to the front end, but embeds it into the "reconstruction prompt" in the background and calls the inference interface of the large model again.

[0052] Let the model evaluate the key metric items in the original message and return them in the specified message format.

[0053] According to the embodiments of this specification, after the server receives the evaluation message, it can determine whether the original message needs to be reconstructed based on the evaluation result in the evaluation message. If no reconstruction is required, the original message is directly output to the front end. If reconstruction is required, the original message is embedded in the "reconstruction prompt" and the next step is executed: message reconstruction.

[0054] In the case where the original message needs to be reconstructed, the server can determine the reason for the reconstruction through the evaluation result (for example, which specific indicator or requirement is not met). Therefore, the original message and the reconstruction reason can be embedded in the "message reconstruction prompt", and the model inference interface is requested again to reconstruct the original message and obtain the "reconstructed message". At this time, since the model has been informed of the specific direction that needs to be adjusted and the instruction is much less than the original prompt, the quality of the "reconstructed message" returned by the message reconstruction request is often able to be greatly improved compared to the quality of the original message.

[0055] Output the message to the front end, and the user side gets the optimized message.

[0056] Such as Figure 3As shown, it is a flowchart of another natural language processing method shown according to an exemplary embodiment. Based on the foregoing embodiment, this embodiment describes a processing process of natural language processing. Exemplarily, in this embodiment, the original business requirement is to extract the requirements mentioned by the user from the description information input by the user in a shopping scenario and return them as a JSON message body in a specified format. Exemplarily, in this embodiment, the JSON message body can be, for example, json_example shown below.

[0057] json_example = {

[0058] "Brand": "The brand that the user mentions they want. Note that if the user mentions multiple brands, they need to be listed separately. If not mentioned, write 'unlimited'.",

[0059] "Type": "The vehicle type mentioned in the user's description. Common types include sedan, SUV, MPV, sports car, light bus, truck, pickup, etc.",

[0060] "Country of origin": "Extract the country of production that the user wants from the user's description. Select from the following list: domestic, joint venture, United States, Japan, Germany, United Kingdom, France, Italy, South Korea, unlimited",

[0061] "Price": "Extract the price included in the user's description conditions. It can be a specified value or a range interval. If not mentioned, write 'unlimited'.",

[0062] "Power mode": "Extract the power mode mentioned in the user's description conditions. Common power modes include gasoline, diesel, pure electric, hybrid, range extender, etc.",

[0063] "Displacement": "Extract the displacement description included in the user's description conditions. It is usually a value or a range interval. Please accurately extract the value in the description. If not mentioned, write 'unlimited'.",

[0064] "Transmission type": "Extract the transmission type included in the user's description conditions. The transmission type should be selected from the following list: manual, automatic, dry dual-clutch, wet dual-clutch, continuously variable transmission (CVT), CVT, automatic-manual, AMT, etc. If not mentioned, write 'unlimited'.",

[0065] "Number of seats": "Extract the description of the number of seats in the user's description. If an interval is described, the numbers within the interval need to be listed separately. If not mentioned, write 'unlimited'.",

[0066] "Sunroof type": "Extract the description of the sunroof in the user's description. Select from the following list: no sunroof, panoramic sunroof, single sunroof, double sunroof, unlimited",

[0067] "Tire Brand": "Extract the description of the required tire brand from the user's description and select from the following list: Michelin, Bridgestone, Hankook, Continental, Pirelli, Goodyear, BFGoodrich; if a brand is mentioned but not in the list, write 'other', if no specific tire brand is mentioned, write 'unlimited'.",

[0068] "Zero to Hundred Acceleration": "Extract the description of the zero to one hundred kilometer acceleration time of the user's car from the user's description and return it in numerical format, write 'unlimited' if not mentioned.",

[0069] "Wheelbase": "Extract the user's requirement for the wheelbase from the user's description, write 'unlimited' if not mentioned. Note that the output wheelbase should be converted to millimeters.",

[0070] }

[0071] Exemplarily, in this embodiment, the original prompt can be the prompt_0 shown below.

[0072] prompt_0 = "You are an automotive sales assistant. The user wants to view our cars and provides description information about the cars. Please extract the user's specific query conditions from it and output them as a JSON message body in the following specified format.\nThe JSON message body format and description you need to output are as follows:\n{%JSON - EXAMPLE%}\nNote that you must output strictly according to the fields in the specified JSON message body without missing any of them. If a relevant condition is not mentioned in the user's description, write 'unlimited' for all of them. If the user's description contains fields not mentioned in the above JSON message body, do not add these fields to the JSON message body. Make sure the fields in the output JSON message body are consistent with the required format. The user's description information is as follows: %USER_INPUT%"

[0073] Among them, {%JSON - EXAMPLE%} is a placeholder for json_example, and %USER_INPUT% is a placeholder for USER_INPUT.

[0074] Exemplarily, in this embodiment, the description information input by the user can be the USER_INPUT shown below.

[0075] USER_INPUT = "My budget is no more than 350,000. There is no requirement for displacement. I want to look at automatic models. The brands I consider include BaoX, BenX, and AoX. It is best to have a panoramic sunroof. It should be a 6-seater or 7-seater. The body color is considered blue or white. The wheels should be large, preferably with Michelin or Bridgestone tires. The zero-to-hundred acceleration should not exceed 9.4 seconds. The wheelbase should be long, not less than 2,800 mm. It needs one-touch start and automatic parking functions. The styling should be eye-catching and the interior should be cool. Sedans and light buses are not considered. CVT or dry dual-clutch transmissions are not acceptable."

[0076] According to the embodiments of this specification, after the large model infers the description information according to the original prompt, the generated answer may be the correct result or the wrong result.

[0077] For example, the large model can generate Answer 1:

[0078]

[0079] This Answer 1 is the correct result.

[0080] Again, for example, the large model can generate Answer 2:

[0081]

[0082]

[0083] This Answer 2 is the wrong result.

[0084] Thus, it can be seen that the large model will probabilistically return wrong answers.

[0085] The main reason for the wrong answer is that the model does not fully execute the constraint conditions in the instruction. The last constraint condition "If the user description contains fields not mentioned in the above JSON message body, do not add these fields to the JSON message body, and make sure that the fields in the output JSON message body are in the required format." is probabilistically ignored, and it is very difficult for the model to stably complete all instructions.

[0086] Based on this, in this embodiment, the evaluation criteria for the original message can be clarified first. In the above example, the cases where the output is wrong are all manifested as the fields in the output JSON message body being inconsistent with the expectations. Therefore, the evaluation criteria are set to check the fields in the output message body. Therefore, the model can be allowed to judge whether the fields in the output original message are consistent with the following list:

[0087] json_0 = ["brand", "type", "country", "price", "power mode", "displacement", "transmission type", "number of seats", "sunroof type", "tire brand", "zero-to-hundred acceleration", "wheelbase"]

[0088] Based on the above evaluation criteria, the following prompt prompt_1 can be constructed to verify the original result:

[0089] prompt_1 = f"You are an automotive sales assistant. Your colleague has organized a user's car selection preferences and compiled them into a JSON message body %MESSAGE_0%.\nPlease extract all the fields in the above JSON message in sequence and add them to a list A, then check if the elements contained in list A are the same as those in list B. List B is as follows: {json_0}. If they are not the same, add the names of the different fields to list differs, and then output in the following specified JSON format.\nThe specified output format is as follows: {json_1}\nNote that only the JSON message body in the specified format needs to be output, without any additional explanations or descriptions. Only compare the field names, regardless of the field values."

[0090] In the above prompt, the specified output format can include, for example:

[0091] json_1 = {"List A": "The elements contained in list A, output in list format.",

[0092] "List B": "The complete output of list B, note not to miss any element.",

[0093] "differs": "The specific fields with differences, output in list format, note not to miss any."}

[0094] In addition, the number of different items can also be output.

[0095] For the above answer 1, based on the verification of prompt_1, the following verification result can be obtained:

[0096]

[0097]

[0098] Number of different items: 0

[0099] For the above answer 2, based on the verification of prompt_1, the following verification result can be obtained:

[0100]

[0101] Number of different items: 5

[0102] According to the embodiments of this specification, verification is performed according to the verification prompt words, and the output accuracy of the model is greatly improved according to the output accuracy in the prompt words. This is because compared with the original prompt words, the number of instructions and the task complexity in the verification prompt words are both greatly reduced, and the model can ensure a high accuracy when executing the instructions in the prompt words.

[0103] If the original answer passes the verification, it means that the original answer is the correct answer. The correct answer can be preprocessed to generate a conversation message and then the conversation message is output. Among them, the preprocessing can include, for example, data cleaning, format conversion, etc.

[0104] Exemplarily, a reconstruction prompt word can be set in this embodiment, and the wrong answer can be reconstructed according to the reconstruction prompt word.

[0105] The reconstruction prompt word can be, for example, prompt_2.

[0106] prompt_2 = "You are a car sales assistant. Your colleague has sorted out a user's car selection preferences and organized them into a JSON message body %MESSAGE_0%.\nBut he made some mistakes. The following items in the output JSON message body are redundant: %DIFFLIST%. Please remove the redundant items and output the correct JSON message body. Note that only the corrected JSON message body needs to be output, without any explanation."

[0107] For Answer 1, after verification, the verification result is that the output is correct and no error correction is required.

[0108] For Answer 2, after verification, the verification result is that the output is wrong. Answer 2 can be reconstructed based on prompt_2. For example, Answer 2 can be embedded at the %MESSAGE_0% in prompt_2, and "differs" in the verification result can be embedded at the %DIFFLIST% in prompt_2, and then input into the large model to obtain the following reconstructed answer:

[0109]

[0110] Next, the reconstructed answer can be preprocessed to generate a conversation message and then the conversation message is output. Among them, the preprocessing can include, for example, data cleaning, format conversion, etc.

[0111] According to the embodiments of this specification, without changing the model, by using the evaluation prompt word and the reconstruction prompt word, the model can self-check for output errors and achieve automatic error correction of the output result. Through automatic error correction, the accuracy of the model's answer can be greatly improved without changing the model.

[0112] Corresponding to the embodiments of the foregoing method, this specification also provides embodiments of a natural language processing device and a terminal to which it is applied.

[0113] The embodiments of the natural language processing device in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the corresponding computer program instructions in the non-volatile memory being read into the memory and run by the processor where it is located. From a hardware level, as Figure 4 shown, it is a hardware structure diagram of the computer device where the natural language processing device in the embodiments of this specification is located. In addition to Figure 4 the shown processor 410, memory 430, network interface 420, and non-volatile memory 440, the server or electronic device where the device 431 is located in the embodiments usually includes other hardware according to the actual functions of the computer device, which will not be elaborated here.

[0114] As Figure 5 shown, Figure 5 is a block diagram of a natural language processing device shown according to an exemplary embodiment of this specification. The device includes:

[0115] An inference module 510, configured to use a large model to infer description information according to an original prompt word to obtain an original answer;

[0116] A verification module 520, configured to verify the original answer according to a judgment prompt word;

[0117] A reconstruction module 530, configured to, if the original answer fails the verification, use the large model to reconstruct the original answer according to a reconstruction prompt word to obtain a target answer.

[0118] Optionally, the reconstruction module may include:

[0119] A reason determination sub-module, configured to determine the reason why the original answer fails the verification according to the verification result;

[0120] An embedding sub-module, configured to embed the original answer and the reason why the original answer fails the verification into the reconstruction prompt word to obtain an intermediate prompt word;

[0121] An inference sub-module, configured to use the large model to infer the intermediate prompt word to obtain a target answer.

[0122] Optionally, the judgment prompt word may include at least one judgment criterion. The verification module may include:

[0123] The parity check sub-module is used to utilize a large model to check the original answer based on the evaluation criteria in the judgment prompt word, and obtain a check result, where the check result includes the evaluation criteria that the original answer conforms to and / or the evaluation criteria that the original answer does not conform to;

[0124] The check result determination sub-module is used to determine that the original answer passes the check if the original answer conforms to all the evaluation criteria, otherwise, determine that the original answer fails the check.

[0125] Optionally, the reason determination sub-module may include:

[0126] The determination unit is used to determine the evaluation criteria that the original answer does not conform to in the check result as the reason for the original answer failing the check.

[0127] Optionally, the above device may further include:

[0128] The extraction module is used to extract requirement information and key indicators from the original prompt word;

[0129] The evaluation criterion determination module is used to determine the evaluation criteria in the judgment prompt word according to the requirement information and key indicators.

[0130] According to the embodiments of the present specification, by checking the content output by the large model and reconstructing the incorrect content therein, a result that meets the requirements is finally output to the front end. Thereby, the quality and accuracy of the model's question and answer can be improved. In addition, it can enable the large model to complete some complex tasks that exceed the scope of the model's capabilities and meet the requirements of more business scenarios.

[0131] Correspondingly, the present specification also provides a natural language processing device, which includes a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to: utilize a large model to reason about the description information according to the original prompt word to obtain an original answer; check the original answer according to the judgment prompt word; if the original answer fails the check, then utilize the large model to reconstruct the original answer according to the reconstruction prompt word to obtain a target answer.

[0132] The implementation processes of the functions and roles of each module in the above device are specifically detailed in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0133] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0134] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] Those skilled in the art will readily conceive of other embodiments of this specification after considering the specification and practicing the invention herein. This specification is intended to cover any variations, uses, or adaptations of this specification, which follow the general principles of this specification and include common general knowledge or conventional technical means in the technical field not claimed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this specification are pointed out by the following claims.

[0136] It should be understood that this specification is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of this specification is only limited by the appended claims.

[0137] The above are only the preferred embodiments of this specification and are not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

Claims

1. A natural language processing method, characterized in that: The method comprises: Using the large model, the descriptive information is inferred based on the original prompt words to obtain the original answer; Verifying the original answer according to the evaluation prompt words; If the original answer fails the verification, the large model is used to reconstruct the original answer according to the reconstruction prompt words to obtain the target answer.

2. The method according to claim 1, characterized in that The method of using the large model to reconstruct the original answer according to the reconstruction prompt word to obtain the target answer includes: Determine, based on the verification result, why the original answer failed the verification; The original answer and the reason why the original answer failed the verification are embedded in the reconstructed prompt word to obtain an intermediate prompt word; The large model is used to infer the intermediate prompt word to obtain a target answer.

3. The method according to claim 2, characterized in that The evaluation prompt word includes at least one evaluation criterion; The original answer is verified according to the evaluation prompt words, including: Using the large model, based on the evaluation criteria in the evaluation prompt words, the original answer is verified to obtain a verification result, wherein the verification result includes the evaluation criteria that the original answer meets and / or the evaluation criteria that it does not meet; If the original answer meets all the evaluation criteria, it is determined that the original answer has passed the verification; otherwise, it is determined that the original answer has failed the verification.

4. The method according to claim 3, characterized in that Determining, based on the verification result, why the original answer failed the verification includes: Determine the evaluation criteria that the original answer in the verification result does not meet as the reason why the original answer fails the verification.

5. The method according to claim 3, characterized in that: The method further comprises: Extract demand information and key indicators from the original prompt words; The evaluation criteria in the evaluation prompt words are determined according to the demand information and key indicators.

6. A natural language processing device, characterized in that: The device comprises: The reasoning module is used to use the large model to infer the description information based on the original prompt words to obtain the original answer; A verification module, used for verifying the original answer according to the evaluation prompt word; The reconstruction module is used to reconstruct the original answer using the large model according to the reconstruction prompt words to obtain the target answer if the original answer fails the verification.

7. The device according to claim 6, characterized in that The reconstruction module includes: A reason determination submodule, used to determine the reason why the original answer failed the verification according to the verification result; An embedding submodule, used for embedding the original answer and the reason why the original answer fails the verification into the reconstructed prompt word to obtain an intermediate prompt word; The reasoning submodule is used to use the large model to reason about the intermediate prompt word to obtain a target answer.

8. The device according to claim 7, characterized in that The evaluation prompt word includes at least one evaluation criterion; the verification module includes: A verification submodule, for verifying the original answer using the large model based on the evaluation criteria in the evaluation prompt words, and obtaining a verification result, wherein the verification result includes the evaluation criteria that the original answer meets and / or does not meet; The verification result determination submodule is used to determine that the original answer has passed the verification if the original answer meets all the evaluation criteria, and otherwise, determine that the original answer has failed the verification.

9. The device according to claim 8, characterized in that The cause determination submodule includes: A determination unit is used to determine the evaluation criteria that the original answer in the verification result does not meet, as the reason why the original answer fails the verification.

10. The device according to claim 8, characterized in that The device also includes: Extraction module, used to extract demand information and key indicators from the original prompt words; The evaluation criteria determination module is used to determine the evaluation criteria in the evaluation prompt words according to the demand information and key indicators.

Images