Data processing system and data processing method
By combining data processing systems and methods of large language models and natural language understanding models, the problem of poor generalization capabilities of traditional task-based assistant natural language processing process frameworks is solved, and higher controllability and reduced data dependence are achieved.
Patent Information
- Application Number
- CN202411977162.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-25
AI Technical Summary
The traditional task assistant has poor generalization capabilities in natural language processing process frameworks, poor controllability within the model, and has too high dependence on data quality and data volume.
A data processing system and method combining large language model (LLM) and natural language understanding model (NLP) is adopted to receive input data through the processor, generate complete request data, and perform data processing between large language model and natural language understanding model to generate intent data and task boundary judgment results, and finally display the reply task result data through the user interface.
It realizes that user input data is processed through large language models and natural language understanding models, and generates corresponding reply task result data, which improves the generalization ability and controllability of task-based assistants, and reduces dependence on data quality and data volume.
Smart Images

Figure CN119886350A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a system, in particular to a data processing system and a data processing method. Background Art
[0002] The natural language processing process framework of traditional task-based assistants mostly uses pipeline serial processing to perform data processing processes such as intent recognition, entity recognition, dialogue management, task execution, and response generation. In this regard, the natural language processing process framework of traditional task-based assistants has the problem of poor generalization ability. At present, although there are implementation solutions that use large language models (LLM) for data processing, there are still problems such as poor controllability within the model and excessive dependence on data quality and data volume. Summary of the invention
[0003] The present invention is directed to a data processing system and a data processing method, which can realize a system framework of a task-based assistant combining a large language model (LLM) and a natural language understanding model (Natural Language Processing, NLP).
[0004] According to an embodiment of the present invention, the data processing system of the present invention includes a storage device and a processor. The storage device is used to store multiple models. The processor is coupled to the storage device and is used to execute multiple models. The processor receives input data, and the processor generates complete request data based on the input data. The processor inputs the complete request data into a large language model to generate standardized complete request data. The processor inputs the standardized complete request data into a natural language understanding model to generate intent data. The processor performs task boundary judgment based on the standardized complete request data and the intent data to generate application interface task data, application interface parameter data and result data. The processor generates reply task result data based on the input data, the standardized complete request data, the application interface task data, the application interface parameter data and the result data. The processor displays the reply task result data through a user interface.
[0005] According to an embodiment of the present invention, the data processing method of the present invention includes the following steps: receiving input data, and generating complete request data based on the input data; inputting the complete request data into a large language model to generate standardized complete request data; inputting the standardized complete request data into a natural language understanding model to generate intent data; performing task boundary judgment based on the standardized complete request data and the intent data to generate application interface task data, application interface parameter data and result data; generating reply task result data based on the input data, the standardized complete request data, the application interface task data, the application interface parameter data and the result data; and displaying the reply task result data through a user interface.
[0006] Based on the above, the data processing system and data processing method of the present invention can process the input data input by the user through a large language model and a natural language understanding model, and generate corresponding reply task result data.
[0007] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is a schematic diagram of a data processing system according to an embodiment of the present invention;
[0009] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;
[0010] Figure 3 is a schematic diagram of a data processing framework for generating standardized complete request data according to an embodiment of the present invention;
[0011] Figure 4 is a schematic diagram of a data processing framework for generating intent data according to an embodiment of the present invention;
[0012] Figure 5 is a schematic diagram of a data processing framework for generating result data according to an embodiment of the present invention;
[0013] Figure 6 It is a schematic diagram of a data processing framework for displaying reply task result data according to an embodiment of the present invention.
[0014] Description of Reference Numerals
[0015] 100: Data processing system;
[0016] 110: processor;
[0017] 120: storage device;
[0018] 301: Input data;
[0019] 302: Input data in text format;
[0020] 303: Complete request data;
[0021] 304: Synonym data;
[0022] 305, 401, 501: standardized complete request data;
[0023] 402, 502: intent data;
[0024] 503: task data;
[0025] 504: parameter data;
[0026] 601: Reply task result data;
[0027] DB1: context history database;
[0028] DB2: synonym database;
[0029] DB3: rule database;
[0030] DB4: User feedback synonym database;
[0031] DB5: user feedback rules database;
[0032] DB6: A large database of language model intent recognition prompt templates;
[0033] DB7: Database of difficult corpus;
[0034] DB8: task training corpus database;
[0035] DB9: user feedback corpus database;
[0036] DB10: task test corpus database;
[0037] DB11: Application Programming Interface Knowledge Base;
[0038] DB12: A large language model construction API prompt template database;
[0039] DB13: Database of example puzzle sets;
[0040] DB14: Application program interface construction template database;
[0041] DB15: User feedback sample database;
[0042] DB16: task test sample database;
[0043] DB17: wrong question collection database;
[0044] S210~S260, S310~S314, S320~S321, S411~S413, S420~S428, S511~S519, S520~S527, S611~S614, S621: steps. DETAILED DESCRIPTION
[0045] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.
[0046] Figure 1 Schematic diagram of a data processing system according to an embodiment of the present invention. Figure 1 The data processing system 100 includes a processor 110 and a storage device 120. The processor 110 is coupled to the storage device 120 and the display device 130. In the present embodiment, the processor 110 may include a central processing unit (CPU), a microprocessor (MCU) or a field programmable gate array (FPGA) or other processing circuits or chips with data computing functions, but the present invention is not limited thereto. The storage device 120 may be a memory, wherein the memory may be, for example, a non-volatile memory such as a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a volatile memory such as a random access memory (RAM), a hard disc drive, a semiconductor memory or other storage device, and is used to store various programs and information mentioned in the present invention. In the present embodiment, the storage device 120 may store a plurality of specific modules, algorithms and / or software, etc., for the processor 110 to read and execute them respectively. It is noteworthy that the modules and units described in each embodiment of the present invention may be individually implemented by one or more corresponding algorithms and / or software, and the related functions and operations described in the embodiments may be implemented according to the execution results of one or more algorithms and / or software.
[0047] In one embodiment, the data processing system 100 may be set in a cloud server or a local application scenario. The data processing system 100 may also include an input device, such as a keyboard, a mouse, a microphone, etc., but the present invention is not limited thereto. In this embodiment, the input device may receive input data input by a user, wherein the input data may be, for example, text data or voice data. The data processing system 100 may provide highly generalized applications for the training process and the operation process of the data processing system 100 through a large language model, and may disassemble transaction instructions through a natural language understanding model, so that the data processing system 100 may achieve a highly controllable effect.
[0048] In one embodiment, the data processing system 100 may be set up in a cloud server, and the user may connect to the data processing system 100 through a terminal device or related terminal equipment to provide input data to the data processing system 100 for data processing, and enable the data processing system 100 to generate corresponding reply task result data and transmit it to the user's terminal device or related terminal equipment.
[0049] In one embodiment, the data processing system 100 may further include a display device, and the processor may display a user interface (UI) through the display device. When the processor 110 generates corresponding reply task result data according to the input data, the data processing system 100 may display the reply task result data through the user interface.
[0050] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 and Figure 2, the data processing system 100 may perform the following steps S210 to S260 to implement the application of the task-based assistant. In step S210, the processor 110 may receive input data and generate complete request data according to the input data. In step S220, the processor 110 may input the complete request data into a large language model to generate standardized complete request data. In step S230, the processor 110 may input the standardized complete request data into a natural language understanding model to generate intent data. In step S240, the processor 110 may perform task boundary judgment based on the standardized complete request data and the intent data to generate application interface task data, application interface parameter data and result data. In step S250, the processor 110 may generate reply task result data based on the input data, the standardized complete request data input, the application interface task data, the application interface parameter data and the result data. In step S260, the processor 110 may display the reply task result data through a user interface. Therefore, the data processing system 100 of this embodiment can implement the function of a task-based assistant through a large language model and a natural language understanding model. In addition, the detailed implementation details of steps S210 to S260 will be described below. Figures 3 to 6 The process framework of the embodiment is used for specific description.
[0051] Figure 3 Schematic diagram of a data processing framework for generating standardized complete request data according to an embodiment of the present invention. Figure 1 and Figure 3 , the data processing system 100 may include a context history database DB1, a synonym database DB2, a rule database DB3, a user feedback synonym database DB4, and a user feedback rule database DB5. Alternatively, the data processing system 100 may access the above-mentioned databases stored in other cloud servers or storage devices through a network connection. The data processing system 100 may execute steps S320 and S321 of the following system design process and steps S310 to S314 of the system operation process.
[0052] In the system design process, at step S320, the processor 110 may receive a user's operation instruction to design synonyms according to the user feedback synonym database DB4 to build the synonym database DB2. At step S320, the processor 110 may receive a user's operation instruction to design synonym replacement rules according to the user feedback rule database DB5.
[0053] In the system operation process, the processor 110 may receive input data 301 input by the user, wherein the input data 301 may be, for example, input data in a language format, but the present invention is not limited thereto. In one embodiment, the processor 110 may also directly receive input data in a text format input by the user. In step S310, the processor 110 may convert the input data 301 to generate input data 302 in a text format. In this embodiment, the processor 110 may input the input data 301 in a voice format into a voice-to-text model to convert it into input data 302 in a text format. In step S311, the processor 110 may search for corresponding context data through the context history database DB1 according to the input data 302 in a text format. In this embodiment, the processor 110 may search for corresponding context data according to the input data 302 in a text format to generate complete request data 303. In step S312, the processor 110 may rewrite the complete request data 303 according to the corresponding related word data based on a preset replacement rule. In step 313, the processor 110 may first search for synonyms according to the complete request data 303 to obtain the synonym data 304. In this embodiment, the processor 110 may read the rule database DB3 to obtain the corresponding preset replacement rule to rewrite the complete request data 303 according to the corresponding related word data (i.e., the synonym data 304).
[0054] For example, a synonym of "sales champion" may be "(noun) sales champion". The corpus may be, for example, "this year (time noun) / of (particle) / sales champion (noun) / is (verb) / who (pronoun)" (i.e., who is the sales champion this year). In this regard, the processor 110 may perform word segmentation and synonym replacement on the corpus, wherein, for example, the processor 110 may replace "sales champion (noun)" in the corpus with the synonym "(noun) sales champion" according to a preset rule.
[0055] In step S314, the processor 110 may confirm the rewriting result through the large language model. In this embodiment, the processor 110 may, for example, confirm through the large language model whether the result of the above-mentioned corpus after word segmentation and rule replacement can be recognized by the large language model and matches the recognition result of the complete request data 303. If not, the processor 110 may re-perform word segmentation and rule replacement. If so, the processor 110 may generate standardized complete request data 305 according to the result after word segmentation and rule replacement.
[0056] Figure 4 Schematic diagram of a data processing framework for generating intent data according to an embodiment of the present invention. Figure 1 and Figure 4, the data processing system 100 may include a natural language understanding model MD1, a large language model intention recognition prompt template database DB6, a problem corpus database DB7, a task training corpus database DB8, a user feedback corpus database DB9, and a task test corpus database DB10. Alternatively, the data processing system 100 may access the above-mentioned databases stored in other cloud servers or storage devices through a network connection. The data processing system 100 may execute steps S420 to S428 of the following system design process and steps S411 to S413 of the system operation process.
[0057] In the system design process, in step S420, the processor 110 can read the task training corpus database according to the operation instructions input by the user to train the natural language understanding model MD1. In step S421, the processor 110 can test the natural language understanding model MD1. In step S422, the processor 110 can confirm the accuracy of the natural language understanding model MD1 based on the verification data to determine whether the natural language understanding model MD1 passes the test. If so, the construction of the natural language understanding model MD1 is completed. If not, in step S423, the processor 110 can expand the corpus with a large language model, wherein the large language model can read the relevant data of the user feedback corpus database DB9 and the task test corpus database DB10 to expand the corpus. In step S424, the processor 110 can screen and confirm the corpus to effectively expand the task training corpus database DB8.
[0058] Before using the large language model, in step S425, the processor 110 may abstractly define the corpus through the large language model. In step S426, the processor 110 may receive an operation instruction input by the user to modify the template or add a new problem. In step S427, the processor 110 may test the intent recognition result of the large language model with the template and the problem set. In step S428, the processor 110 may confirm the accuracy of the large language model based on the verification data to determine whether the large language model passes the test. If not, the processor 110 may re-execute step S426. If so, the construction of the large language model intent recognition prompt template database DB6 and the problem corpus database DB7 is completed.
[0059] In this embodiment, in step S425, the processor 110 may adjust the definition of the following notification tasks based on the correct and incorrect corpus examples and the abstract definition of the corpus through the large language model. For example, the initial definition of the notification task may be "this type of task is to assist the user in conveying messages to other people. The messages that the user wants to convey may include daily affairs, meeting arrangements, pending matters, archives, etc." The processor 110 may receive the correct corpus examples and incorrect corpus examples input by the user. The correct corpus examples may be "help me send the problem solution to the customer Tieke, see the attachment for details", "please help me notify Li Si and Wang Wu that there will be heavy rain in the evening" or "please tell Xiao Ming that today's project report needs to be reorganized", etc. The incorrect corpus examples may be "help me remind Li Si that his report will be handed in to me tomorrow", "please tell Wang Wu that his report needs to be submitted before 3 pm tomorrow" or "Zhang San, your task is to complete the market research report, which needs to be submitted next Tuesday", etc. The processor 110 can analyze the initial definition, correct corpus examples, and incorrect corpus examples of notification tasks through a large language model, and obtain an adjusted definition of the notification tasks. For example, based on this embodiment, the adjusted definition of the notification tasks may be "this type of task is to assist users in conveying messages to other people. The messages that users want to convey usually involve daily reminders, information sharing, or status reports. The content of the message is mainly a simple notification or reminder, and does not contain clear instructions, delivery requirements, or instructions that require an immediate response from the recipient. In addition, common notification content may include meeting reminders, weather conditions, daily itineraries, etc." Among them, "the content of the message is mainly a simple notification or reminder, and does not contain clear instructions, delivery requirements, or instructions that require an immediate response from the recipient." is the adjustment obtained by the processor 110 through a large language model in step S425.
[0060] In this embodiment, in step S426, the processor 110 can set and adjust the large language model intention recognition prompt template database DB6 and the difficult corpus database DB7 according to the operation instruction input by the user. The user can judge the intention definition outputted in the above step S425 and the corpus examples in the user feedback corpus database DB9 according to the application requirements. If the content meets the application requirements, the large language model intention recognition prompt template database DB6 and the difficult corpus database DB7 can be set and adjusted through the processor 110.
[0061] In the system operation process, in step S411, the processor 110 can perform intent recognition on the standardized complete request data 401 through the natural language understanding model MD1. In step S412, the processor 110 can determine whether advanced recognition is required based on whether the recognition result is successful. If not, it means that the natural language understanding model MD1 successfully recognizes the standardized complete request data 401. The processor 110 can generate intent data 402 through the natural language understanding model MD1. If so, it means that the natural language understanding model MD1 cannot successfully recognize the standardized complete request data 401. In step S413, the processor can perform intent recognition on the standardized complete request data 401 through a large language model to generate intent data 402. The large language model can perform intent recognition on the standardized complete request data 401 according to the large language model intent recognition prompt template provided by the large language model intent recognition prompt template database DB6 and the difficult problem corpus provided by the difficult problem corpus database DB7.
[0062] Figure 5 Schematic diagram of a data processing framework for generating result data according to an embodiment of the present invention. Figure 1 as well as Figure 5 , the data processing system 100 may include an application programming interface knowledge base DB11, a large language model construction application programming interface prompt template database DB12, an example problem set database DB13, an application programming interface (API) construction template database DB14, a user feedback example database DB15, and a task test example database DB16. Alternatively, the data processing system 100 may access the above-mentioned databases stored in other cloud servers or storage devices through a network connection. The data processing system 100 may execute steps S520 to S527 of the following system design process and steps S511 to S519 of the system operation process.
[0063] In the system design process, in step S520, the processor 110 can read and test the application interface template from the application interface construction template database DB14. In step S521, the processor 110 can determine whether the application interface template passes the test. The processor 110 can, for example, determine whether the application interface template meets the preset template conditions, such as the template format or related template requirements. If so, the construction of the application interface knowledge base DB11 is completed. If not, in step S522, the processor 110 can read the user feedback example database DB15 and the task test example database DB16 to expand the examples with a large language model. In step S523, the processor 110 can receive the operation instructions input by the user to design the application interface template to update the application interface construction template database DB14 so as to test the application interface template again.
[0064] In step S524, the processor 110 may abstractly define the example through the large language model. In step S525, the processor 110 may receive an operation instruction input by the user to modify the template or add a new problem. In step S526, the processor 110 may test the application interface construction result of the large language model with the template and the problem set. In step S527, the processor 110 may determine whether the test is passed based on the accuracy of the large language model. If not, the processor 110 may receive an operation instruction input by the user again to modify the template or add a new problem. If so, the construction of the large language model construction application interface prompt template database DB12 and the example problem set database DB13 is completed.
[0065] In the system operation process, in step S511, the processor 110 can determine the task boundary based on the standardized complete request data 501 and the intention data 502. The processor 110 can determine whether the standardized complete request data 501 and the intention data 502 have matching tasks (i.e., the corresponding application program interface program logic). In step S512, the processor 110 can determine whether it is outside the boundary. If so (indicating that there is no matching task), in step S519, the processor 110 can generate result data according to the boundary recovery mechanism. If not (indicating that there is a matching task), in step S513, the processor 110 can read the application program interface knowledge data from the application program interface knowledge base DB11. In step S514, the processor 110 can accurately construct the task data and parameter data based on the application program interface knowledge data. In step S515, the processor 110 can determine whether the precise construction of the task data and parameter data is successful. If yes (indicating that the task data and parameter data of the application program interface corresponding to the user input have been constructed), if no (indicating that the task data and parameter data of the application program interface corresponding to the user input have not been constructed), in step S517, the processor 110 can read the knowledge data of the fuzzy constructed application program interface from the large language model construction application program interface prompt template database DB12 and the example puzzle set database DB13, and then use the large language model with the template provided by the large language model construction application program interface prompt template database DB12 and the puzzle set provided by the example puzzle set database DB13 to construct the task data and parameter data of the application program interface corresponding to the user input according to the fuzzy constructed application program interface. In step S516, the processor 110 can generate a normalized output format based on the result of the precise construction of the application program interface or the result of the fuzzy construction of the application program interface, and generate task data 503 (task data of the application program interface) and parameter data 504 (parameter data of the application program interface). In step S518, the processor 110 can perform the task according to the task data 503 and the parameter data 504 to generate result data.
[0066] In this embodiment, in step S522, the content provided by the user feedback example may include information about the application program interface, such as "application program interface description", "seed application program interface example", "application program interface parameters", and the processor 110 generates a similar application program interface example through a large language model in step S522. The "application program interface description" may be, for example, "the function capability is to retrieve the interface document of the business middle platform interface in the user input". "Seed application program interface examples" may be, for example, "Call business platform API = [project set acquisition (bm.pisc.project.set.get)], input parameter = [leader number (leader_no) = input parameter (leader number (leader_no))], and remove the project status 40 (closed data) after obtaining the return parameter. --> bm.pisc.project.set.get", "Call eoc api [ / api / eoc / v2 / emp / user] to obtain employee information based on the leader number in the input parameter. If it does not exist, an error message will be given [The corresponding employee information does not exist!]; if it exists, first call the business middle platform interface [bm.ssc.supplier.get] and then call the business middle platform interface [bmd.imsc.item.all.get]" or "Call EOC based on the leader number in the input parameter API [ / api / eoc / v2 / emp / user] is used to obtain employee information. If the information is not found, an error message will be displayed [employee information for the current user does not exist]. After the information is found, [bmd.imsc.item.all.get] and [bmd.basc.company.get] will be called in parallel. "Application program interface parameters" can be, for example, "bm.crm.project.get" or "bmd.user.get". According to the above information in this embodiment, the processor 110 can generate an application program interface example similar to a "seed application program interface example" through a large language model, such as "calling the EOC API [ / api / eoc / v2 / customer / info] according to the customer number in the input parameter to obtain customer information. If the customer does not exist, an error message [the customer information is not found] is reported; if found, the business middle platform interface [bm.crm.project.get] is called to obtain project information related to the customer" or "calling the EOC API [ / api / eoc / v2 / emp / user] according to the employee ID in the input parameter to obtain employee detailed information. If it does not exist, an error message [employee information not found] is reported; if it exists, the business middle platform interface [bmd.user.get] is called to obtain relevant business data of the employee", etc. In this embodiment, the processor 110 can filter the application program interface examples output in step S522 according to the operation instructions input by the user in step S523.
[0067] In this embodiment, in step S524, the task description of the processor 110 to abstractly define the application interface example through the large language model may be, for example, "adjust the definition of the application interface description according to the correct and incorrect application interface examples". In this regard, the initial definition of the application interface description may be, for example, "the function capability is to retrieve the interface document of the business middle platform interface in the user input". The correct application interface example may be, for example, "call the business platform interface [bmd.order.info.get] according to the order number in the input parameter to obtain the order details, and after obtaining, remove the data with the order status of [cancelled]; then call [bmd.inventory.check.get] to obtain the inventory check information" or "the input supplier ID first calls the business middle platform interface [bm.ssc.supplier.get] to obtain the supplier information, and if it does not exist, it will report an error [no corresponding supplier information]; if it exists, then call [bm.purchase.contract.get] to obtain the relevant purchase contract information" and so on. An example of an incorrect application interface may be, for example, "Call the application interface [ / api / eoc / v2 / customer / info] according to the customer number in the input parameter to obtain customer information. If the customer does not exist, an error message [the customer information is not found] is reported; if found, the business middle platform interface [bmd.crm.project.get] is called to obtain project information related to the customer" or "Call the API [ / api / eoc / v2 / emp / user] according to the employee ID in the input parameter to obtain employee details. If it does not exist, an error message [employee information not found] is reported; if it exists, the business middle platform interface [bmd.user.get] is called to obtain the employee's relevant business data." In this embodiment, the processor 110 can analyze the initial definition of the application interface description, the correct application interface example, and the incorrect application interface example through a large language model, and obtain the adjusted definition of the application interface description, which may be, for example, "This function is used to identify and retrieve interface documents related to the business middle platform based on user input, and is limited to business middle platform interfaces starting with [bmd] or [bm]. This function will not process non-business middle platform or external API interfaces." In this embodiment, the processor 110 may confirm or modify the adjusted definition of the API description outputted in step S524 according to the operation instruction inputted by the user in step S525 .
[0068] Figure 6 FIG. 1 is a schematic diagram of a data processing framework for displaying reply task result data according to an embodiment of the present invention. Figure 1 as well as Figure 6, the data processing system 100 may include a context history database DB1, a user feedback synonym database DB4, a user feedback rule database DB5, a user feedback corpus database DB9, a user feedback example database DB15, and a wrong question collection database DB17. Alternatively, the data processing system 100 may access the above-mentioned databases stored in other cloud servers or storage devices through a network connection. The data processing system 100 may execute the following steps S621 of the system design process and S611 to S614 of the system operation process.
[0069] In the system design process, in step S621, the processor 110 may read the wrong question set database DB17 to review the wrong question set and perform analysis. The processor 110 may update at least one of the user feedback synonym database DB4, the user feedback rule database DB5, the user feedback corpus database DB9, and the user feedback example database DB15 according to the analysis result.
[0070] In the system operation process, in step S611, the processor 110 may generate the reply task result data 601 according to the input data, task data, parameter data and result data generated by the above-mentioned processes. Alternatively, in the context of the above-mentioned step S519 (there is no corresponding task), the processor 110 may generate the reply task result data 601 according to the input data and result data generated by the above-mentioned processes. In step S612, the processor 110 may update the context history record database DB1 according to the reply task result data. In step S613, the processor 110 may display the reply task result data 601 through the user interface displayed by the display device. In step S614, the processor 110 may receive user feedback through the input device to generate a wrong question set according to the user's subsequent operation on the reply task result data 601, so as to effectively and dynamically correct the system.
[0071] In summary, the data processing system and data processing method of the present invention can effectively realize the application and development process framework of a task-based assistant that combines a natural language processing process framework with a large language model, and can effectively generate corresponding reply task result data based on the input data in the voice format input by the user, so as to achieve a good user experience and task-based assistant function.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data processing system, characterized in that: include: A storage device for storing a plurality of models; as well as a processor, coupled to the storage device and configured to execute the plurality of models, wherein the processor receives input data, and the processor generates complete request data according to the input data, wherein the processor inputs the complete request data into a large language model to generate standardized complete request data, and the processor inputs the standardized complete request data into a natural language understanding model to generate intent data, wherein the processor performs task boundary judgment according to the standardized complete request data and the intention data to generate application program interface task data, application program interface parameter data and result data, and the processor generates reply task result data according to the input data, the standardized complete request data, the application program interface task data, the application program interface parameter data and the result data, The processor displays the reply task result data through a user interface.
2. The data processing system according to claim 1, characterized in that The processor inputs the input data in speech format to a speech-to-text model to convert the input data into text format.
3. The data processing system according to claim 1, characterized in that: The processor searches for corresponding context data according to the input data to generate the complete request data, and the processor inputs the complete request data and corresponding related word data into the large language model to generate the standardized complete request data.
4. The data processing system according to claim 3, characterized in that: The processor rewrites the complete request data according to the corresponding related word data based on a preset replacement rule, and confirms the rewriting result through the large language model to generate the standardized complete request data.
5. The data processing system according to claim 1, characterized in that: The processor performs intent recognition on the standardized complete request data through the natural language understanding model, When the natural language understanding model cannot successfully identify the standardized complete request data, the processor performs intent recognition on the standardized complete request data through the large language model to generate the intent data. When the natural language understanding model successfully identifies the standardized complete request data, the processor generates the intent data through the natural language understanding model.
6. The data processing system according to claim 5, characterized in that: The large language model performs intent recognition on the standardized complete request data according to a large language prompt template and a difficult problem corpus.
7. The data processing system according to claim 1, characterized in that: When the processor determines that the request data corresponds to the task boundary according to the standardized complete request data and the intent data, the processor reads the corresponding application program interface to generate the application program interface task data and the application program interface parameter data. When the processor determines that the data corresponds to outside the task boundary according to the standardized complete request data and the intention data, the processor generates the result data according to a boundary recovery mechanism.
8. The data processing system according to claim 7, characterized in that: When the processor successfully accurately constructs the API task data and the API parameter data, the processor generates a normalized output format, the API task data and the API parameter data, When the processor is unable to successfully construct the application interface task data and the application interface parameter data accurately, the processor reads the knowledge data of the fuzzily constructed application interface, and the processor combines templates and problem set construction to generate the normalized output format, the application interface task data and the application interface parameter data according to the fuzzily constructed application interface.
9. The data processing system according to claim 1, characterized in that: The processor also generates the reply task result data according to the context history record, and updates the context history record according to the reply task result data.
10. The data processing system according to claim 1, characterized in that: The processor corrects at least one of a user feedback synonym database, a user feedback rule database, a user feedback corpus database, and a user feedback example database according to correction data corresponding to the reply task result data.
11. A data processing method, characterized in that: include: receiving input data, and generating complete request data according to the input data; Inputting the complete request data into a large language model to generate standardized complete request data; Inputting the standardized complete request data into a natural language understanding model to generate intent data; Performing task boundary judgment according to the standardized complete request data and the intent data to generate application program interface task data, application program interface parameter data and result data; Generate reply task result data according to the input data, the standardized complete request data, the application program interface task data, the application program interface parameter data and the result data; as well as The reply task result data is displayed through a user interface.
12. The data processing method according to claim 11, characterized in that: Also includes: The input data in speech format is input into a speech-to-text model to be converted into the input data in text format.
13. The data processing method according to claim 11, characterized in that: The steps of generating the standardized complete request data include: Searching for corresponding context data according to the input data to generate the complete request data; and The complete request data and the corresponding related word data are input into the large language model to generate the standardized complete request data.
14. The data processing method according to claim 13, characterized in that: The step of generating the standardized complete request data further includes: Rewrite the complete request data according to the corresponding related word data based on a preset replacement rule; and The rewriting result is confirmed by the large language model to generate the standardized complete request data.
15. The data processing method according to claim 11, characterized in that: The steps of generating the intention data include: Performing intent recognition on the standardized complete request data by using the natural language understanding model; When the natural language understanding model cannot successfully identify the standardized complete request data, performing intent identification on the standardized complete request data by using the large language model to generate the intent data; and When the natural language understanding model successfully identifies the standardized complete request data, the intent data is generated by the natural language understanding model.
16. The data processing method according to claim 15, characterized in that: The large language model performs intent recognition on the standardized complete request data according to a large language prompt template and a difficult problem corpus.
17. The data processing method according to claim 11, characterized in that: The steps of generating the result data include: When it is determined that the standardized complete request data and the intent data correspond to within the task boundary, reading the corresponding application program interface to generate the application program interface task data and the application program interface parameter data; and When it is determined that the standardized complete request data and the intention data correspond to outside the task boundary, the result data is generated according to a boundary recovery mechanism.
18. The data processing method according to claim 17, characterized in that: The steps of generating the application program interface task data and the application program interface parameter data include: When the API task data and the API parameter data are accurately constructed successfully, generating a normalized output format, the API task data and the API parameter data; and When the application interface task data and the application interface parameter data cannot be accurately constructed successfully, the knowledge data of the fuzzily constructed application interface is read, and the template and the problem set are used to generate the normalized output format, the application interface task data and the application interface parameter data according to the fuzzily constructed application interface.
19. The data processing method according to claim 11, characterized in that: The step of generating the reply task result data comprises: The reply task result data is also generated according to the context history record; and The context history record is updated according to the reply task result data.
20. The data processing method according to claim 11, characterized in that: Also includes: At least one of the user feedback synonym database, the user feedback rule database, the user feedback corpus database, and the user feedback example database is corrected according to the correction data corresponding to the reply task result data.