Big data platform error processing method and device based on big language model, equipment and storage medium

Abstract

The invention provides a big data platform error processing method and device based on a big language model, electronic equipment, a computer readable storage medium and a computer program product, and relates to the field of artificial intelligence, in particular to the technical fields of intelligent error correction, natural language processing, big language models and the like. The method comprises the following steps: in response to a received error correction request, obtaining error reporting information generated by a big data platform and a structured query language (SQL) statement associated with the error reporting information; constructing prompt word information based on the error reporting information and the SQL statement; and inputting the prompt word information into the large language model, and generating an error correction suggestion for the error report information. According to the method, the troubleshooting period during error reporting of the big data platform is shortened, and the processing efficiency of a user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to artificial intelligence technical fields such as intelligent error correction, natural language processing and large language models, and in particular to a big data platform error processing method, device, electronic device, computer-readable storage medium and computer program product based on a large language model. Background Art

[0002] At present, global enterprises use big data platforms to conduct refined operations, thereby accelerating daily business processes and decision-making, helping data drive front-line business, and bringing business growth. When a data analysis task reports an error, it is necessary to analyze the cause of the error and obtain a solution to ensure the normal progress of the data analysis task. Summary of the invention

[0003] The disclosed embodiments propose a big data platform error processing method, device, electronic device, computer-readable storage medium and computer program product based on a large language model, which can improve the processing efficiency when task errors are reported.

[0004] In a first aspect, an embodiment of the present disclosure proposes a big data platform error processing method based on a large language model, comprising: in response to receiving an error correction request, obtaining error information generated by the big data platform and a structured query language SQL statement associated with the error information; constructing prompt word information based on the error information and the SQL statement; inputting the prompt word information into the large language model to generate error correction suggestions for the error information.

[0005] In the second aspect, the embodiment of the present disclosure proposes a big data platform error processing device based on a large language model, including: an information acquisition module, configured to obtain error information generated by the big data platform and a structured query language SQL statement associated with the error information in response to receiving an error correction request; a prompt word construction module, configured to construct prompt word information based on the error information and the SQL statement; a generation module, configured to input the prompt word information into the large language model to generate error correction suggestions for the error information.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it is able to implement the big data platform error processing method based on a large language model as described in any implementation method in the first aspect.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the big data platform error processing method based on a large language model as described in any implementation method in the first aspect when executed.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the big data platform error processing method based on a large language model as described in any implementation method in the first aspect.

[0009] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0011] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0012] Figure 2 A flowchart of a method for processing errors on a big data platform based on a large language model provided in an embodiment of the present disclosure;

[0013] Figure 3 A flowchart of another big data platform error processing method based on a large language model provided in an embodiment of the present disclosure;

[0014] Figure 4 A flowchart of another method for processing errors on a big data platform based on a large language model provided in an embodiment of the present disclosure;

[0015] Figure 5 A flowchart of a method for processing an error on a big data platform based on a large language model in an application scenario provided by an embodiment of the present disclosure;

[0016] Figure 6 A structural block diagram of a big data platform error processing device based on a large language model provided in an embodiment of the present disclosure;

[0017] Figure 7 A schematic diagram of the structure of an electronic device suitable for executing a big data platform error processing method based on a large language model provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0019] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0020] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the method, apparatus, electronic device, and computer-readable storage medium for error processing on a big data platform based on a large language model of the present disclosure can be applied.

[0021] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0022] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 and server 105 can be installed with various applications for realizing information communication between the two, such as web browser applications, search applications, instant messaging applications, etc.

[0023] Terminal devices 101, 102, 103 and server 105 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.; when terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, and no specific limitation is made here.

[0024] The server 105 can provide various services through various built-in applications. Taking the background management server that can provide solutions based on error correction requests as an example, the server 105 can achieve the following effects when running the web browser application: first, in response to receiving an error correction request, the error information generated by the big data platform and the structured query language SQL statement associated with the error information are obtained, and then prompt word information is constructed based on the error information and the SQL statement. Finally, the prompt word information is input into the big language model to generate error correction suggestions for the error information.

[0025] It should be noted that, in addition to being obtained from the terminal devices 101, 102, 103 through the network 104, the error correction request, error information, and SQL statement can also be pre-stored in the server 105 in various ways. Therefore, when the server 105 detects that these data have been stored locally (for example, when starting to process the previously retained solution generation task), it can choose to directly obtain these data locally. In this case, the exemplary system architecture 100 may also not include the terminal devices 101, 102, 103 and the network 104.

[0026] Since generating a solution based on an error correction request requires more computing resources and stronger computing power, the big data platform error processing method based on a large language model provided in the subsequent embodiments of the present disclosure is generally executed by a server 105 with stronger computing power and more computing resources. Accordingly, the big data platform error processing device based on a large language model is generally also set in the server 105. However, it should also be pointed out that when the terminal devices 101, 102, and 103 also have computing power and computing resources that meet the requirements, the terminal devices 101, 102, and 103 can also complete the above-mentioned operations that are handed over to the server 105 through the background management server installed thereon, and then output the same result as the server 105. Especially in the case where there are multiple terminal devices with different computing powers at the same time, but when the background management server determines that the terminal device has stronger computing power and more remaining computing resources, it can allow the terminal device to perform the above-mentioned operations, thereby appropriately reducing the computing pressure of the server 105. Correspondingly, the big data platform error processing device based on a large language model can also be set in the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also not include the server 105 and the network 104 .

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0028] Please refer to Figure 2 , Figure 2 A flowchart of a method for processing a big data platform error based on a large language model provided in an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0029] Step 201: In response to receiving an error correction request, obtaining error information generated by the big data platform and a structured query language SQL statement associated with the error information.

[0030] In this embodiment, the execution subject (eg Figure 1 When the server 105 shown receives an error correction request, it obtains the error information generated by the big data platform and the structured query language SQL statement associated with the error information. Specifically, after the execution subject receives the error correction request through various methods (such as front-end form submission, text input, voice input, etc.), it will obtain the error information during the execution of the big data platform and record the SQL statement associated with the error information. Among them, the big data platform is a highly integrated system designed to meet the needs of storage, processing and analysis of massive data, and provide data-driven insights and decision support for various enterprises and organizations. Error information refers to error prompts or messages that appear when operating the big data platform, including the type of error and the cause of the error.

[0031] In this embodiment, the structured query language SQL statement is a standard programming language for managing and operating a relational big data platform. SQL is used to perform various big data platform operations, including data query, update, insert and delete.

[0032] Step 202: Construct prompt word information based on the error message and the SQL statement.

[0033] In this embodiment, the execution subject constructs prompt word information based on the error message and SQL statement obtained in step 201. Among them, prompt word information usually refers to the text or question input to the model, which is used to guide the model to generate relevant answers. In this embodiment, prompt word information refers to the text input to the large language model generated according to the error message and SQL statement, which is used to guide the large language model to generate relevant error correction suggestions. For example, the error message is that the data type does not match, and the SQL statement that matches the error message is {INSERT INTO users(id,username,age)VALUES(1,'Alice','Twenty-five');}, and the prompt word information constructed according to the error message and the SQL statement can be: Error message: Data type does not match; SQL statement: {INSERT INTO users(id,username,age)VALUES(1,'Alice','Twenty-five');}; Please generate a solution to handle this error problem.

[0034] Step 203: Input the prompt word information into the large language model to generate error correction suggestions for the error information.

[0035] In this embodiment, the prompt word information constructed in step 202 is input into the large language model to generate error correction suggestions for the error information. Specifically, the prompt word information may include error information, SQL statements, and instructions for guiding the output of the large language model. The execution subject inputs the prompt word information into the large language model. The large language model (LLM) is an artificial intelligence model based on deep learning, which is specifically used to understand and generate natural language text. In this embodiment, the large language model can generate error correction suggestions for the error information, and return the error correction suggestions to the user. The user can modify the SQL statement according to the error correction suggestions to ensure the normal operation of the big data platform. Among them, the error correction suggestion includes error correction guidance and error correction instructions. The error correction guidance can be a correct SQL statement after modification, or it can be an error correction step for solving the error problem. The error correction instructions can include error explanation, error correction ideas, preventive measures, verification schemes, etc.

[0036] The big data platform error processing method based on the big language model provided by the embodiment of the present disclosure, in response to receiving an error correction request, constructs prompt word information according to the error information returned by the big data platform and the corresponding SQL statement, and uses the big language model to generate corresponding error correction suggestions, thereby improving development efficiency, shortening the error troubleshooting cycle, and reducing the time cost of error troubleshooting.

[0037] In some optional implementations of this embodiment, the execution subject may determine that an error correction request has been received in response to an error report from the big data platform. Specifically, when the execution subject monitors an error report event on the big data platform and determines that an error correction needs to be performed on the big data platform error report event, the error report information generated by the database and the structured query language SQL statement associated with the error report information may be obtained.

[0038] In some optional implementations of this embodiment, the execution subject may determine that an error correction request has been received in response to receiving a trigger signal of an error correction control. Specifically, when an error event occurs on the big data platform, the user may send an error correction request to the execution subject by triggering the error correction control, and when the execution subject receives the trigger signal of the error correction control, it is determined that the error correction request has been received. The error correction control may be an error correction button located on the front-end interface, and the user sends an error correction request to the execution subject by clicking the error correction button, which facilitates the user to interact with the execution subject.

[0039] Please refer to Figure 3 , Figure 3A flowchart of another method for processing errors on a big data platform based on a large language model provided by an embodiment of the present disclosure, wherein process 300 includes the following steps:

[0040] Step 301: In response to receiving an error correction request, obtain error information generated by the big data platform and a structured query language SQL statement associated with the error information.

[0041] In this embodiment, the execution subject (eg Figure 1 When the server 105 shown receives the error correction request, it obtains the error information generated by the big data platform and the structured query language SQL statement associated with the error information. Specifically, when the big data platform reports an error, it will send an error correction request to the execution subject and generate an error message for describing the cause of the error. After receiving the error correction request, the execution subject will obtain the error information during the execution of the big data platform and record the SQL statement associated with the error information.

[0042] In this embodiment, the common reasons for the big data platform to report an error are as follows: 1. Syntax error, which means that the syntax of the SQL statement does not meet the requirements of the big data platform, such as forgetting to use keywords, missing quotes or brackets, etc.; 2. Data type mismatch, which means that when inserting or updating data, the data type is inconsistent with the table definition, such as inserting a string into an integer field; 3. Stored procedure or function error, which means an error occurs when calling a function, such as incorrect parameter passing or logical errors, etc.; 4. Insufficient computing resources, which means that the server resources of the big data computing queue are insufficient, such as memory, disk space reaching the upper limit, etc.; 5. Permission issues, which means that the current user does not have permission to perform a specific operation, such as trying to update or delete a table without permission; 6. Timeout error, which means that the operation execution exceeds the specified time limit, such as a query that has not responded for a long time. In addition to the above, there are some other reasons for errors in calculations, which are not listed here one by one.

[0043] In this embodiment, based on the common reasons for the big data platform to report errors, the error information may include SQL syntax error information and / or task failure error information. Among them, the SQL syntax error information is the error information generated by the big data platform when a syntax error, data type mismatch, stored procedure or function error occurs, and the task failure error information is the error information generated by the big data platform when a permission problem, insufficient computing resources, timeout error or other common errors occur.

[0044] In this embodiment, the structured query language SQL statement is a standard programming language for managing and operating a relational big data platform. SQL is used to perform various big data platform operations, including data query, update, insert and delete.

[0045] Step 302: Invoke the target prompt template according to the error message.

[0046] In this embodiment, the execution entity invokes the target prompt template according to the error message. Among them, the target prompt template can be a format template pre-set by developers for defining the format of the generated prompt information.

[0047] In some optional implementation manners of this embodiment, the target prompt template can be constructed based on the following methods: Obtain sample error messages from the sample library, as well as the corresponding sample prompts and sample error correction suggestions for the sample error messages; Generate initial prompts corresponding to the sample error messages based on the initial prompt template; In response to the matching value between the initial prompt and the sample prompt being less than the first threshold, update the initial prompt template; Construct the target prompt template based on the updated initial prompt template. Among them, the sample library contains multiple sample sets, and each sample set contains sample error messages and the corresponding sample prompts and sample error correction suggestions for the sample error messages. The execution entity generates initial prompts corresponding to the sample error messages based on the initial prompt template. The initial prompt template is pre-set, and the initial prompt can be generated according to the sample error messages in the sample set and the initial prompt template.

[0048] In some optional implementation manners of this embodiment, the execution entity can calculate the first matching value between the initial prompt and the sample prompt through various methods such as character matching, text similarity algorithm, term frequency-inverse document frequency (TF-IDF) method, and deep learning model. For example, the TF-IDF method is used to calculate the first matching value between the initial prompt and the sample prompt. Specifically, first, the initial prompt and the sample prompt are respectively tokenized, and common meaningless words (such as "de", "shi", "zai", etc.) are removed to obtain the keywords corresponding to the initial prompt and the sample prompt. Then, calculate the frequency (TF) of each keyword appearing in the initial prompt and the sample prompt respectively. Next, calculate the inverse document frequency (IDF) of each keyword. Finally, calculate the TF-IDF vectors of the initial prompt and the sample prompt, and use cosine similarity to calculate the similarity between the two TF-IDF vectors, and use this similarity as the first matching value between the initial prompt and the sample prompt.

[0049] In some alternative implementation manners of this embodiment, when the matching value between the initial prompt and the sample prompt is less than a preset first threshold, semantic detection can be performed on the initial prompt, and the initial prompt can be updated according to the results of the semantic detection. The content of the semantic detection may include at least one of the following: logical detection, statement detection, typo detection. A prompt evaluation criterion can be preset, and developers can perform manual evaluation and adjustment on the initial prompt template according to the prompt evaluation criterion to make the generated initial prompt more in line with the true intention of the user, so as to construct a target prompt template that better meets the user's needs.

[0050] In some alternative implementation manners of this embodiment, the execution subject constructs a target prompt template based on the updated initial prompt template. Specifically, based on the updated initial prompt template, an updated prompt corresponding to the sample error message is generated, and the updated prompt information is input into the large language model to generate an updated error correction suggestion. In response to the matching value between the updated error correction suggestion and the sample error correction suggestion being less than a preset second threshold, the updated initial prompt template is updated again to obtain the target prompt template.

[0051] In some alternative implementation manners of this embodiment, the second matching value between the updated error correction suggestion and the sample error correction suggestion can be calculated in multiple ways, such as character matching, text similarity algorithm, term frequency-inverse document frequency (TF-IDF) method, deep learning model, etc. For example, the TF-IDF method is used to calculate the second matching value between the updated error correction suggestion and the sample error correction suggestion. Specifically, first, the updated error correction suggestion and the sample error correction suggestion are respectively tokenized, and common meaningless words (such as "of", "is", "in", etc.) are removed to obtain the keywords corresponding to the updated error correction suggestion and the sample error correction suggestion. Then, the frequency (TF) of each keyword appearing in the updated error correction suggestion and the sample error correction suggestion is calculated. Next, the inverse document frequency (IDF) of each keyword is calculated. Finally, the TF-IDF vectors of the updated error correction suggestion and the sample error correction suggestion are calculated, and the cosine similarity is used to calculate the similarity between the two TF-IDF vectors, and this similarity is used as the second matching value between the updated error correction suggestion and the sample error correction suggestion.

[0052] In some optional implementations of the present embodiment, the target prompt word template may be updated in the following manner: based on user feedback, the current target prompt word template is updated to generate an updated prompt word template, so that two groups of users use the current target prompt word template and the updated prompt word template respectively, and the two groups of users score the current target prompt word template and the updated prompt word template respectively according to preset evaluation criteria, and the performance difference between the current target prompt word template and the updated prompt word template is analyzed based on the scoring; if the performance of the current target prompt word template is better than that of the updated prompt word template, the current target prompt word template is still used as the target prompt word template; if the performance of the updated prompt word template is better than that of the current target prompt word template, the updated prompt word template is used as the target prompt word template.

[0053] In some optional implementations of this embodiment, the target prompt word template can be updated through regular review and evaluation. Specifically, the use effect of the target prompt word template is regularly analyzed to check the quality and relevance of the generated prompt word information, find out where improvements are needed, collect user feedback on the target prompt word template, determine what needs to be adjusted, and update the target prompt word template.

[0054] In some optional implementations of this embodiment, new prompt words may be added according to the newly added big data platform error information, and the language style and content of the target prompt word template may be adjusted according to the characteristics of different user groups to update the target prompt word template.

[0055] Step 303: construct prompt word information based on the target prompt word template, error message and SQL statement.

[0056] In this embodiment, the execution subject can construct prompt word information based on the target prompt word template according to the error message and the SQL statement. The target prompt word template is a general prompt word template generated by steps 3021 to 3024. The execution subject can assemble the prompt project according to the strategy pattern based on the target prompt word template, and then fill the assembled prompt project according to the error message and the SQL statement to generate prompt word information. Among them, the strategy pattern is a behavioral design pattern, which defines a series of algorithms and encapsulates each algorithm so that the algorithms can be interchangeable. In this embodiment, the strategy pattern is used to flexibly construct the structure of the prompt project according to different error messages. Prompt usually refers to a text prompt or command input to the model. When interacting with the large language model in this embodiment, the text (question, command or topic) input by the user is a prompt, and the model generates corresponding output according to this prompt. Prompt engineering refers to the process of generating and optimizing prompts when interacting with the large language model, so that the large language model generates the best output results.

[0057] Step 304: Input the prompt word information into the large language model to generate error correction suggestions for the error information.

[0058] In this embodiment, the specific operation of step 304 is already Figure 2 Step 203 is described in detail in the illustrated embodiment and will not be described again here.

[0059] Step 305: associate the error message, prompt word information and error correction suggestion and save them in the sample library.

[0060] In this embodiment, the execution entity may also associate the error information, prompt word information and generated correction suggestions obtained during the error correction process, and save them as a sample set in the sample library to facilitate subsequent continuous adjustment and optimization of the target prompt word template.

[0061] The big data platform error processing method based on the big language model provided in this embodiment can quickly generate prompt words corresponding to the error message by calling the pre-trained target prompt word template, provide clear guidance to help the big language model generate accurate answers, and help the big language model better understand the needs of users and interact efficiently and accurately. In addition, the target prompt word template can also be optimized and reused according to a variety of different types of error messages, reducing time and labor costs.

[0062] Please refer to Figure 4 , Figure 4 A flowchart of another method for processing errors on a big data platform based on a large language model provided by an embodiment of the present disclosure, wherein process 400 includes the following steps:

[0063] Step 401: In response to receiving an error correction request, obtaining error information generated by the big data platform and a structured query language SQL statement associated with the error information.

[0064] In this embodiment, the specific operation of step 401 is already Figure 2 In the illustrated embodiment, step 201 is introduced in detail and will not be repeated here.

[0065] Step 402: construct prompt word information based on the error message and the SQL statement.

[0066] In this embodiment, the specific operation of step 402 is already Figure 2 Step 202 is described in detail in the illustrated embodiment and will not be described again here.

[0067] Step 403: According to the interface standard of the large language model, configure the interface parameters of the model calling interface.

[0068] In this embodiment, the execution subject configures the interface parameters of the model call interface according to the interface standard of the large language model to ensure that the execution subject can successfully call the large language model. Among them, the interface standard refers to the rules and protocols that define the interaction between different components, systems or services in computer systems, software development or network communications. These standards ensure that different systems or modules can seamlessly exchange information and interoperate. The model call interface is used to interact with the large language model, including sending input text, configuring model parameters, and receiving returned results.

[0069] Step 404: calling the large language model through the configured model calling interface, and inputting the prompt word information into the large language model to generate error correction suggestions for the error information.

[0070] In this embodiment, the execution subject calls the large language model through the configured model calling interface and inputs the prompt word information into the large language model. The large language model generates error correction suggestions for the error information and returns the error correction suggestions to the execution subject.

[0071] The big data platform error processing method based on the large language model provided in this embodiment uses the power of the large language model to solve the problem that ordinary users cannot solve task errors by themselves when performing data analysis on the big data platform, and strongly rely on data platform developers to manually solve the problems, thereby avoiding long troubleshooting cycles and low efficiency, improving user experience, and improving business decision-making efficiency.

[0072] To deepen understanding, the present disclosure also provides a specific implementation solution in combination with a specific application scenario, see Figure 5 As shown. In this implementation, the execution subject of the big data platform error processing method based on the large language model can be an intelligent error correction system. The front-end interaction module 510 in the intelligent error correction system can interact with the intelligent error correction module 520, the prompt engineering module 530 and the large language model 540. The specific big data platform error processing process 500 based on the large language model includes:

[0073] Step 501: When the big data platform reports an error, the user actively triggers the intelligent error correction button through the front-end interaction module 510 to send an error correction request to the intelligent error correction module 520.

[0074] Step 502: The intelligent error correction module 520 receives the error correction request, obtains the error information generated by the big data platform and the structured query language SQL statement associated with the error information.

[0075] Step 503: calling the prompt engineering module 530, and the prompt engineering module 530 obtains the optimized target prompt word template.

[0076] Step 504: construct prompt word information according to the target prompt word template, error message and SQL statement.

[0077] Step 505 : calling the large language model 540 , and configuring the interface parameters of the model calling interface according to the interface standard of the large language model 540 .

[0078] Step 506: The large language model 540 generates error correction suggestions for the error information according to the prompt word information.

[0079] Step 507: Return the error correction suggestion to the front-end interaction module 520, and the front-end interaction module 520 outputs the error correction suggestion.

[0080] The big data platform error processing method based on the large language model provided in this embodiment uses the power of the large language model to solve the problem that ordinary users cannot solve task errors by themselves when performing data analysis on the big data platform, and strongly rely on data platform developers to manually solve the problems, thereby avoiding long troubleshooting cycles and low efficiency, improving user experience, and improving business decision-making efficiency.

[0081] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of the device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0082] like Figure 6 As shown, the big data platform error processing device 600 based on the large language model of this embodiment may include: an information acquisition module 601, a prompt word construction module 602, and a generation module 603.

[0083] The information acquisition module 601 is configured to acquire the error information generated by the big data platform and the structured query language SQL statement associated with the error information in response to receiving the error correction request.

[0084] The prompt word building module 602 is configured to build prompt word information based on the error message and the SQL statement.

[0085] The generation module 603 is configured to input the prompt word information into the large language model to generate error correction suggestions for the error information.

[0086] In this embodiment, in the big data platform error processing device 600 based on the big language model: the specific processing of the information acquisition module 601, the prompt word construction module 602, and the generation module 603 and the technical effects brought by them can be referred to respectively. Figure 2 The relevant descriptions of steps 201 - 203 in the corresponding embodiment are not repeated here.

[0087] In some optional implementations of this embodiment, the prompt word construction module 602 includes: a first calling module 604, configured to call the target prompt word template according to the error information; a prompt word construction submodule 605, configured to construct prompt word information based on the target prompt word template, the error information and the SQL statement.

[0088] In some optional implementations of the present embodiment, the target prompt word template in the first calling module 604 is constructed based on the following steps: a sample acquisition unit is configured to acquire sample error information, as well as sample prompt words and sample error correction suggestions corresponding to the sample error information from a sample library; a generation unit is configured to generate initial prompt words corresponding to the sample error information based on the initial prompt word template; a first update unit is configured to update the initial prompt word template in response to a matching value between the initial prompt word and the sample prompt word being less than a first threshold; and a template construction unit is configured to construct a target prompt word template based on the updated initial prompt word template.

[0089] In some optional implementations of the present embodiment, the template construction unit in the first calling module 604 is further configured to: generate updated prompt words corresponding to the sample error information based on the updated initial prompt word template; input the updated prompt word information into the large language model to generate updated error correction suggestions; in response to the matching value between the updated error correction suggestions and the sample error correction suggestions being less than a second threshold, update the updated initial prompt word template again to obtain the target prompt word template.

[0090] In some optional implementations of the present embodiment, updating of the initial prompt word template in the first update unit in the first calling module 604 includes: a detection unit, configured to perform semantic detection on the initial prompt word, wherein the content of the semantic detection includes at least one of the following: logic detection, sentence detection, and typo detection; and a second update unit, configured to update the initial prompt word template according to the result of the semantic detection.

[0091] In some optional implementations of this embodiment, the first calling module 604 further includes: a storage unit configured to associate and save the error reporting information, the prompt word information, and the error correction suggestion in a sample library.

[0092] In some optional implementations of this embodiment, the generation module 603 includes: a parameter configuration module 606, configured to configure the interface parameters of the model calling interface according to the interface standard of the large language model; a second calling module 607, configured to call the large language model through the configured model calling interface, and input the prompt word information into the large language model to generate error correction suggestions for the error information.

[0093] In some optional implementations of the present embodiment, the big data platform error processing device 600 based on the large language model of the present embodiment also includes: a first response unit, configured to determine that an error correction request has been received in response to a big data platform error; or a second response unit, configured to determine that an error correction request has been received in response to receiving a trigger signal of an error correction control.

[0094] This embodiment exists as a device embodiment corresponding to the above method embodiment. The big data platform error processing device based on the large language model provided by this embodiment can quickly generate prompt words corresponding to the error message by calling the pre-trained target prompt word template, and provide clear guidance to help the large language model generate accurate answers, help the large language model better understand the user's needs, and interact efficiently and accurately. In addition, the target prompt word template can also be optimized and reused through a variety of different types of error messages, reducing time and labor costs.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the big data platform error processing method based on a large language model described in any of the above embodiments can be implemented when the at least one processor executes.

[0096] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the big data platform error processing method based on a large language model described in any of the above embodiments when executed.

[0097] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which, when executed by a processor, can implement the big data platform error processing method based on a large language model described in any of the above embodiments.

[0098] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0099] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a big data platform error processing method based on a large language model. For example, in some embodiments, the big data platform error processing method based on a large language model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the big data platform error processing method based on the large language model described above may be executed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute a big data platform error processing method based on a large language model in any other appropriate manner (for example, by means of firmware).

[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0104] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0108] According to the technical solution of the embodiment of the present disclosure, by calling the pre-trained target prompt word template, the prompt word corresponding to the error message can be quickly generated, providing clear guidance to help the large language model generate accurate answers, helping the large language model to better understand the user's needs and perform efficient and accurate interaction. In addition, the target prompt word template can also be optimized and reused through a variety of different types of error messages, reducing time and labor costs.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A big data platform error processing method based on a large language model, comprising: In response to receiving the error correction request, obtaining error information generated by the big data platform and a structured query language SQL statement associated with the error information; Constructing prompt word information based on the error message and the SQL statement; The prompt word information is input into a large language model to generate error correction suggestions for the error information.

2. The method according to claim 1, wherein: The constructing prompt word information based on the error message and the SQL statement includes: Calling a target prompt word template according to the error message; Prompt word information is constructed based on the target prompt word template, the error message and the SQL statement.

3. The method according to claim 2, wherein: The target prompt word template is constructed based on the following steps: Obtaining sample error information, as well as sample prompt words and sample error correction suggestions corresponding to the sample error information from the sample library; Based on the initial prompt word template, generate the initial prompt word corresponding to the sample error message; In response to a matching value between the initial prompt word and the sample prompt word being less than a first threshold, updating the initial prompt word template; The target prompt word template is constructed based on the updated initial prompt word template.

4. The method according to claim 3, wherein: The step of constructing the target prompt word template based on the updated initial prompt word template includes: Based on the updated initial prompt word template, generating an updated prompt word corresponding to the sample error message; Inputting the updated prompt word information into a large language model to generate updated error correction suggestions; In response to the matching value between the updated error correction suggestion and the sample error correction suggestion being less than a second threshold, the updated initial prompt word template is updated again to obtain the target prompt word template.

5. The method according to claim 3, wherein: The updating of the initial prompt word template includes: Performing semantic detection on the initial prompt word, wherein the semantic detection includes at least one of the following: logic detection, sentence detection, and typo detection; The initial prompt word template is updated according to the result of the semantic detection.

6. The method according to claim 3, further comprising: The error reporting information, the prompt word information and the error correction suggestion are associated and saved in the sample library.

7. The method according to claim 1, wherein: The step of inputting the prompt word information into a large language model to generate a correction suggestion for the error message includes: According to the interface standard of the large language model, configure the interface parameters of the model calling interface; The large language model is called through the configured model calling interface, and the prompt word information is input into the large language model to generate error correction suggestions for the error information.

8. The method according to any one of claims 1 to 7, further comprising: In response to the big data platform reporting an error, determining that the error correction request is received; or In response to receiving a trigger signal of an error correction control, it is determined that the error correction request is received.

9. The method according to claim 1, wherein: The error information includes SQL syntax error information and / or task failure error information.

10. A big data platform error processing device based on a large language model, comprising: An information acquisition module, configured to acquire error information generated by the big data platform and a structured query language SQL statement associated with the error information in response to receiving an error correction request; A prompt word construction module, configured to construct prompt word information based on the error message and the SQL statement; The generation module is configured to input the prompt word information into a large language model to generate error correction suggestions for the error information.

11. The device according to claim 10, wherein: The prompt word building module includes: A first calling module is configured to call a target prompt word template according to the error message; The prompt word construction submodule is configured to construct prompt word information based on the target prompt word template, the error message and the SQL statement.

12. The device according to claim 11, wherein The target prompt word template in the first calling module is constructed based on the following steps: A sample acquisition unit, configured to acquire sample error information, as well as sample prompt words and sample error correction suggestions corresponding to the sample error information from a sample library; A generating unit, configured to generate an initial prompt word corresponding to the sample error message based on the initial prompt word template; A first updating unit, configured to update the initial prompt word template in response to a matching value between the initial prompt word and the sample prompt word being less than a first threshold; The template construction unit is configured to construct the target prompt word template based on the updated initial prompt word template.

13. The device according to claim 12, wherein: The template construction unit is further configured to: Based on the updated initial prompt word template, generating an updated prompt word corresponding to the sample error message; Inputting the updated prompt word information into a large language model to generate updated error correction suggestions; In response to the matching value between the updated error correction suggestion and the sample error correction suggestion being less than a second threshold, the updated initial prompt word template is updated again to obtain the target prompt word template.

14. The device according to claim 12, wherein: The updating of the initial prompt word template in the first updating unit includes: A detection unit is configured to perform semantic detection on the initial prompt word, wherein the semantic detection includes at least one of the following: logic detection, sentence detection, and typo detection; The second updating unit is configured to update the initial prompt word template according to the result of semantic detection.

15. The apparatus according to claim 12, further comprising: The storage unit is configured to associate the error reporting information, the prompt word information and the error correction suggestion and save them in the sample library.

16. The device according to claim 10, wherein: The generation module comprises: A parameter configuration module, configured to configure interface parameters of a model calling interface according to an interface standard of the large language model; The second calling module is configured to call the large language model through the configured model calling interface, and input the prompt word information into the large language model to generate error correction suggestions for the error information.

17. The device according to any one of claims 10 to 16, further comprising: A first response unit is configured to determine that the error correction request is received in response to an error reported by the big data platform; or The second response unit is configured to determine that the error correction request is received in response to receiving a trigger signal of the error correction control.

18. The device according to claim 10, wherein: The error information includes SQL syntax error information and / or task failure error information.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the big data platform error processing method based on a large language model as described in any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the big data platform error processing method based on a large language model according to any one of claims 1 to 9.

21. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the big data platform error processing method based on a large language model according to any one of claims 1 to 9.

Images