Data processing method based on natural language technology and related equipment thereof

Abstract

The invention belongs to the technical field of data acquisition, and relates to a data processing method based on a natural language technology and related equipment thereof. Carrying out decomposition processing on the analysis result in the natural language form; performing semantic understanding on a decomposition processing result, and determining an extraction intention; based on the extraction intention, constructing a data extraction service code at the target data extraction end; executing the data extraction service code, and extracting target data; sending the target data to a preset processing template, and performing data processing to obtain a data processing result; and feeding back a data processing result to an issuing end of the data extraction instruction. According to the method, the natural language technology is adopted, the extraction intention of the data extraction instruction is recognized, the service code self-generation mode is combined, the service codes are generated and executed, data extraction is carried out, full-link processing from semantic comprehension to code generation to task execution is automatically completed, and the data extraction efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of data acquisition technology, and is applied to scenarios where data is acquired from a target data storage terminal according to a data acquisition instruction. It relates to a data processing method based on natural language technology and related equipment. Background Technology

[0002] With the development of artificial intelligence and cloud computing, conversational interfaces have become a popular solution to the aforementioned problems. How to conveniently and quickly query structured data in databases using conversational natural language has become an emerging research hotspot. However, current research mostly focuses on querying single tables within databases, covering very few query keywords, typically only SELECT and WHERE clauses. This cannot meet the needs of real-world scenarios. In real-world scenarios, a user's query intent may not be covered by a single database or table; it often requires querying multiple tables, such as in data extraction, processing, and analysis in big data scenarios within financial businesses.

[0003] Currently, some products on the market have attempted to combine natural language processing with data processing, but they all have obvious limitations. For example, "Ping An Ai Code" has the ability to generate code from natural language, but it has not yet been deeply integrated with the data retrieval platform. Users still need to switch between multiple systems and cannot yet achieve a closed-loop experience of "data extraction + data processing + data analysis" based on a single business requirement. Therefore, how to achieve full-link data processing based on a single business requirement has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of this application is to propose a data processing method and related equipment based on natural language technology, so as to realize full-link data processing according to a sentence or a paragraph of business requirements.

[0005] Firstly, embodiments of this application provide a data processing method based on natural language technology, employing the following technical solution:

[0006] Data processing methods based on natural language processing technology include the following steps:

[0007] Receive data extraction instructions;

[0008] The data extraction instructions are parsed, and the parsing results in natural language form are decomposed.

[0009] Perform semantic understanding on the decomposition results to determine the extraction intent of the data extraction instructions;

[0010] Based on the extraction intent, data extraction business code is constructed at the target data extraction end;

[0011] Execute the data extraction business code to extract the target data;

[0012] The target data is sent to a preset processing template for data processing to obtain the data processing result;

[0013] The data processing results are fed back to the end that issued the data extraction command.

[0014] Secondly, embodiments of this application also provide a data processing apparatus based on natural language technology, which adopts the following technical solution:

[0015] Data processing devices based on natural language processing technology include:

[0016] The instruction receiving module is used to receive data extraction instructions;

[0017] The instruction processing module is used to parse the data extraction instruction and decompose the parsing results in natural language form.

[0018] The semantic understanding module is used to perform semantic understanding on the decomposition processing results and determine the extraction intent of the data extraction instruction;

[0019] The business code construction module is used to construct data extraction business code on the target data extraction end based on the extraction intent;

[0020] The target data extraction module is used to execute the data extraction business code and extract the target data;

[0021] The data processing module is used to send the target data to a preset processing template for data processing to obtain the data processing result;

[0022] The data feedback module is used to feed back the data processing results to the end that issues the data extraction command.

[0023] Thirdly, embodiments of this application also provide a computer device that adopts the technical solution described below:

[0024] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data processing method based on natural language technology described above.

[0025] Fourthly, embodiments of this application also provide a computer-readable storage medium, which adopts the technical solutions described below:

[0026] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the data processing method based on natural language technology as described above.

[0027] Compared with the prior art, the embodiments of this application have the following main advantages:

[0028] The data processing method based on natural language processing technology described in this application involves receiving and parsing data extraction instructions; decomposing the parsing results in natural language form; semantically understanding the decomposition results to determine the extraction intent; constructing data extraction business code at the target data extraction end based on the extraction intent; executing the data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the data processing result; and feeding back the data processing result to the end that issued the data extraction instruction. This application uses natural language processing technology to identify the extraction intent of the data extraction instruction and combines it with a self-generated business code method to generate and execute business code for data extraction. This achieves automatic completion of the entire processing chain from "semantic understanding" to "code generation" to "task execution." Especially in the scenario of financial business big data processing where business data is complex and massive, this method can automatically complete the entire data extraction chain, improving data extraction efficiency. Attached Figure Description

[0029] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0031] Figure 2 This is a flowchart of an embodiment of the data processing method based on natural language technology according to this application;

[0032] Figure 3 This is a flowchart of a specific embodiment of the data extraction and knowledge graph construction in the data processing method based on natural language technology described in this application;

[0033] Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 302 shown;

[0034] Figure 5This is a flowchart of a specific embodiment of the data processing method based on natural language technology described in this application for intercepting and processing sensitive fields;

[0035] Figure 6 yes Figure 3 A flowchart of a specific embodiment of step 303 shown;

[0036] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0037] Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 204 shown;

[0038] Figure 9 This is a schematic diagram of a structure of an embodiment of a data processing apparatus based on natural language technology according to this application;

[0039] Figure 10 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0041] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0042] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0043] like Figure 1As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0044] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0045] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptop computer 1011, tablet computer 1012 or mobile phone 1013, terminal device 101 can also be e-book reader, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer and desktop computer, etc.

[0046] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0047] It should be noted that the data processing method based on natural language technology provided in the embodiments of this application is generally executed by a server, and correspondingly, the data processing device based on natural language technology is generally located in the server.

[0048] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0049] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of a data processing method based on natural language processing technology according to this application. The data processing method based on natural language processing technology includes the following steps:

[0050] Step 201: Receive data extraction instructions.

[0051] In this embodiment, the data extraction instruction includes a data retrieval instruction in natural language form, such as: "Find critical illness insurance customers whose claims amount exceeded 100,000 in January 2025". This is a data retrieval instruction in natural language form entered by the target user in the data retrieval interface provided by the target data acquisition platform. The term "data retrieval" refers to the meaning of data acquisition and data extraction. The target data acquisition platform is, for example, a target database, a target data warehouse, or a target data lake.

[0052] Specifically, the target user enters a data extraction command in the data extraction platform interface, and the processing terminal receives the data extraction command. The target user includes users who input data and users who query data.

[0053] Step 202: Parse the data extraction instruction and decompose the parsing results in natural language form.

[0054] In this embodiment, parsing the data extraction instruction means obtaining specific instruction data. For example, by parsing the above data extraction instruction, the instruction data "find critical illness insurance customers whose claims amount exceeds 100,000 in January 2025" is obtained. The decomposition processing of the parsing results in natural language form means decomposing them into business time range, business filtering conditions, business object information, business type, etc., according to a preset decomposition processing strategy. Specifically, it decomposes information such as January 2025, claims business, exceeding 100,000, and critical illness insurance customers.

[0055] By acquiring and parsing the data extraction instructions, and decomposing the parsing results in natural language form, semantic understanding of the data extraction instructions can be achieved at the word level, and the extraction intent of the data extraction instructions can be identified at a lower dimension.

[0056] Step 203: Perform semantic understanding on the decomposition processing results to determine the extraction intent of the data extraction instruction.

[0057] In this embodiment, the semantic understanding of the decomposition processing results to determine the extraction intent of the data extraction instruction is achieved based on a pre-set business dictionary library, such as an insurance product dictionary, a field meaning library, and a claims rule library, thereby improving the accuracy of semantic understanding.

[0058] Step 204: Based on the extraction intent, construct the data extraction business code on the target data extraction end.

[0059] In this embodiment, the step of constructing data extraction business code on the target data extraction end based on the extraction intent is implemented by first constructing a general function for users to extract data from the database and setting the input parameters of the general function as variable fields; then, obtaining the specific parameters in the extraction intent, replacing the input parameters with the specific parameters, and compiling, thereby constructing the data extraction business code on the target data extraction end.

[0060] Step 205: Execute the data extraction business code to extract the target data.

[0061] By executing the data extraction business code, the target data is extracted, that is, the target data is extracted according to the extraction intent of the target user.

[0062] Step 206: Send the target data to the preset processing template for data processing to obtain the data processing result.

[0063] Specifically, the target data extracted above refers to the raw data extracted from the target database. However, in actual display or feedback, the target data is first sent to a preset processing template for data processing to obtain the data processing result. The processing template includes a data display template, which presets the fields to be filled in for each target data field, and also includes a fixed output format template, such as outputting the target data in JSON output format to the endpoint that issued the data extraction command. Here, the specific processing template is set according to actual business or display needs.

[0064] Step 207: Feed back the data processing result to the end that issued the data extraction instruction.

[0065] In this embodiment, the data extraction command issuing end is, for example, the user query end where the target user inputs the data extraction command.

[0066] In this embodiment, the process involves receiving and parsing a data extraction instruction; decomposing the parsing results in natural language form; semantically understanding the decomposition results to determine the extraction intent; constructing data extraction business code at the target data extraction end based on the extraction intent; executing the data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the data processing result; and feeding back the data processing result to the end that issued the data extraction instruction. This application employs natural language processing technology to identify the extraction intent of the data extraction instruction and combines it with a self-generated business code method to generate and execute business code for data extraction. This achieves automatic completion of the entire processing chain from "semantic understanding" to "code generation" to "task execution." Especially in scenarios involving complex and massive financial business big data processing, this method can automate the entire data extraction chain, improving data extraction efficiency.

[0067] Continue to refer to Figure 3 In some specific implementations, a step of data extraction and knowledge graph construction is included before step 203. Figure 3 This is a flowchart of a specific embodiment of the data extraction and knowledge graph construction method based on natural language processing technology described in this application, including:

[0068] Step 301: Collect historical real data extraction instructions;

[0069] Specifically, a data extraction knowledge graph is constructed by collecting batches of historical real data extraction commands and combining them with the real data extraction results.

[0070] Step 302: Parse the data extraction instruction and decompose the parsing results in natural language form;

[0071] Specifically, the data extraction instructions are parsed, and the parsing results in natural language form are decomposed to extract business time range, business filtering conditions, business object information, business type, etc.

[0072] Step 303: Input the decomposition processing results into the pre-built business domain knowledge base, and construct a data extraction knowledge graph based on the decomposition processing results.

[0073] Specifically, the decomposed business time range, business filtering conditions, business object information, business type, etc., are input into a pre-built business domain knowledge base. Combined with the business time range, business filtering conditions, business object information, and business type, a data extraction knowledge graph is constructed.

[0074] By combining historical data extraction instructions, a data extraction knowledge graph is constructed. This allows for the direct generation of data extraction business code based on the data extraction knowledge graph after parsing the actual data extraction instructions received in the future.

[0075] Continue to refer to Figure 4 , Figure 4 yes Figure 3 A flowchart of a specific embodiment of step 302 shown includes:

[0076] Step 401: Input the parsing result in the natural language form into the pre-built multi-stage task planning engine;

[0077] Specifically, the pre-built multi-stage task planning engine can identify the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result according to preset recognition rules or recognition strategies.

[0078] Step 402: Using the multi-stage task planning engine, identify the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result.

[0079] By identifying the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format contained in the parsing results, it is possible to generate multi-stage business code subsequently. For example, based on the task time range, business code is constructed to obtain data within the task time range as the first data to be obtained; then, based on the task filtering conditions, data is obtained again from the data obtained in the first time based on the task time range to obtain data that meets both the task time range and the task filtering conditions, which is used as the second data to be obtained. This process is repeated step by step according to different parsing result contents to obtain data in layers until the final data is obtained.

[0080] Continue to refer to Figure 5 In some specific implementations, after step 402, a step of intercepting and processing sensitive fields is also included. Figure 5 This is a flowchart of a specific embodiment of the data processing method based on natural language technology described in this application, which includes the interception and processing of sensitive fields.

[0081] Step 501: Based on the preset sensitive field interception mechanism, identify whether the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format have triggered the corresponding interception mechanism.

[0082] Step 502: If any of the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format triggers the corresponding interception mechanism, a prompt message is sent to the issuing end of the data extraction instruction, and the sensitive fields of the triggering item are desensitized.

[0083] By using a preset sensitive field interception mechanism, the system can identify whether the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format have triggered the corresponding interception mechanism. This allows for the rapid identification of sensitive fields entered by the user and the desensitization processing during transmission, ensuring the security of data transmission.

[0084] Continue to refer to Figure 6 , Figure 6 yes Figure 3 A flowchart of a specific embodiment of step 303 shown includes:

[0085] Step 601: Input the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result into the pre-built business domain knowledge base;

[0086] Step 602: Create a data extraction knowledge node based on the task entity information;

[0087] Step 603: Create connection constraints between nodes based on the task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format;

[0088] Step 604: Summarize the data extraction knowledge nodes and the connection constraints between the nodes to construct the data extraction knowledge graph.

[0089] By using task entity information, such as data form names and business entity names, as nodes, and defining the connection relationships between nodes based on task filtering conditions, task time ranges, task aggregation methods, task grouping dimensions, and task output formats, a data extraction knowledge graph is constructed. This facilitates the subsequent extraction of entity information from data acquisition instructions, combined with the parsed task filtering conditions, task time ranges, task aggregation methods, task grouping dimensions, and task output formats, to obtain the extraction path within the data extraction knowledge graph.

[0090] Continue to refer to Figure 7 , Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0091] Step 701: Input the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result into the pre-built business domain knowledge base;

[0092] Step 702: Identify the data extraction knowledge nodes that the task entity information is matched with in the data extraction knowledge graph, and use them as target knowledge nodes;

[0093] Step 703: Determine the connection constraint relationship between the target knowledge nodes based on the task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format;

[0094] Step 704: Based on the connection constraint relationship between the target knowledge node and the target knowledge node, determine the extraction intent of the data extraction instruction, wherein the extraction intent includes the extraction object and extraction constraint parameters, the extraction object includes the target knowledge node, and the extraction constraint parameters include at least one of the task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format.

[0095] Specifically, steps 701 to 704 involve combining the data extraction knowledge graph to identify the extraction intent of the data extraction command, thereby enabling faster semantic understanding of the data extraction command input by the target user.

[0096] Continue to refer to Figure 8 , Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0097] Step 801: Identify the extraction object and extraction limiting parameters contained in the extraction intent;

[0098] Step 802: Determine the extraction connection interface based on the data address information of the object to be extracted;

[0099] Step 803: Using the extraction connection interface as the first data retrieval parameter and the extraction limitation parameter as the second data retrieval parameter, match the preset data extraction service function;

[0100] Step 804: Input the first data retrieval parameter and the second data retrieval parameter into the preset data extraction service function, compile the code, and obtain the data extraction business code.

[0101] By pre-setting a general data acquisition function, and changing the corresponding data retrieval parameters only according to the actual business processing logic, new business code can be compiled and generated quickly, improving data extraction efficiency.

[0102] In this embodiment, before executing the step of feeding back the data processing result to the data extraction instruction issuing end, the method further includes: a mechanism for appending data acquisition instructions, i.e., determining whether a latest data extraction append instruction has been received; if a latest data extraction append instruction is received, the data processing result is cached first, and feedback is stopped; the online data acquisition instruction is replaced with the latest data extraction append instruction, and steps 202 to 203 are continued, i.e., parsing the latest data extraction append instruction and performing decomposition processing on the parsing result in natural language form; performing semantic understanding on the decomposition processing result to determine the extraction intent of the latest data extraction append instruction; then, summarizing the extraction intent of the previous data extraction instruction and the extraction intent of the latest data extraction append instruction to obtain a comprehensive extraction intent; based on the comprehensive extraction intent, constructing the latest data extraction business code at the target data extraction end; executing the latest data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the latest data processing result; and feeding back the latest data processing result to the data extraction instruction issuing end.

[0103] In this embodiment, the process involves receiving and parsing a data extraction instruction; decomposing the parsing results in natural language form; semantically understanding the decomposition results to determine the extraction intent; constructing data extraction business code at the target data extraction end based on the extraction intent; executing the data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the data processing result; and feeding back the data processing result to the end that issued the data extraction instruction. This application employs natural language processing technology to identify the extraction intent of the data extraction instruction and, combined with a self-generated business code method, generates and executes business code for data extraction, achieving automatic completion of the entire processing chain from "semantic understanding" to "code generation" to "task execution." Simultaneously, it supports instruction appending and sensitive field identification mechanisms, ensuring data extraction while maintaining data transmission security. Especially in scenarios involving complex and massive financial business big data processing, this method can automate the entire data extraction chain, improving data extraction efficiency.

[0104] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0105] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0106] In this embodiment, the process involves receiving and parsing a data extraction instruction; decomposing the parsing results in natural language form; semantically understanding the decomposition results to determine the extraction intent; constructing data extraction business code at the target data extraction end based on the extraction intent; executing the data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the data processing result; and feeding back the data processing result to the end that issued the data extraction instruction. This application employs natural language processing technology to identify the extraction intent of the data extraction instruction and, combined with a self-generated business code method, generates and executes business code for data extraction, achieving automatic completion of the entire processing chain from "semantic understanding" to "code generation" to "task execution." Simultaneously, it supports instruction appending and sensitive field identification mechanisms, ensuring data extraction while maintaining data transmission security. Especially in scenarios involving complex and massive financial business big data processing, this method can automate the entire data extraction chain, improving data extraction efficiency.

[0107] Further reference Figure 9 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data processing device based on natural language technology, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0108] like Figure 9 As shown, the data processing device 900 based on natural language technology described in this embodiment includes: an instruction receiving module 901, an instruction processing module 902, a semantic understanding module 903, a business code construction module 904, a target data extraction module 905, a data processing module 906, and a data feedback module 907. Wherein:

[0109] Instruction receiving module 901 is used to receive data extraction instructions;

[0110] The instruction processing module 902 is used to parse the data extraction instruction and decompose the parsing results in natural language form;

[0111] The semantic understanding module 903 is used to perform semantic understanding on the decomposition processing results and determine the extraction intent of the data extraction instruction;

[0112] The business code construction module 904 is used to construct data extraction business code at the target data extraction end based on the extraction intent;

[0113] The target data extraction module 905 is used to execute the data extraction business code to extract the target data;

[0114] The data processing module 906 is used to send the target data to a preset processing template for data processing to obtain the data processing result;

[0115] The data feedback module 907 is used to feed back the data processing results to the end that issues the data extraction command.

[0116] This application achieves automated end-to-end processing from semantic understanding to code generation to task execution by receiving and parsing data extraction instructions; decomposing the parsing results in natural language form; semantically understanding the decomposition results to determine the extraction intent; constructing data extraction business code at the target data extraction end based on the extraction intent; executing the data extraction business code to extract the target data; sending the target data to a preset processing template for data processing to obtain the data processing result; and feeding back the data processing result to the end that issued the data extraction instruction. This application employs natural language processing technology to identify the extraction intent of the data extraction instruction and, combined with a self-generated business code method, generates and executes business code to extract data, thus achieving automated end-to-end processing from semantic understanding to code generation to task execution. Simultaneously, it supports instruction appending and sensitive field identification mechanisms to ensure data extraction security. Especially in the context of financial business big data processing, where business data is complex and massive, this method can automate the entire data extraction process, improving data extraction efficiency.

[0117] In this embodiment, the data processing device 900 based on natural language technology further includes a historical data extraction instruction acquisition module and a data extraction knowledge graph construction module. Wherein:

[0118] The historical data extraction instruction acquisition module is used to acquire historical real data extraction instructions;

[0119] The instruction processing module 902 is also used to parse the data extraction instruction and decompose the parsing results in natural language form;

[0120] The data extraction knowledge graph construction module is used to input the decomposition processing results into a pre-built business domain knowledge base, and construct a data extraction knowledge graph based on the decomposition processing results.

[0121] In this embodiment, the instruction processing module 902 further includes an instruction parsing unit, a parsing result input unit, and a parsing result recognition unit. Wherein:

[0122] The instruction parsing unit is used to parse the data extraction instruction to obtain the parsing result in natural language form;

[0123] The parsing result input unit is used to input the parsing result in natural language form into a pre-built multi-stage task planning engine;

[0124] The parsing result recognition unit is used to identify, using the multi-stage task planning engine, the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result.

[0125] In this embodiment, the data processing device 900 based on natural language technology further includes a sensitive field interception trigger module and a desensitization processing module. Wherein:

[0126] The sensitive field interception trigger module is used to identify whether the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format have triggered the corresponding interception mechanism according to the preset sensitive field interception mechanism.

[0127] The desensitization processing module is used to send a prompt message to the data extraction instruction issuing end and perform sensitive field desensitization processing on the triggering item if any of the following triggers the corresponding interception mechanism: task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format.

[0128] In this embodiment, the data extraction knowledge graph construction module includes: a content input unit, a knowledge node creation unit, a connection constraint relationship creation unit, and a data extraction knowledge graph construction unit. Wherein:

[0129] The parsing content input unit is used to input the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result into the pre-built business domain knowledge base;

[0130] The knowledge node creation unit is used to create data extraction knowledge nodes based on the task entity information.

[0131] The connection constraint relationship creation unit is used to create connection constraint relationships between nodes based on the task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format.

[0132] The data extraction knowledge graph construction unit is used to summarize the data extraction knowledge nodes and the connection constraints between the nodes to construct the data extraction knowledge graph.

[0133] In this embodiment, the semantic understanding module 903 includes: a content parsing input unit, a target knowledge node identification unit, a connection constraint relationship determination unit, and an intent extraction determination unit. Wherein:

[0134] The parsing content input unit is used to input the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing result into the pre-built business domain knowledge base;

[0135] The target knowledge node identification unit is used to identify the data extraction knowledge node that the task entity information hits in the data extraction knowledge graph as the target knowledge node;

[0136] The connection constraint determination unit is used to determine the connection constraint relationship between the target knowledge nodes based on the task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format.

[0137] The extraction intent determination unit is used to determine the extraction intent of the data extraction instruction based on the target knowledge node and the connection constraint relationship between the target knowledge nodes, wherein the extraction intent includes the extraction object and extraction constraint parameters.

[0138] In this embodiment, the business code construction module 904 includes a parameter extraction and identification unit, a connection interface extraction and determination unit, a service function extraction and matching unit, and a code compilation and acquisition unit. Wherein:

[0139] The extraction parameter identification unit is used to identify the extraction object and extraction limiting parameters contained in the extraction intent;

[0140] The connection interface extraction determination unit is used to determine the connection interface extraction based on the data address information of the extraction object.

[0141] The service function matching unit is used to match a preset data extraction service function with the extraction connection interface as the first data retrieval parameter and the extraction limitation parameter as the second data retrieval parameter.

[0142] The code compilation and acquisition unit is used to input the first data retrieval parameter and the second data retrieval parameter into the preset data extraction service function, perform code compilation, and obtain the data extraction business code.

[0143] In this embodiment, the data processing device 900 based on natural language technology further includes a data extraction append instruction judgment module, a pause feedback control module, and an extraction intent summary module. Wherein:

[0144] The data extraction append instruction judgment module is used to determine whether the latest data extraction append instruction has been received;

[0145] The pause feedback control module is used to cache the data processing result and stop feedback if a new data extraction and append instruction is received.

[0146] The extraction intent summary module is used to summarize the extraction intents of previous data extraction instructions and the extraction intents of the latest data extraction append instruction to obtain a comprehensive extraction intent.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0148] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0149] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a basic structural block diagram of the computer device in this embodiment.

[0150] The computer device 10 includes a memory 10a, a processor 10b, and a network interface 10c, which are interconnected via a system bus. It should be noted that... Figure 10Only a computer device 10 with component memory 10a, processor 10b, and network interface 10c is shown. However, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0151] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0152] The memory 10a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10a may be an internal storage unit of the computer device 10, such as the hard disk or memory of the computer device 10. In other embodiments, the memory 10a may also be an external storage device of the computer device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 10a may include both internal storage units and external storage devices of the computer device 10. In this embodiment, the memory 10a is typically used to store the operating system and various application software installed on the computer device 10, such as computer-readable instructions for data processing methods based on natural language technology. In addition, the memory 10a can also be used to temporarily store various types of data that have been output or will be output.

[0153] In some embodiments, the processor 10b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 10b is typically used to control the overall operation of the computer device 10. In this embodiment, the processor 10b is used to execute computer-readable instructions stored in the memory 10a or to process data, for example, to execute computer-readable instructions of the data processing method based on natural language processing technology.

[0154] The network interface 10c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 10 and other electronic devices.

[0155] The computer device proposed in this embodiment belongs to the field of data acquisition technology and is applied in scenarios where data is retrieved from a target data storage terminal according to a data acquisition instruction. This application receives and parses a data extraction instruction; decomposes the parsing result in natural language form; performs semantic understanding on the decomposition result to determine the extraction intent; constructs data extraction business code on the target data extraction terminal based on the extraction intent; executes the data extraction business code to extract the target data; sends the target data to a preset processing template for data processing to obtain the data processing result; and feeds back the data processing result to the terminal that issued the data extraction instruction. This application uses natural language technology to identify the extraction intent of the data extraction instruction and, combined with a self-generated business code method, generates and executes business code to perform data extraction, achieving automatic completion of the entire chain from "semantic understanding" to "code generation" to "task execution." Simultaneously, it supports instruction appending and sensitive field identification mechanisms to perform data extraction while ensuring data transmission security. Especially in scenarios involving complex and massive financial business big data processing, this method can automate the entire data extraction chain, improving data extraction efficiency.

[0156] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the data processing method based on natural language technology as described above.

[0157] The computer-readable storage medium proposed in this embodiment belongs to the field of data acquisition technology and is applied to scenarios where data is retrieved from a target data storage terminal according to a data acquisition instruction. This application receives and parses a data extraction instruction; decomposes the parsing result in natural language form; performs semantic understanding on the decomposition result to determine the extraction intent; constructs data extraction business code on the target data extraction terminal based on the extraction intent; executes the data extraction business code to extract the target data; sends the target data to a preset processing template for data processing to obtain the data processing result; and feeds back the data processing result to the terminal that issued the data extraction instruction. This application uses natural language technology to identify the extraction intent of the data extraction instruction and, combined with a self-generated business code method, generates and executes business code to perform data extraction, achieving automatic completion of the entire chain from "semantic understanding" to "code generation" to "task execution." Simultaneously, it supports instruction appending and sensitive field identification mechanisms to perform data extraction while ensuring data transmission security. Especially in the scenario of financial business big data processing where business data is complex and massive, this method can automatically complete the entire chain of data extraction, improving data extraction efficiency.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0159] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to make the disclosure of this application more thorough and comprehensive. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application. Software tools or components not belonging to this company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

Claims

1. A data processing method based on natural language processing technology, characterized in that, Includes the following steps: Receive data extraction instructions; The data extraction instructions are parsed, and the parsing results in natural language form are decomposed. Perform semantic understanding on the decomposition results to determine the extraction intent of the data extraction instructions; Based on the extraction intent, data extraction business code is constructed at the target data extraction end; Execute the data extraction business code to extract the target data; The target data is sent to a preset processing template for data processing to obtain the data processing result; The data processing results are fed back to the end that issued the data extraction command.

2. The data processing method based on natural language processing technology according to claim 1, characterized in that, Before performing the step of semantic understanding of the decomposition processing results to determine the extraction intent of the data extraction instruction, the method further includes: Commands to extract historical, authentic data; The data extraction instructions are parsed, and the parsing results in natural language form are decomposed. The decomposition results are input into a pre-built business domain knowledge base, and a data extraction knowledge graph is constructed based on the decomposition results.

3. The data processing method based on natural language processing technology according to claim 1 or 2, characterized in that, The steps for decomposing the parsing results of natural language forms specifically include: The parsing results in the natural language form are input into a pre-built multi-stage task planning engine; Using the multi-stage task planning engine, the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing results are identified.

4. The data processing method based on natural language technology according to claim 3, characterized in that, After performing the step of using the multi-stage task planning engine to identify the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format contained in the parsing result, the method further includes: Based on the preset sensitive field interception mechanism, identify whether the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format have triggered the corresponding interception mechanism; If any of the task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format triggers the corresponding interception mechanism, a prompt message is sent to the issuing end of the data extraction instruction, and the sensitive fields of the triggering item are desensitized.

5. The data processing method based on natural language processing technology according to claim 2, characterized in that, The steps of inputting the decomposition results into a pre-built business domain knowledge base and constructing a data extraction knowledge graph based on the decomposition results specifically include: The task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing results are input into the pre-built business domain knowledge base; Data extraction knowledge nodes are created based on the task entity information; Create connection constraints between nodes based on the task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format; By summarizing the data extraction knowledge nodes and the connection constraints between the nodes, the data extraction knowledge graph is constructed.

6. The data processing method based on natural language technology according to claim 2, characterized in that, The step of semantically understanding the decomposition results and determining the extraction intent of the data extraction instruction specifically includes: The task entity information, task filtering conditions, task time range, task aggregation method, task grouping dimension and task output format contained in the parsing results are input into the pre-built business domain knowledge base; Identify the data extraction knowledge nodes in the data extraction knowledge graph that the task entity information is matched with, and use them as target knowledge nodes; Based on the task filtering conditions, task time range, task aggregation method, task grouping dimension, and task output format, determine the connection constraint relationship between the target knowledge nodes; Based on the target knowledge node and the connection constraint relationship between the target knowledge nodes, the extraction intent of the data extraction instruction is determined, wherein the extraction intent includes the extraction object and extraction constraint parameters.

7. The data processing method based on natural language processing technology according to any one of claims 1 or 6, characterized in that, The step of constructing data extraction business code on the target data extraction end based on the extraction intent specifically includes: Identify the extraction object and extraction limiting parameters contained in the extraction intent; The extraction connection interface is determined based on the data address information of the object to be extracted; Using the extraction connection interface as the first data retrieval parameter and the extraction limitation parameter as the second data retrieval parameter, a preset data extraction service function is matched; The first data retrieval parameter and the second data retrieval parameter are input into the preset data extraction service function, and the code is compiled to obtain the data extraction business code.

8. A data processing device based on natural language processing technology, characterized in that, include: The instruction receiving module is used to receive data extraction instructions; The instruction processing module is used to parse the data extraction instruction and decompose the parsing results in natural language form. The semantic understanding module is used to perform semantic understanding on the decomposition processing results and determine the extraction intent of the data extraction instruction; The business code construction module is used to construct data extraction business code on the target data extraction end based on the extraction intent; The target data extraction module is used to execute the data extraction business code and extract the target data; The data processing module is used to send the target data to a preset processing template for data processing to obtain the data processing result; The data feedback module is used to feed back the data processing results to the end that issues the data extraction command.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data processing method based on natural language technology as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data processing method based on natural language technology as described in any one of claims 1 to 7.

Images