Data processing method and device and computer equipment
By preprocessing the problem data to be processed and structured searching in the knowledge graph, and combining with the big model to generate response data, the problem of low Q&A efficiency in the existing technology is solved, and efficient and accurate knowledge resource utilization is achieved.
Patent Information
- Application Number
- CN202411725988.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-02-11
AI Technical Summary
When the existing technology deals with complex and diverse problems, the Q&A is inefficient and cannot effectively utilize the knowledge resources of the enterprise.
By preprocessing the received pending problem data, identifying key entities and query intents, and searching in a structured way in the pre-constructed knowledge graph, combining search results and pending problem data generation to adapt to pre-training input data of the large model to obtain response data.
Improve the efficiency of question and answers, accurately understand the pending problem data entered by users, quickly extract useful data from a large amount of data, ensure the accuracy of the response data, and reduce manual intervention and processing time.
Smart Images
Figure CN119415662A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product. Background Art
[0002] At present, the informatization and dataization of various enterprises are constantly developing, and a large amount of knowledge resources have been accumulated within the enterprises. Therefore, how to efficiently manage and utilize the above knowledge resources to achieve fast and accurate question-and-answer services has become a problem to be solved.
[0003] However, the existing question-answering methods for knowledge resources usually rely on keyword matching or manually set rules, focus on a single retrieval or reasoning mechanism, and are only suitable for simple data question-answering. They cannot handle complex and diverse questions in a timely manner, which directly affects the utilization of data resources. Therefore, there is currently a problem of low question-answering efficiency. Summary of the invention
[0004] Based on this, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium and computer program product to address the technical problem of low question-answering efficiency.
[0005] In a first aspect, the present application provides a data processing method, comprising:
[0006] Preprocessing the received question data to be processed to identify key entities and query intent in the question data to be processed;
[0007] According to the key entity and the query intent, a search is performed in a structured manner in a pre-constructed knowledge graph to obtain a search result; the knowledge graph is constructed based on dynamically updated enterprise private domain data;
[0008] Based on the retrieval results and the question data to be processed, input data adapted to the pre-trained large model is generated to obtain answer data for the question data to be processed through the large model.
[0009] In one of the embodiments, the preprocessing of the received question data to be processed includes: using natural language processing technology to perform sentence segmentation, word segmentation and part-of-speech tagging on the question data to be processed, and based on the results of sentence segmentation, word segmentation and part-of-speech tagging, identifying key entities and query intent in the question data to be processed.
[0010] In one of the embodiments, the knowledge graph is constructed in the following manner: extracting the enterprise private domain data; constructing a knowledge graph based on the extracted enterprise private domain data; wherein the knowledge graph is composed of knowledge triples, each knowledge triplet containing entities, attributes and relationships; and, when it is detected that the enterprise private domain data is updated, extracting the updated enterprise private domain data, and updating the knowledge graph based on the updated enterprise private domain data.
[0011] In one of the embodiments, the retrieval is performed in a structured manner in a pre-constructed knowledge graph according to the key entity and the query intent to obtain the retrieval result, including: constructing a query template for the problem data to be processed according to the key entity and the query intent; sending the query template to the pre-constructed knowledge graph for retrieval; matching multiple groups of the knowledge triples related to the query template in the pre-constructed knowledge graph, and using the multiple groups of the knowledge triples as the retrieval result; wherein the multiple groups of the knowledge triples are sorted from high to low according to the relevance to the query template.
[0012] In one of the embodiments, the input data suitable for the pre-trained large model is generated based on the retrieval results and the problem data to be processed, including: combining the retrieval results with the problem data to be processed to obtain preliminary input data; and re-processing the preliminary input data to obtain the input data suitable for the pre-trained large model.
[0013] In one of the embodiments, after obtaining the response data of the problem data to be processed through the large model, it also includes: extracting the response data; post-processing the response data to obtain final response data, and sending the final response data to the user terminal; the post-processing includes data analysis, data cleaning and data reorganization.
[0014] In a second aspect, the present application further provides a data processing device, comprising:
[0015] A data identification module is used to pre-process the received question data to be processed so as to identify key entities and query intent in the question data to be processed;
[0016] A retrieval module, used to search in a structured manner in a pre-built knowledge graph according to the key entity and the query intent to obtain a retrieval result; the knowledge graph is built based on dynamically updated enterprise private domain data;
[0017] An input data generation module is used to generate input data adapted to a pre-trained large model based on the retrieval results and the problem data to be processed, so as to obtain response data of the problem data to be processed through the large model.
[0018] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:
[0019] Preprocessing the received question data to be processed to identify key entities and query intent in the question data to be processed;
[0020] According to the key entity and the query intent, a search is performed in a structured manner in a pre-constructed knowledge graph to obtain a search result; the knowledge graph is constructed based on dynamically updated enterprise private domain data;
[0021] Based on the retrieval results and the question data to be processed, input data adapted to the pre-trained large model is generated to obtain answer data for the question data to be processed through the large model.
[0022] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0023] Preprocessing the received question data to be processed to identify key entities and query intent in the question data to be processed;
[0024] According to the key entity and the query intent, a search is performed in a structured manner in a pre-constructed knowledge graph to obtain a search result; the knowledge graph is constructed based on dynamically updated enterprise private domain data;
[0025] Based on the retrieval results and the question data to be processed, input data adapted to the pre-trained large model is generated to obtain answer data for the question data to be processed through the large model.
[0026] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, which implements the following steps when executed by a processor:
[0027] Preprocessing the received question data to be processed to identify key entities and query intent in the question data to be processed;
[0028] According to the key entity and the query intent, a search is performed in a structured manner in a pre-constructed knowledge graph to obtain a search result; the knowledge graph is constructed based on dynamically updated enterprise private domain data;
[0029] Based on the retrieval results and the question data to be processed, input data adapted to the pre-trained large model is generated to obtain answer data for the question data to be processed through the large model.
[0030] The above data processing method, device, computer equipment, storage medium and computer program product have the following beneficial effects in the process of data processing: first, the received problem data to be processed is preprocessed to identify the key entities and query intent in the problem data to be processed; then, according to the key entities and query intent, a search is performed in a structured manner in the pre-constructed knowledge graph to obtain the search results; the knowledge graph is constructed based on the dynamically updated enterprise private domain data; finally, based on the search results and the problem data to be processed, input data adapted to the pre-trained large model is generated to obtain the answer data of the problem data to be processed through the large model. In the above process, by identifying the key entities and query intent, the problem data to be processed input by the user can be accurately understood; based on the key entities and query intent, a search is carried out in a structured manner in the knowledge graph including the continuously updated enterprise private domain data, and data useful for answering the problem data to be processed can be quickly extracted from a large amount of data. In addition, the knowledge graph is continuously updated with the continuously updated enterprise private domain data, which ensures the accuracy of the answer data; therefore, the above process reduces manual intervention, saves the processing time of the problem data to be processed, and improves the efficiency of question answering. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0032] Figure 1 An application environment diagram of a data processing method in an embodiment;
[0033] Figure 2 is a flow chart of a data processing method in one embodiment;
[0034] Figure 3 A schematic diagram of a flow chart of a data processing step in an embodiment;
[0035] Figure 4 is a structural block diagram of a data processing device in one embodiment;
[0036] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0038] The data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the user terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. First, the server 104 pre-processes the problem data to be processed received from the user terminal 102 to identify the key entities and query intent in the problem data to be processed; then, the server 104 searches in a structured manner in the pre-constructed knowledge graph according to the key entities and query intent to obtain the search results; the knowledge graph includes the continuously updated enterprise private domain data; finally, the server 104 generates input data adapted to the pre-trained large model based on the search results and the problem data to be processed, so that the pre-trained large model outputs the answer data of the problem data to be processed. Among them, the user terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.
[0039] In an exemplary embodiment, Figure 2 As shown, a data processing method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S206. Among them:
[0040] Step S202: pre-process the received question data to be processed to identify key entities and query intent in the question data to be processed.
[0041] Among them, key entities refer to the important entities in the problem to be processed; query intent refers to the specific needs referred to by the problem to be processed.
[0042] As an example, preprocessing may be data cleaning and formatting of the problem data to be processed; key entities may include user information, time, and location of the problem input; and query intent may be retrieving information or requesting help.
[0043] Step S204, search in a structured manner in the pre-constructed knowledge graph according to the key entities and query intent to obtain search results; the knowledge graph is constructed based on dynamically updated enterprise private domain data.
[0044] Among them, the knowledge graph is a database that stores knowledge in the form of a graph structure; the pre-built knowledge graph is a dynamic database containing the company's private domain data; the company's private domain data refers to the data information related to customers, products and operations collected within the company, which is usually not disclosed and is the company's core resource; the structured approach refers to organizing data in a systematic and orderly form with a clear data format; the retrieval results can include relevant information related to the problem to be processed and available solutions.
[0045] As an example, the data format of structured data may include a table, a database or other ordered data structures, and the relationship between each data has a clear definition.
[0046] Step S206, based on the retrieval results and the question data to be processed, generate input data adapted to the pre-trained large model to obtain response data for the question data to be processed through the large model.
[0047] The input data may be structured data or unstructured data, depending on the input requirements of the pre-trained large model.
[0048] In the above data processing method, first, the received problem data to be processed is preprocessed to identify the key entities and query intent in the problem data to be processed; then, according to the key entities and query intent, a search is performed in a structured manner in the pre-constructed knowledge graph to obtain the search results; the knowledge graph is constructed based on the dynamically updated enterprise private domain data; finally, based on the search results and the problem data to be processed, input data adapted to the pre-trained large model is generated to obtain the answer data of the problem data to be processed through the large model. In the above process, by identifying the key entities and query intent, the problem data to be processed input by the user can be accurately understood; based on the key entities and query intent, a search is carried out in a structured manner in the knowledge graph including the continuously updated enterprise private domain data, which can quickly extract data useful for answering the problem data to be processed from a large amount of data. In addition, the knowledge graph is continuously updated with the continuously updated enterprise private domain data, which ensures the accuracy of the answer data; therefore, the above process reduces manual intervention, saves the processing time of the problem data to be processed, and improves the efficiency of question answering.
[0049] In an exemplary embodiment, the received question data to be processed is preprocessed, including: using natural language processing technology to perform sentence segmentation, word segmentation and part-of-speech tagging on the question data to be processed, and based on the results of sentence segmentation, word segmentation and part-of-speech tagging, identifying key entities and query intent in the question data to be processed.
[0050] Among them, sentence segmentation refers to dividing a text into independent sentences; word segmentation refers to breaking down a sentence into separate words; part-of-speech tagging refers to marking the part of speech for each word in the text, which can include nouns, verbs, adjectives, etc.
[0051] As an example, sentence segmentation usually identifies the end of a sentence by punctuation marks; word segmentation can be based on dictionary segmentation, statistical learning methods, or deep learning models to segment words.
[0052] In this embodiment, sentence segmentation, word segmentation and part-of-speech tagging can make the structure of the sentence clearer, eliminate possible ambiguity in the problem data to be processed, improve the data quality of the problem data to be processed, and make it easier to extract keywords, which facilitates the subsequent understanding of the problem data to be processed and improves the overall data processing efficiency.
[0053] In one embodiment, the knowledge graph is constructed in the following manner: extracting enterprise private domain data; constructing a knowledge graph based on the extracted enterprise private domain data; wherein the knowledge graph is composed of knowledge triples, each knowledge triplet contains entities, attributes and relationships; and, when it is detected that the enterprise private domain data is updated, extracting the updated enterprise private domain data, and updating the knowledge graph based on the updated enterprise private domain data.
[0054] Among them, extraction refers to obtaining specific information fragments from a large amount of raw data. Commonly used technologies include text mining, data crawling, etc.; knowledge triples are the basic units that constitute the knowledge graph; entities refer to specific objects or concepts, attributes refer to the characteristics of entities or descriptions of entities, and relationships refer to the connections between entities.
[0055] As an example, building a knowledge graph can be to integrate the extracted enterprise private domain data into a visual graph structure, creating a framework that connects entities, attributes, and relationships, which usually includes steps such as data modeling, data relationship confirmation, and data fusion of the enterprise private domain data.
[0056] In this embodiment, by extracting the enterprise's private domain data, the knowledge graph can integrate various scattered data resources into a unified knowledge base, and directly and clearly represent the relationship between various data in the form of a graph; in addition, the knowledge graph can help provide a reliable data foundation, so as to obtain the response data for the problem data to be processed more quickly, thereby improving the efficiency of the question-and-answer process.
[0057] Furthermore, in one embodiment, a search is performed in a structured manner in a pre-constructed knowledge graph according to key entities and query intent to obtain search results, including: constructing a query template for the problem data to be processed according to the key entities and query intent; sending the query template to the pre-constructed knowledge graph for retrieval; matching multiple groups of knowledge triples related to the query template in the pre-constructed knowledge graph, and using the multiple groups of knowledge triples as search results; wherein the multiple groups of knowledge triples are sorted from high to low according to their relevance to the query template.
[0058] Among them, the query template is a structured query form, which is constructed based on key entities and query intent to form a standardized query statement; relevance sorting is the process of prioritizing the results based on the degree of match between the retrieval results and the query template, usually sorted according to similarity scores, highly correlated attributes or relationships, to ensure that users get the most relevant information.
[0059] In this embodiment, through templated query, consistency and standardization can be guaranteed, which facilitates subsequent retrieval of the knowledge graph; by sorting the relevance of multiple groups of knowledge triples, users can find the required information more quickly, thereby improving the user experience.
[0060] Furthermore, in one embodiment, based on the retrieval results and the problem data to be processed, input data suitable for the pre-trained large model is generated, including: combining the retrieval results with the problem data to be processed to obtain preliminary input data; and reprocessing the preliminary input data to obtain input data suitable for the pre-trained large model.
[0061] Among them, preliminary input data refers to the data set after combining the search results with the problem data to be processed, which includes the original query intent and relevant information retrieved from the knowledge graph to form a preliminary information set; reprocessing refers to re-analyzing and processing the preliminary input data to optimize the information format and content to adapt it to the requirements of the pre-trained large model. The specific reprocessing method may include data cleaning, formatting, feature extraction and other steps.
[0062] In this embodiment, by combining the retrieval data and the question data to be processed, the user query and the relevant information in the knowledge graph can be effectively integrated to obtain more comprehensive preliminary input data information; by reprocessing the preliminary input data into input data suitable for the pre-trained large model, the function of the pre-trained large model can be better utilized, so that the answer data can be output more quickly and the question and answer efficiency can be improved.
[0063] In one embodiment, after obtaining the response data of the problem data to be processed through the large model, it also includes: extracting the response data; post-processing the response data to obtain the final response data, and sending the final response data to the user terminal; the post-processing includes data analysis, data cleaning and data reorganization.
[0064] Among them, response data refers to the preliminary response data generated based on the knowledge graph and the pre-trained large model; post-processing refers to further processing of the response data; and the final response data refers to the data obtained after post-processing.
[0065] As an example, the response data may be in the form of text, structured information or knowledge triples; post-processing may include data parsing: the process of converting the response data into a structured format that is easier to process, usually involving text analysis and format conversion, data cleaning: the process of checking the response data for errors, deleting redundant information and normalizing the format, data reorganization: the process of reorganizing the cleaned data according to a certain format or structure to obtain data that meets the requirements, etc.; the final response data may be in the form of natural language, structured lists or graphical information, etc.
[0066] In this embodiment, by post-processing the response data, the quality of the final response data sent to the user terminal can be guaranteed, and the reliability of the final response data is improved; in addition, the final response data can adapt to different types of user terminals, thereby improving the user experience and the interaction efficiency.
[0067] This application provides a data processing method. In order to better understand the process of the above data processing method, combined with Figure 3 As shown, the specific process of the data processing method of this application is described in detail below, including the following steps:
[0068] Step S302, constructing a knowledge graph based on the enterprise's private domain data.
[0069] The entities, relationships, and attributes that construct the knowledge graph can be manually or automatically extracted from the enterprise's private domain data.
[0070] Specifically, it includes a knowledge graph module, which is used to construct and store knowledge graphs, receive enterprise private domain data, extract entities, relationships and attributes, construct knowledge graphs, and store them in appropriate storage technologies.
[0071] Step S304, receiving the user's problem data to be processed, pre-processing the problem data to be processed, and obtaining input data.
[0072] Among them, users can input natural language questions through terminals such as mobile phones and computers; after the server receives the user's question (question data to be processed), it performs preprocessing, including word segmentation, part-of-speech tagging, etc., to identify key entities and query intent; according to the key entities and query intent in the user's question, the server retrieves relevant information in the knowledge graph, and extracts knowledge triples, entities and relationships that are directly or potentially related to the question; the server combines the retrieved knowledge with the user's question to form structured or unstructured input data, and may perform further processing or formatting to meet the input requirements of the large model.
[0073] Specifically, it includes a question parsing module, which is used to parse user questions, receive user questions, perform preprocessing, and identify key entities and query intentions; it also includes a knowledge graph retrieval module, which is used to retrieve relevant information in the knowledge graph, and retrieve relevant knowledge triples, entities and relationships in the knowledge graph based on the output of the question parsing module; and it includes a large model input preparation module, which is used to prepare large model input data, combine the output of the knowledge graph retrieval module with user questions, and form structured or unstructured input data.
[0074] Step S306, input the input data into the pre-trained large model to obtain response data.
[0075] The server inputs the prepared input data into the pre-trained large model, and the pre-trained large model generates response data based on the input data.
[0076] Specifically, it includes a pre-trained large model knowledge question and answer module, which is used to generate response data, receive the output of the pre-trained large model input preparation module, and generate answers.
[0077] Step S308: extract the response data and perform post-processing, and send the final response data obtained by the post-processing to the user terminal.
[0078] The server extracts the response data from the output of the pre-trained large model, performs post-processing such as deduplication, sorting, formatting, etc., and presents it to the user through the terminal.
[0079] Specifically, it includes an answer generation and output module, which is used to process and output the final answer data, extract the answer data from the output of the large model knowledge question and answer module, perform post-processing, and present it to the user through the terminal; it also includes a feedback and optimization module, which is used to collect user feedback and optimize the question and answer system composed of the above modules, collect user feedback on the answers, and adjust the parameters of the knowledge graph, retrieval strategy or pre-trained large model according to the feedback.
[0080] More importantly, the above modules are executed in sequence according to the order of the process, and together realize the large-model enterprise private domain knowledge question and answer method based on the knowledge graph.
[0081] In the above embodiment, by combining the precise retrieval capability of the knowledge graph and the extensive knowledge coverage and deep reasoning capability of the pre-trained large model, it is possible to more accurately understand user questions and quickly retrieve relevant information in the knowledge graph, and then, after reasoning and generation by the pre-trained large model, finally give accurate and detailed answers, showing higher accuracy and efficiency when dealing with complex, ambiguous or cross-domain problems; in addition, by providing a graphical and structured knowledge display method, it can help users better understand the relationship and connection between knowledge, thereby improving the user's cognitive experience and satisfaction; further, the construction of the knowledge graph is an ongoing process, and with the increase and update of the enterprise's private domain data, the method can continuously absorb new knowledge and maintain timeliness and accuracy; at the same time, the pre-trained large model also has good scalability, and can be adapted to different application scenarios and needs through fine-tuning or training, further enhancing flexibility.
[0082] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0083] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.
[0084] In an exemplary embodiment, Figure 4 As shown, a data processing device is provided, including: a data identification module 401, a retrieval module 402 and an input data generation module 403, wherein:
[0085] The data identification module 401 is used to pre-process the received question data to be processed, so as to identify key entities and query intent in the question data to be processed;
[0086] The retrieval module 402 is used to search in a structured manner in a pre-built knowledge graph according to key entities and query intent to obtain retrieval results; the knowledge graph is constructed based on dynamically updated enterprise private domain data;
[0087] The input data generation module 403 is used to generate input data adapted to the pre-trained large model based on the retrieval results and the problem data to be processed, so as to obtain the response data of the problem data to be processed through the large model.
[0088] Furthermore, in one embodiment, the data identification module 401 is also used to use natural language processing technology to perform sentence segmentation, word segmentation and part-of-speech tagging on the problem data to be processed, and based on the results of sentence segmentation, word segmentation and part-of-speech tagging, identify key entities and query intent in the problem data to be processed.
[0089] Furthermore, in one embodiment, the retrieval module 402 is also used to extract enterprise private domain data; construct a knowledge graph based on the extracted enterprise private domain data; wherein the knowledge graph is composed of knowledge triples, each knowledge triplet contains entities, attributes and relationships; and, when it is detected that the enterprise private domain data is updated, extract the updated enterprise private domain data, and update the knowledge graph based on the updated enterprise private domain data.
[0090] Furthermore, in one embodiment, the retrieval module 402 is also used to construct a query template for the problem data to be processed based on the key entities and the query intent; send the query template to a pre-built knowledge graph for retrieval; match multiple groups of knowledge triples related to the query template in the pre-built knowledge graph, and use the multiple groups of knowledge triples as retrieval results; wherein the multiple groups of knowledge triples are sorted from high to low according to their relevance to the query template.
[0091] Furthermore, in one embodiment, the input data generation module 403 is also used to combine the search results with the problem data to be processed to obtain preliminary input data; the preliminary input data is processed again to obtain input data suitable for the pre-trained large model.
[0092] Furthermore, in one embodiment, the input data generation module 403 is also used to extract response data; post-process the response data to obtain final response data, and send the final response data to the user terminal; the post-processing includes data analysis, data cleaning and data reorganization.
[0093] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0094] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.
[0095] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0096] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0097] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0098] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0099] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0101] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0102] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A data processing method, characterized in that: The method comprises: Preprocessing the received question data to be processed to identify key entities and query intent in the question data to be processed; According to the key entity and the query intent, a search is performed in a structured manner in a pre-constructed knowledge graph to obtain a search result; the knowledge graph is constructed based on dynamically updated enterprise private domain data; Based on the retrieval results and the question data to be processed, input data adapted to the pre-trained large model is generated to obtain answer data for the question data to be processed through the large model.
2. The method according to claim 1, characterized in that The preprocessing of the received problem data to be processed includes: The problem data to be processed is segmented into sentences, words and parts of speech using natural language processing technology, and based on the results of the sentence segmentation, word segmentation and part of speech tagging, key entities and query intent in the problem data to be processed are identified.
3. The method according to claim 1, characterized in that The knowledge graph is constructed in the following way: Extracting the enterprise private domain data; Constructing a knowledge graph based on the extracted enterprise private domain data; wherein the knowledge graph is composed of knowledge triples, each knowledge triple containing an entity, an attribute, and a relationship; and, When it is detected that the enterprise private domain data is updated, the updated enterprise private domain data is extracted, and the knowledge graph is updated based on the updated enterprise private domain data.
4. The method according to claim 3, characterized in that The searching in a structured manner in the pre-built knowledge graph according to the key entity and the query intent to obtain the search results includes: Constructing a query template for the problem data to be processed according to the key entity and the query intent; Sending the query template to the pre-built knowledge graph for retrieval; Matching multiple groups of the knowledge triples related to the query template in the pre-constructed knowledge graph, and using the multiple groups of the knowledge triples as the retrieval results; wherein the multiple groups of the knowledge triples are sorted from high to low according to the relevance to the query template.
5. The method according to claim 1, characterized in that The step of generating input data adapted to a pre-trained large model based on the search results and the problem data to be processed includes: Combining the search results with the problem data to be processed to obtain preliminary input data; The preliminary input data is processed again to obtain the input data adapted to the pre-trained large model.
6. The method according to claim 1, characterized in that After obtaining the answer data of the problem data to be processed through the large model, the method further includes: extracting the response data; The response data is post-processed to obtain final response data, and the final response data is sent to the user terminal; the post-processing includes data analysis, data cleaning and data reorganization.
7. A data processing device, characterized in that: The device comprises: A data identification module is used to pre-process the received question data to be processed so as to identify key entities and query intent in the question data to be processed; A retrieval module, used to search in a structured manner in a pre-built knowledge graph according to the key entity and the query intent to obtain a retrieval result; the knowledge graph is built based on dynamically updated enterprise private domain data; An input data generation module is used to generate input data adapted to a pre-trained large model based on the retrieval results and the problem data to be processed, so as to obtain response data of the problem data to be processed through the large model.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.