Data processing method, problem processing method, computing device and storage medium

By constructing data storage maps and multi-stage data inference, the problem of low inference efficiency and accuracy of neural network models in complex data processing is solved, and efficient data processing is achieved.

CN119005328APending Publication Date: 2024-11-22ALIBABA (CHINA) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410895933.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When neural network models process complex and large data volumes, their inference efficiency and accuracy are low, making it difficult to meet practical application needs.

Method used

By constructing a data storage map, the initial reference data associated with the initial data is determined, and the data processing model is used to perform multi-stage data inference, and the data is gradually updated until the inference conditions are met, and the inference efficiency and accuracy are improved.

Benefits of technology

It significantly improves the inference efficiency and accuracy of neural network models in complex data processing, and meets the needs of data processing in practical applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005328A_ABST
    Figure CN119005328A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a data processing method, a problem processing method, computing equipment and a storage medium, and the data processing method comprises the following steps: determining initial data, and determining initial reference data associated with the initial data from a data storage graph associated with the initial data; performing data reasoning on the initial data and the initial reference data by using a data processing model to obtain an initial reasoning result; under the condition that it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, data updating is conducted on the initial data through a data processing model according to the initial reasoning result, and updated data is obtained; determining update reference data associated with the update data from the data storage map, and performing data reasoning on the update data and the update reference data by using the data processing model to obtain an update reasoning result; and determining the update reasoning result as a target data processing result under the condition of determining that the update data does not meet the data reasoning condition according to the update reasoning result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular, to a data processing method; one or more embodiments of this specification also relate to a problem handling method, another data processing method, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the continuous development of artificial intelligence technology, neural network models can be used to process various types of data to meet the needs of practical applications.

[0003] In the prior art, the data to be processed can be input into a neural network model, and the neural network model can be used to perform data reasoning to obtain corresponding data reasoning results; however, when the neural network model processes more complex and large-volume data, there will be problems of low reasoning efficiency and low accuracy. Therefore, how to improve the data reasoning efficiency and accuracy of the neural network model has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a problem processing method, another data processing method, a data processing device, a problem processing device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defect of low reasoning efficiency of neural network models in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0006] Determine initial data and a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map;

[0007] Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain an initial reasoning result;

[0008] When it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, the initial data is updated using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data;

[0009] Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result;

[0010] When it is determined according to the update reasoning result that the update data does not satisfy the data reasoning condition, the update reasoning result is determined as the target data processing result.

[0011] According to a second aspect of an embodiment of this specification, there is provided a data processing device, including:

[0012] a data determination module, configured to determine initial data and a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map;

[0013] A first result determination module is configured to use a data processing model to perform data reasoning on the initial data and the initial reference data to obtain an initial reasoning result;

[0014] a data updating module configured to, when it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, update the initial data using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data;

[0015] A second result determination module is configured to determine update reference data associated with the update data from the data storage map, and use the data processing model to perform data reasoning on the update data and the update reference data to obtain an update reasoning result;

[0016] The third result determination module is configured to determine the updated reasoning result as the target data processing result when it is determined that the updated data does not meet the data reasoning condition according to the updated reasoning result.

[0017] According to a third aspect of the embodiments of this specification, a problem handling method is provided, including:

[0018] Determine an initial question text and a data storage map associated with the initial question text, and determine initial reference data associated with the initial question text from the data storage map;

[0019] Using the data processing model, performing question reasoning on the initial question text and the initial reference data to obtain an initial question reasoning result;

[0020] When it is determined that the initial question text satisfies the question reasoning condition according to the initial question reasoning result, the initial question text is updated by using the data processing model according to the initial question reasoning result to obtain an updated question text, wherein the amount of data in the updated question text is less than the amount of data in the initial question text;

[0021] Determine the update reference data associated with the update question text from the data storage map, and use the data processing model to perform question reasoning on the update question text and the update reference data to obtain an update question reasoning result;

[0022] When it is determined according to the update question reasoning result that the update question text does not satisfy the question reasoning condition, the update question reasoning result is determined as the target question processing result.

[0023] According to a fourth aspect of the embodiments of this specification, there is provided a problem handling device, including:

[0024] A data determination module is configured to determine an initial question text and a data storage map associated with the initial question text, and determine initial reference data associated with the initial question text from the data storage map;

[0025] A first result determination module is configured to use a data processing model to perform question reasoning on the initial question text and the initial reference data to obtain an initial question reasoning result;

[0026] A question updating module is configured to, when it is determined that the initial question text meets the question reasoning condition according to the initial question reasoning result, update the initial question text according to the initial question reasoning result using the data processing model to obtain an updated question text, wherein the data amount of the updated question text is less than the data amount of the initial question text;

[0027] A second result determination module is configured to determine the update reference data associated with the update question text from the data storage map, and use the data processing model to perform question reasoning on the update question text and the update reference data to obtain an update question reasoning result;

[0028] The third result determination module is configured to determine the update question reasoning result as the target question processing result when it is determined that the update question text does not meet the question reasoning condition according to the update question reasoning result.

[0029] According to a fifth aspect of the embodiments of this specification, another data processing method is provided, which is applied to a server, including:

[0030] Receive initial data sent by a client, determine a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map;

[0031] Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain an initial reasoning result;

[0032] When it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, the initial data is updated using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data;

[0033] Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result;

[0034] When it is determined according to the update reasoning result that the update data does not satisfy the data reasoning condition, determining the update reasoning result as the target data processing result;

[0035] The target data processing result is sent to the client.

[0036] According to a sixth aspect of an embodiment of this specification, a computing device is provided, including:

[0037] Memory and processor;

[0038] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above two data processing methods or problem processing methods are implemented.

[0039] According to the seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the above two data processing methods or problem processing methods are implemented.

[0040] According to an eighth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which implements the steps of the above two data processing methods or problem processing methods when executed by a processor.

[0041] The data processing method provided in one or more embodiments of the present specification, in the process of using the data processing model to process data, first, the initial reference data associated with the initial data is determined from the data storage map associated with the initial data, and the data processing model is used to perform the first stage of data reasoning on the initial data and the initial reference data to obtain the initial reasoning result. Since the initial reference data is used in the process of data reasoning, the data processing model can perform data reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, the initial data is updated according to the initial reasoning result, thereby obtaining updated data with a smaller amount of data than the initial data; finally, the updated reference data associated with the updated data is determined from the data storage map, and the data processing model is used to perform the second stage of data reasoning on the updated data with a smaller amount of data and the updated reference data, thereby obtaining a more accurate updated reasoning result, thereby improving the data reasoning efficiency of the data processing model, and avoiding the problem of low reasoning efficiency of the neural network model in the process of processing more complex and large data.

[0042] Moreover, when it is determined based on the updated reasoning result that the updated data does not meet the data reasoning conditions, the data reasoning is terminated and the updated reasoning result is determined as the target data processing result, thereby meeting the need to use neural network models to process various types of data in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of a RAG-based solution provided by an embodiment of this specification;

[0044] Figure 2 It is an application schematic diagram of a data processing method provided by an embodiment of this specification;

[0045] Figure 3 is a flow chart of a data processing method provided by an embodiment of this specification;

[0046] Figure 4 It is a schematic diagram of a data storage map in a data processing method provided in one embodiment of this specification;

[0047] Figure 5 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0048] Figure 6 is a schematic diagram of an LLM function in a data processing method provided by an embodiment of this specification;

[0049] Figure 7is a flow chart of a problem handling method provided by an embodiment of this specification;

[0050] Figure 8 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0051] Fig. 9 It is a structural schematic diagram of a problem handling device provided by an embodiment of this specification;

[0052] Fig.10 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0053] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification. Therefore, this specification is not limited to the specific implementation disclosed below.

[0054] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0057] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model can also be called a foundation model / foundation model (Foundation Model 1). The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (Large Language Model 1, LLM), a multi-modal pre-training model (multi-modal pre-training model 1), etc.

[0058] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0059] First, the terms involved in one or more embodiments of this specification are explained.

[0060] Large Language Model: is an AI model designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of text understanding and generation tasks. LLMs are characterized by their large size and contain billions of parameters, which help them learn complex patterns in language data.

[0061] Knowledge graph: A technology that structures information and forms a relationship network, which aims to integrate and express human knowledge and information. It can represent entities (such as people, places, events, etc.) and their relationships (such as parent-child relationships, work relationships, etc.) in the form of a graph so that machines can understand and use this knowledge. The knowledge graph is not only an information base, but also can extract and infer new knowledge from a large amount of structured and unstructured data in an automated way.

[0062] Retrieval-augmented Generation (RAG): a retrieval-augmented generation model that combines retrieval and generation techniques; the retrieval-augmented generation model combines language models and information retrieval techniques. Specifically, when the model needs to generate text or answer questions, it will first retrieve relevant information from a large collection of documents, and then use the retrieved information to guide text generation, thereby improving the quality and accuracy of predictions.

[0063] Markov Chain (MC): It is a stochastic process with Markov property in probability theory and mathematical statistics and exists in discrete index set and state space; its main idea is: a random process that changes from one state to another in state space. This process requires the property of "no memory": the probability distribution of the next state can only be determined by the current state, and the previous events in the time series are irrelevant.

[0064] Chain of Thought (CoT): Chain of Thought is an advanced prompting project that aims to enhance the performance of large language models on various complex reasoning tasks. The performance of large models can be significantly improved by allowing large models to gradually participate in the process of decomposing a complex problem into sub-problems step by step and solving them in sequence. The intermediate steps of this series of reasoning are called Chain of Thought.

[0065] Tools (too ls): various external tools that can be called by large models.

[0066] Reduction: refers to the process of converting a complex problem into one or more known, simpler problems. This conversion allows the original problem to be solved indirectly by solving these simpler problems.

[0067] The process of taking refers to a series of steps or strategies adopted when performing data reasoning. For example, a series of steps or strategies taken when solving mathematical problems include selecting appropriate theorems, formulas, algorithms or methods to gradually approach the solution to the problem. For example, when solving a complex integration problem, it may be necessary to take a series of steps such as substitution and integration by parts. These can all be regarded as "processes of taking".

[0068] Markov chain thinking: a model used to describe and infer system behavior, that is, to build LLM based on Markov chain and large model chain thinking.

[0069] Reasoning task: refers to the task of inferring unknown information through known information and reasoning rules.

[0070] ES: It is an open source distributed search engine. It is also a distributed document database in which each field can be indexed and the data in each field can be searched. It can be horizontally expanded to hundreds of servers to store and process data.

[0071] LangChain: is an open source framework that allows developers working on AI to combine large language models such as GPT-4 with external computation and data sources.

[0072] refine mechanism: It can be understood as a strategy to enhance the processing capabilities of the language model, aiming to improve the accuracy and coherence of the model in handling complex tasks and long-term conversations.

[0073] With the continuous development of artificial intelligence technology, neural network models can be used to process various types of data to meet the needs of practical applications. For example, with the rapid development of artificial intelligence and natural language processing technology, large language models have become a hot research field in recent years; large language models refer to neural network models based on deep learning, which can predict the next possible word or sentence through training with a large corpus. Large language models can have strong language understanding and generation capabilities by learning language patterns and semantic relationships in large data sets; however, large models usually perform well when dealing with relatively simple logical reasoning tasks, but encounter difficulties when dealing with more complex logical reasoning tasks, and their logical reasoning task capabilities are limited; this is because complex logical reasoning tasks often require higher-level reasoning capabilities, such as multi-step reasoning and the establishment of reasoning chains.

[0074] In response to the above problems, this specification provides a solution based on RAG. RAG refers to enhancing the ability of LLM by plugging in other data sources in addition to the knowledge that LLM has learned. Figure 1 It can be seen that Figure 1 is a schematic diagram of a RAG-based solution provided by an embodiment of this specification, see Figure 1In order to enhance the capabilities of LLM, other data sources can be plugged in, including external vector databases, documents, networks, external knowledge graphs or other sources, directly connecting to the existing ES, or connecting to the search engines in the existing production environment. The access methods are similar. First, the document loader (DocLoader) retrieves the content related to the context given by the user in the plug-in data source, and merges the data through the data transformer module (Datatransformers); secondly, after merging, the embedding layer (embedd ing) is input for processing, and the processed data is stored in the vector storage (vector storage); finally, the search engine (Retrievers) obtains the required data from the vector storage and provides it to the question and answering system (Question and Answering, QA), which gives the data to the LLM used, and finally the LLM makes an answer based on the model's own cognition. In the final analysis, it is a map-reduce process. Of course, the langchain is further refined, and mechanisms such as refine are added.

[0075] However, it should be noted that this solution has certain defects: mainly because it does not use historical information and the granularity of the retrieval content is not detailed enough, resulting in low inference accuracy.

[0076] Based on this, in this specification, a data processing method is provided. One or more embodiments of this specification also involve a problem processing method, another data processing method, a data processing device, a problem processing device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0077] See also Figure 2 , Figure 2 A schematic diagram of an application of a data processing method provided according to an embodiment of the present specification is shown. Figure 2It can be seen that the user can send questions to the server 104 through the terminal 102. The server 104 is the server used by the data processing method in one or more embodiments of the present specification; after receiving the question, the server 104 queries the reference data associated with the question from the knowledge graph, and inputs the question and the reference data into the LLM large model, and uses the large model to perform one-step reasoning on the current question, and obtains the reasoning result of the first stage after the one-step reasoning, and then obtains a new question using the question and the reasoning result; continues to perform the operation of querying the reference data associated with the new question from the knowledge graph, and uses the large model to perform one-step reasoning on the new question and the reference data to obtain the reasoning result of the second stage, and again obtains a question that is consistent with the answer to the new question but simpler based on the reasoning result and the new question; repeats this process until the answer can be obtained through question solving, and the answer corresponding to the user's question is obtained; and then the answer is sent to the terminal 102. In the above steps, by converting the long sequence reasoning process in the original complex problem into multiple short sequence reasoning processes, the reasoning cost of the large model is significantly reduced; and, combined with the structured knowledge of the knowledge graph, the large model can better plan the reasoning path and improve the reasoning efficiency of the large model.

[0078] Among them, the server is a server applied by another data processing method in one or more embodiments of this specification, and the server may include but is not limited to a server, a server cluster, a data center, a cloud server, etc.; the server can be connected to one or more terminals through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The terminals here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc.; the terminal can interact with the user through a user interaction interface to realize the call of the large model, thereby realizing the method provided in the embodiments of this specification.

[0079] See also Figure 3 , Figure 3 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0080] Step 302: Determine initial data and a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map.

[0081] The initial data can be understood as data that needs to be processed using a data processing model, and the initial data can be any type of data, for example, the initial data can be text data, code data, language data, video data, image data, etc. In other words, the initial data can be initial question text, initial code data, initial voice data, initial image data, initial video data, etc., and no specific limitation is made here.

[0082] The data storage map can be understood as a map for storing reference data. The data storage map can be a schematic diagram of a data storage map in a data processing method provided by an embodiment of a tree-structured book. Figure 4 It can be seen that the data storage map can be a tree-structured map, and the nodes in the map represent a reference data; it should be noted that Figure 4 The displayed map takes the map constructed by the high school mathematics knowledge base as an example to explain the data storage map, but does not limit the data structure and data in the data storage map.

[0083] The initial reference data can be understood as reference data associated with the initial data; the reference data can be understood as data used to help the data processing model perform data reasoning on the initial data; for example, when the initial data is a high school mathematics problem, the initial reference data can be the knowledge point data corresponding to high school mathematics; when the initial data is initial code data, the initial reference data can be data such as public documents for optimizing the code. In the data processing method provided in one or more embodiments of this specification, it is considered that complex logical reasoning tasks often require higher-level reasoning capabilities, such as the establishment of multi-step reasoning and reasoning chains. The knowledge graph provides a structured knowledge representation, including entities (such as people, places, events, etc.), attributes and relationships (such as parental relationships, work relationships, etc.), which can be used to represent facts and concepts in the real world. Therefore, it is hoped that the language understanding and generation capabilities of large-scale pre-trained models can be combined with the structured knowledge of knowledge graphs to enable large models to better plan data reasoning paths, realize controllable text generation in single-step reasoning, and effectively improve the reasoning capabilities of large models through multiple rounds of attempts.

[0084] The data processing model can be understood as a neural network model that can perform data processing. The data processing model can be a neural network model or a machine learning model (such as LLM). It should be noted that when the data processing method is applied to different scenarios, the data processing model is also different. For example, when the data processing method is applied to a problem reasoning scenario, the initial data can be an initial problem text; the initial reasoning result can be an initial problem reasoning result; the data processing model can be a problem processing model for problem reasoning and updating the initial problem text; based on this, by inputting the initial problem text into the problem processing model, the answer to the problem output by the problem processing model (i.e., the target data processing result) is obtained, thereby meeting the need to use a neural network model for problem reasoning in practical applications.

[0085] For another example, when the data processing method is applied to a code optimization scenario, the initial data may be initial code data; the initial reasoning result may be an initial code optimization result for the initial code data; the data processing model may be a code optimization model for performing code optimization and updating the initial code data; based on this, by inputting the initial code data into the code optimization model, the optimized code data (i.e., the target data processing result) output by the code optimization model is obtained, thereby meeting the need to use a neural network model for code optimization in actual applications.

[0086] The initial reasoning result can be understood as the processing result obtained after using the data processing model to perform the first stage of data reasoning on the initial data; the first stage of data reasoning can be understood as data reasoning on the initial sub-data in the initial data; for example, when the initial data is an initial question text, the initial sub-data can be a sub-problem constituting the initial question text; based on this, the initial reasoning result can be the problem reasoning result obtained by using the data processing model to perform problem reasoning on the sub-problems in the initial question text; for another example, when the initial data is initial code data, the initial sub-data can be one or more code statements in the initial code data; based on this, the initial reasoning result can be the code optimization result obtained by using the code optimization model to perform code optimization on one or more code statements in the initial code data.

[0087] In one or more embodiments provided in this specification, since the initial data can be of multiple types, in order to ensure that the data processing model can better perform data reasoning, multiple types of candidate data storage graphs are constructed for the initial data for data reasoning, and the specific implementation method is as follows.

[0088] The determining of the initial data and the data storage map associated with the initial data includes:

[0089] Determine initial data and multiple candidate data storage maps;

[0090] Based on the data type information of the initial data, a data storage map associated with the initial data is determined from the multiple candidate data storage maps.

[0091] Among them, the candidate data storage map can be of various types. For example, the candidate data storage map can be: a data storage map constructed based on high school mathematics knowledge, a data storage map constructed based on high school English knowledge, a data storage map constructed based on film field knowledge, etc.

[0092] The data type information of the initial data can be understood as the data type corresponding to the initial data. For example, when the initial data is an initial question text, the data type information can be understood as the field of technical problems in which the initial question text is (such as high school mathematics type, movie type, language translation type, etc.); when the initial data is an initial video, the data type information can be understood as the type of video the initial video data is (such as sports video type, pet video type, landscape video type, etc.).

[0093] Specifically, the data processing method provided in this specification can pre-construct multiple candidate data storage maps. After the initial data is determined, multiple candidate data storage maps can be determined, wherein the candidate data storage maps can be stored in a map storage unit (such as a local disk, a database, etc.); then, based on the data type information of the initial data, the candidate data storage map corresponding to the data type information is determined from multiple candidate data storage maps, and used as the data storage map associated with the initial data, thereby facilitating the subsequent accurate acquisition of initial reference data with a high degree of correlation with the initial data, thereby ensuring the accuracy of data reasoning by the subsequent data processing model.

[0094] In one or more embodiments provided in this specification, since a large amount of data may exist in a data storage map, in order to avoid using too much data as initial reference data, which causes the data processing model to need to analyze a large amount of data, the initial reference data associated with the initial data can be queried from the data storage map; the specific implementation method is as follows.

[0095] The determining of initial reference data associated with the initial data from the data storage map includes:

[0096] Determine the initial data as a data query statement;

[0097] Based on the data query statement, query and obtain query data corresponding to the data query statement from the data storage graph;

[0098] The query data is determined as the initial reference data associated with the initial data.

[0099] Among them, the data query statement can be understood as a statement used to query reference data from the data storage graph.

[0100] Specifically, after determining the data storage map associated with the initial data, the data processing method provided in this specification can determine the initial data as a data query statement, and query the data storage map based on the data query statement, so as to obtain one or more query data associated with the data query statement; then, the query data is determined as the initial reference data associated with the initial data, so as to facilitate subsequent data reasoning based on a smaller number of reference data with a higher degree of correlation with the initial data, thereby reducing the data reasoning pressure of the data processing model and improving the accuracy of the reasoning results.

[0101] In one or more embodiments provided in the present specification, in the process of querying initial reference data associated with initial data from a data storage map, the number of query data obtained by the query may be large. In order to avoid using too much data as initial reference data, which causes the data processing model to need to analyze a large amount of data, some query data can be screened from multiple query data as initial reference data. The specific implementation method is as follows.

[0102] The query data is multiple;

[0103] The determining the query data as the initial reference data associated with the initial data comprises:

[0104] determining the plurality of query data as a plurality of candidate reference data;

[0105] The initial reference data associated with the initial data is determined from the multiple candidate reference data using the data processing model, wherein the data processing model is a machine learning model.

[0106] The candidate reference data may be understood as data that needs to be screened and from which initial reference data is determined.

[0107] Specifically, when multiple query data are queried from the data storage map, the multiple query data are used as candidate reference data, and the multiple candidate reference data and the initial data are input into the data processing model, and the multiple candidate reference data and the initial data are analyzed using the data processing model, and the candidate reference data with the highest correlation with the initial data is determined from the multiple candidate reference data as the initial reference data, wherein the data quantity of the initial reference data is less than the data quantity of the multiple candidate reference data.

[0108] Based on the above embodiments, it can be seen that the data processing method provided in this specification can use the data processing model to filter out data with a high correlation with the initial data from multiple query data as initial reference data in order to avoid using too much data as initial reference data, which causes the data processing model to need to analyze a large amount of data. This can reduce the data reasoning pressure of the data processing model while improving the accuracy of the reasoning results.

[0109] In one or more embodiments provided in this specification, before using the data storage graph for data reasoning, it is necessary to construct multiple data storage graphs to ensure that the data storage graph associated with the initial data can be quickly determined later. The specific implementation method is as follows.

[0110] Before determining the initial data and the data storage map associated with the initial data, the method further includes:

[0111] Determine the graph structure to be constructed and / or the graph data to be constructed;

[0112] The graph construction model is utilized to construct the graph according to the graph structure to be constructed and / or the graph data to be constructed, so as to obtain the data storage graph.

[0113] Among them, the graph construction model can be understood as a machine learning model used to construct a data storage graph. For example, the graph construction model can be a large language model.

[0114] Specifically, the data processing method provided in this specification can be implemented in three ways during the process of constructing a data storage map.

[0115] The first method is to determine the graph data to be constructed for constructing a data storage graph, such as high school mathematics knowledge data, film field related data, etc.

[0116] The graph data to be constructed is input into the graph construction model, and the graph construction model is used to perform data structure analysis on the graph data to be constructed, so as to determine the data structure corresponding to the graph data to be constructed, and the graph construction model can fill the graph data to be constructed into the data structure, so as to efficiently complete the construction of the data storage graph.

[0117] The second method is to determine the graph structure to be constructed for constructing the data storage graph, for example, a high school mathematics knowledge framework, a world geography knowledge framework, etc.

[0118] The graph structure to be constructed is input into the graph construction model, and the graph construction model is used to fill the knowledge data learned into the graph structure to be constructed, thereby completing the construction of the data storage graph.

[0119] The third method is: determining the graph structure to be constructed and the graph data to be constructed for constructing the data storage graph;

[0120] The graph structure to be constructed and the graph data to be constructed are input into the graph construction model, analyzed using the graph construction model, and the graph data to be constructed are filled into the graph structure to be constructed, thereby completing the construction of the data storage graph.

[0121] In the above embodiments, the construction of the data storage map is achieved through a variety of methods, thereby improving the applicability of the data processing method in constructing the data storage map, thereby realizing the map construction according to the actual application situation and meeting the actual needs of different users.

[0122] Step 304: Using the data processing model, perform data reasoning on the initial data and the initial reference data to obtain initial reasoning results.

[0123] In one or more embodiments provided in this specification, taking into account the problem of low reasoning efficiency when a neural network model processes relatively complex data with a large amount of data, the data processing method provided in this specification can use multi-stage data reasoning to process the initial data, and each stage can perform data reasoning on the initial sub-data in the initial data until the data reasoning for the initial data is completed; the specific implementation method is as follows.

[0124] The using of the data processing model to perform data reasoning on the initial data and the initial reference data to obtain an initial reasoning result includes:

[0125] Inputting the initial data and the initial reference data into the data processing model, performing data segmentation on the initial data using the data processing model, and determining a plurality of initial sub-data contained in the initial data;

[0126] The sub-data to be inferred is determined from the multiple initial sub-data, and semantic reasoning is performed on the sub-data to be inferred according to the initial reference data to obtain the initial reasoning result.

[0127] Among them, the initial sub-data can be understood as the sub-data constituting the initial data. When the initial data is the initial question text, the initial sub-data can be the sub-question text in the initial question text. For example, the initial question text can be "What is 1+2×3 equal to?", and correspondingly, the multiple sub-question texts are "What is 1+2 equal to?", "What is 2×3 equal to?"; the initial question text can be "Who is the father of A's wife?", and correspondingly, the multiple sub-question texts are "Who is A's wife?", "Who is the father of the wife?"; when the initial data is the initial video data, the initial sub-data can be one or more video frames in the initial video data; when the initial data is the initial image data or the initial audio data, the initial sub-data can be a local image area in the initial image data or a small segment of audio data in the initial audio data, etc.; when the initial data is the initial code data, the initial sub-data can be one or more code statements in the initial code data. For example, the initial code data can be "int main(){

[0128]

[0129] Correspondingly, the initial sub-data may be one or more code statements in the initial code data. For example, the initial sub-data may be two code statements “int a=5;, int b=10;”.

[0130] Among them, the sub-data to be inferred can be understood as the sub-data among multiple initial sub-data that currently needs to use the data processing model for data reasoning; the sub-data to be inferred can be any initial sub-data among multiple initial sub-data, or the initial sub-data determined according to data reasoning rules (such as mathematical operation rules, code running rules) that requires first-stage data reasoning; for example, the first sub-problem among multiple initial sub-problems, or the sub-problem determined according to mathematical operation rules that requires first-stage data reasoning.

[0131] The initial reasoning result can be understood as the result obtained by using the data processing model to perform semantic reasoning on the sub-data to be inferred. For example, when the sub-data to be inferred is the sub-problem "What is 2×3 equal to", the initial problem reasoning result (i.e., the initial reasoning result) can be "2×3 equals 6"; when the sub-data to be inferred is "int a=5;, int b=10;", two code statements that need to be optimized, the initial code optimization result (i.e., the initial reasoning result) can be "int a=5;, int b=10;, which can be written as one code statement int a=5, b=10;".

[0132] The following is an explanation of the data processing method provided in this specification by taking the application of the data processing method provided in this specification in a problem reasoning scenario as an example, wherein the initial data is the initial problem text, the data processing model is the problem processing model, the initial sub-data is the initial sub-problem, the sub-data to be inferred is the sub-problem to be inferred, the initial reference data is the reference data related to the initial problem text and used to answer the initial problem text, and the initial reasoning result is the initial problem reasoning result; based on this, the data processing method provided in this specification can be applied to a data processing node, which, after determining the initial problem text, inputs the initial problem text into a trained problem processing model, uses the problem processing model to perform problem extraction on the initial problem text, thereby identifying multiple initial sub-problems contained in the initial problem text; then, selects a sub-problem to be inferred from the multiple initial sub-problems, and performs the first stage of problem reasoning on the sub-problem to be inferred based on the reference data, thereby obtaining the initial problem reasoning result for the sub-problem to be inferred.

[0133] It should be noted that the data processing node can be a service end, server, virtual machine, container, cloud server, client, mobile terminal and other devices, or the data processing node can be an application, software system and other software devices, which are not specifically limited here.

[0134] In the above embodiment, in the process of using the data processing model to perform data reasoning on the initial data to obtain the initial reasoning result, the sub-data to be inferred is determined from the multiple initial sub-data contained in the initial data by using the data processing model, and data reasoning is performed on the sub-data to be inferred based on the initial reference data to obtain the initial reasoning result; because the data processing model can quickly and accurately perform data reasoning on relatively simple sub-data with a smaller amount of data, therefore, when the data processing model is used to process the initial sub-data in a multi-stage data reasoning manner, data reasoning can be efficiently realized for the initial data, thereby avoiding the problem of low reasoning efficiency when the neural network model is processing relatively complex data with a larger amount of data.

[0135] In one or more embodiments provided in this specification, the data processing node used by the data processing method can receive initial data sent by the user based on the client, and can subsequently perform data processing on the initial data. The specific implementation method is as follows.

[0136] The determining of initial data comprises:

[0137] Receiving the initial data sent by the client, wherein the initial data is generated by the client according to a data processing operation performed by the user on the data processing page;

[0138] The client may be a mobile terminal, a smart device or other server-side device, or the client may be an application, a software system or other software.

[0139] The data processing page may be understood as a user interaction interface displayed in the client for implementing user data processing; the data processing page may be a web page, an application program interface, and the like.

[0140] The data processing operation may be an operation performed by a user based on the data processing page, such as inputting data, clicking a button, etc. to trigger a data processing control in the data processing page.

[0141] Continuing with the above example, the client can display a data processing page to the user, and the user performs data processing operations based on the data processing controls (such as buttons, input boxes, etc.) in the data processing page, thereby triggering the client to generate initial data, and send the initial data to the data processing node, instructing the data processing node to use the data processing model to perform data processing operations on the initial data.

[0142] In the above embodiment, the user implements the data processing operation on the initial data by using the client and the data processing page, thereby satisfying the user's need to use the data processing model to process the initial data.

[0143] In one or more embodiments provided in this specification, it is taken into consideration that the data processing model may select inaccurate reference data for data reasoning during data reasoning based on reference data; therefore, in order to ensure the accuracy of the data reasoning results, the accuracy of the reasoning results may be tested after the data reasoning is performed, and the specific implementation method is as follows.

[0144] The using of the data processing model to perform data reasoning on the initial data and the initial reference data to obtain an initial reasoning result includes:

[0145] Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain a reasoning result to be detected;

[0146] The data processing model is used to perform a result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested passes the result accuracy test, the inference result to be tested is determined as the initial inference result.

[0147] The inference result to be detected can be understood as the inference result that needs to be detected.

[0148] Among them, the determination that the reasoning result to be detected passes the result accuracy test can be understood as using the reasoning result to be detected to perform reverse reasoning on the initial data, and determining whether the initial data can be inferred based on the reasoning result to be detected. If so, it is determined that the reasoning result to be detected passes the result accuracy test. Otherwise, it is determined that the reasoning result to be detected does not pass the result accuracy test.

[0149] Specifically, the initial data and the initial reference data are input into a data processing model, and the data processing model is used to perform data reasoning on the initial data and the initial reference data to obtain reasoning results to be tested that need to be tested for accuracy.

[0150] Then, the reasoning result to be detected and the initial data are input into the data processing model, and the data processing model is used to perform reverse reasoning on the initial data based on the reasoning result to be detected. When it is determined that the initial data can be inferred based on the reasoning result to be detected, it is determined that the reasoning result to be detected meets the result accuracy test, and the reasoning result to be detected is determined as the initial reasoning result.

[0151] In the above embodiment, after data inference is performed, the accuracy of the "inference result to be detected" is tested to ensure the accuracy of the data inference result.

[0152] In one or more embodiments provided in this specification, it is taken into consideration that the data processing model may select inaccurate reference data for data reasoning during data reasoning based on reference data; therefore, in order to ensure the accuracy of the data reasoning results, the accuracy of the reasoning results may be tested after the data reasoning is performed, and the specific implementation method is as follows.

[0153] After the data processing model is used to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be detected, the method further includes:

[0154] Using the data processing model, performing a result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested does not pass the result accuracy test, determining a plurality of candidate reference data associated with the initial data, wherein the initial reference data is determined from the plurality of candidate reference data;

[0155] Determining, using the data processing model, updated initial reference data associated with the initial data from the plurality of candidate reference data, wherein the updated initial reference data is different from the initial reference data;

[0156] The updated initial reference data is determined as the initial reference data, and the step of using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be tested is continued until it is determined that the reasoning result to be tested meets the result accuracy test.

[0157] The updated initial reference data may be understood as reference data re-determined from a plurality of candidate reference data.

[0158] Specifically, the initial data and the initial reference data are input into a data processing model, and the data processing model is used to perform data reasoning on the initial data and the initial reference data to obtain a reasoning result to be tested that needs to be tested for accuracy;

[0159] The inference result to be detected and the initial data are input into a data processing model, and the data processing model is used to perform reverse inference on the initial data based on the inference result to be detected. When it is determined that the initial data cannot be inferred based on the inference result to be detected, it is determined that the inference result to be detected does not meet the result accuracy test, based on which, it is necessary to determine a plurality of candidate reference data associated with the initial data;

[0160] Re-input multiple candidate reference data and initial data into the data processing model, use the data processing model to analyze the multiple candidate reference data and the initial data, and determine the updated initial reference data with the highest correlation with the initial data from the multiple candidate reference data. It should be noted that the data processing model records the reasoning results and reference data generated in each stage of the data reasoning process. Therefore, the data processing model can determine the updated initial reference data that is different from the initial reference data during the process of re-determining the updated initial reference data.

[0161] Finally, the updated initial reference data is re-determined as the initial reference data, and the step of using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be tested is continued until it is determined that the reasoning result to be tested meets the result accuracy test.

[0162] Step 306: When it is determined that the initial data meets the data reasoning conditions based on the initial reasoning result, the initial data is updated using the data processing model based on the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data.

[0163] Among them, the data reasoning condition can be understood as the condition that the data processing model needs to be used again for data reasoning. The data reasoning condition can be set according to the actual application scenario. For example, satisfying the data reasoning condition can be understood as not completing the data reasoning operation for all the initial sub-data contained in the initial data, or the data can be updated based on the data reasoning result (such as the initial reasoning result, the updated reasoning result). For another example, not satisfying the data reasoning condition can be understood as completing the data reasoning operation for all the initial sub-data contained in the initial data, or the data cannot be updated based on the data reasoning result (such as the initial reasoning result, the updated reasoning result).

[0164] Updated data can be understood as data obtained after updating the initial data. For example, when the initial data is "1+2×3 equals to what" and the initial reasoning result is "2×3 equals to 6", the updated data can be "1+6 equals to what"; for another example, when the initial reasoning result is "int a=5;, int b=10; which can be written as a code statement in a=5, b=10;", the updated data can be "

[0165]

[0166] In one or more embodiments provided in this specification, during the process of multi-stage data reasoning, it is necessary to verify whether to continue reasoning at each stage so as to determine in a timely manner whether the reasoning is completed. The specific implementation method is as follows.

[0167] The determining, based on the initial reasoning result, that the initial data satisfies a data reasoning condition includes:

[0168] Determining initial sub-data contained in the initial data using the data processing model;

[0169] The initial sub-data and the initial reasoning result are semantically analyzed, and when it is determined that there are multiple initial sub-data and there is sub-data to be updated associated with the initial reasoning result among the multiple initial sub-data, it is determined that the initial data meets the data reasoning condition.

[0170] Among them, the sub-data to be updated can be understood as the sub-data that needs to be updated using the initial reasoning results. For example, when multiple sub-problem texts are "What is 1+2 equal to?", "What is 2×3 equal to?", and the initial reasoning result is "2×3 equals 6", the large model can be used to determine from multiple sub-problem texts that the sub-problem "What is 2×3 equal to" is the sub-data to be updated, and the sub-data to be updated can be subsequently updated based on the initial reasoning result.

[0171] Continuing with the above example, after using the data processing model to determine the initial problem reasoning result "2×3 equals 6", the initial problem reasoning result and the initial problem are input into the data processing model again for verification to determine whether to continue problem reasoning (i.e., problem reduction). When it is determined that the initial problem contains multiple sub-problems, and among the multiple sub-problems there is a sub-problem to be updated corresponding to "2×3 equals 6" "What is 2×3 equals", it is determined that problem reduction can continue.

[0172] In the above embodiment, during the multi-stage data reasoning process, by verifying whether to continue reasoning at each stage, it is possible to promptly determine whether the reasoning is completed, thereby avoiding wasting computing resources due to repeated reasoning.

[0173] In one or more embodiments provided in this specification, the initial data is an initial question text, the initial reasoning result is an initial question reasoning result, and the data processing model is a question processing model;

[0174] When it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, updating the initial data using the data processing model according to the initial reasoning result to obtain updated data includes:

[0175] When it is determined that the initial question text meets the question reasoning condition according to the initial question reasoning result, the initial question text is updated by using the question processing model according to the initial question reasoning result to obtain an updated question text.

[0176] The problem reasoning condition may be problem reduction.

[0177] For the explanation of this embodiment, reference may be made to the corresponding or corresponding explanations in one or more other embodiments.

[0178] In one or more embodiments provided in this specification, taking into account the problem of low reasoning efficiency when a neural network model processes relatively complex data with a large amount of data, the data processing method provided in this specification can process the initial data through multi-stage data reasoning, and in the processing process of each stage, the data reasoning result is used to update the data to be reasoned (such as initial data) in the current stage to obtain data with a smaller amount of data. The specific implementation method is as follows.

[0179] The method of updating the initial data using the data processing model according to the initial reasoning result to obtain updated data includes steps 1 to 3:

[0180] Step 1: Input the initial inference result, the initial reference data and the initial data into the data processing model.

[0181] Step 2: using the data processing model, determine a plurality of initial sub-data contained in the initial data, and determine, from the plurality of initial sub-data, the sub-data to be updated that is associated with the initial reasoning result and the initial reference data.

[0182] Specifically, the using the data processing model to determine a plurality of initial sub-data contained in the initial data, and determining, from the plurality of initial sub-data, sub-data to be updated that is associated with the initial reasoning result and the initial reference data, includes:

[0183] Inputting the initial reasoning result and the initial data into the data processing model, performing data extraction on the initial data using the data processing model, and determining the multiple initial sub-data contained in the initial data;

[0184] Semantic analysis is performed on each initial sub-data and the initial reasoning result to obtain the sub-data to be updated associated with the initial reasoning result and the initial reference data.

[0185] The data extraction may be understood as a semantic analysis operation or a data segmentation operation.

[0186] Continuing with the above example, after using the data processing model to determine the reasoning result of the initial problem "2×3 equals 6", the initial problem reasoning result, the initial reference data and the initial problem are input into the large model again for semantic analysis. From the multiple sub-problems contained in the initial problem, the updated sub-problem "2×3 equals what" corresponding to "2×3 equals 6" and the initial reference data is determined to facilitate subsequent problem reduction.

[0187] In the embodiments provided in this specification, the data processing model is used to determine the multiple initial sub-data contained in the initial data. It can also be understood that the data processing model is used to segment the initial data to determine the multiple initial sub-data contained in the initial data.

[0188] In the above embodiment, in the process of using the data processing model to determine multiple initial sub-data contained in the initial data, and determining the sub-data to be updated associated with the initial reasoning result from the multiple initial sub-data, the data processing model can be used to perform semantic analysis on the multiple initial sub-data contained in the initial data, so as to determine the sub-data to be updated associated with the initial reasoning result, so as to facilitate the subsequent use of the initial reasoning result to accurately update the sub-data to be updated, avoid the semantic deviation of the initial data due to update errors, and ensure the accuracy of the target data processing results.

[0189] Step 2: Using the data processing model, based on the initial inference result and the initial reference data, update the sub-data to be updated in the initial data to obtain the updated data.

[0190] In one or more embodiments provided in this specification, the data processing model may be used to update the sub-data to be updated in the initial data based on the initial reasoning result to obtain the updated data, specifically including:

[0191] Using the data processing model, based on the initial reasoning result, the sub-data to be updated in the initial data is replaced to obtain the updated data; or

[0192] The data processing model is used to semantically rewrite the sub-data to be updated in the initial data based on the initial reasoning result to obtain the updated data.

[0193] Continuing with the above example, the initial problem reasoning result can be "2×3 equals 6" or "6"; based on this, when the data processing model is used to determine the updated sub-problem "2×3 equals what" corresponding to the initial problem reasoning result "6", the updated sub-problem "2×3 equals what" in the initial problem text "1+2×3 equals what" can be directly replaced based on the initial problem reasoning result to obtain the replaced problem text "1+6 equals what", thereby achieving problem reduction.

[0194] When the initial question reasoning result is “2×3 equals 6”, the data processing model can be used to semantically rewrite the sub-data “2×3 equals how much” to be updated in the initial question text “1+2×3 equals how much” based on the initial question reasoning result “2×3 equals 6” to obtain the semantically rewritten question text “1+6 equals how much”, thereby achieving problem reduction.

[0195] In the above embodiment, the data to be inferred (eg, initial data) at the current stage is updated by using the data inference result to obtain data with a smaller amount of data, thereby improving the data processing efficiency of the subsequent data processing model.

[0196] In one or more embodiments provided in this specification, the data processing model can be used to update the sub-data to be updated in the initial data based on the initial reasoning result and the initial reference data to obtain the updated data. This can be understood as: using the data processing model, based on the initial reasoning result and the initial reference data, semantically rewriting the sub-data to be updated in the initial data to obtain the updated data, thereby achieving the goal of updating the data to be inferred (such as the initial data) in the current stage by using the data reasoning results and the reference data, and obtaining data with a smaller amount of data, thereby improving the data processing efficiency of the subsequent data processing model.

[0197] Step 308: Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result.

[0198] Among them, determining the updated reference data associated with the updated data from the data storage map can refer to the step of determining the initial reference data associated with the initial data from the data storage map in the above embodiment, and will not be elaborated here.

[0199] Among them, the data processing model is used to perform data reasoning on the updated data and the updated reference data to obtain updated reasoning results. The steps of using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the initial reasoning results in the above embodiment can be referred to, and no further details will be given here.

[0200] Continuing with the above example, the updated question text can be "What is 1+6 equal to?" and the updated reference data can be addition knowledge information of elementary school mathematics; it is input into the large model for semantic reasoning to obtain the corresponding reasoning result "1+6 equals 7".

[0201] Step 310: When it is determined that the updated data does not satisfy the data reasoning condition according to the updated reasoning result, the updated reasoning result is determined as the target data processing result.

[0202] Among them, the target data processing result is understood to be the processing result for the initial data. For example, when the initial data is the initial question text, the target data processing result can be the answer to the question corresponding to the question text; when the initial data is the initial code data, the target data processing result can be the code data obtained after optimizing the initial code data; it should be noted that the initial reasoning result can be the reasoning result for a certain initial sub-data in the initial data, and the updated reasoning result can be the reasoning result for a certain updated sub-data in the updated data; the target data processing can be the overall processing result for the initial data; and, when the updated reasoning result determines that the updated data does not meet the data reasoning conditions, the updated reasoning result can be determined as the target data processing result.

[0203] Continuing with the above example, when it is determined that the update question reasoning result corresponding to the update question text (i.e., the update reasoning result) is "1+6 equals 7", the update question reasoning result and the update question are input into the large model again for semantic analysis. When it is determined that the update question contains only one sub-question "1+6 equals multiple", and the sub-question is associated with the update question reasoning result, it is determined that the multi-stage data reasoning is completed, thereby converting the update question reasoning result "1+6 equals 7" into the question answer "1+2×3 equals 7".

[0204] In one or more embodiments provided in this specification, during the process of multi-stage data reasoning, it is necessary to verify whether to continue reasoning at each stage so as to determine in a timely manner whether the reasoning is completed. The specific implementation method is as follows.

[0205] The determining, according to the update reasoning result, that the update data does not satisfy the data reasoning condition includes:

[0206] Determining the update sub-data included in the update data by using the data processing model;

[0207] Semantic analysis is performed on the update sub-data and the update reasoning result, and when it is determined that the update sub-data is one and the update sub-data is associated with the update reasoning result, it is determined that the update data does not satisfy the data reasoning condition.

[0208] The update sub-data may be understood as the sub-data included in the update data. For example, when the update data is an update question text, the update sub-data may be understood as the sub-question text included in the update question text.

[0209] Continuing with the above example, when it is determined that the update problem reasoning result corresponding to the update problem text (i.e., the update reasoning result) is "1+6 equals 7", the update problem reasoning result and the update problem are input into the large model again for semantic analysis. When it is determined that the update problem contains only one sub-problem "1+6 equals multiple", and this sub-problem is associated with the update problem reasoning result, it is determined that the multi-stage data reasoning is completed.

[0210] In the above embodiment, during the multi-stage data reasoning process, by verifying whether to continue reasoning at each stage, it is possible to promptly determine whether the reasoning is completed, thereby avoiding wasting computing resources due to repeated reasoning.

[0211] In one or more embodiments provided in this specification, the data processing node used by the data processing method can display the target data processing results to the client after receiving the initial data sent by the user based on the client and performing data processing on the initial data. The specific implementation method is as follows.

[0212] After determining, based on the update reasoning result, that the update data does not satisfy the data reasoning condition, the update reasoning result is determined as the target data processing result, the method further includes:

[0213] The target data processing result is sent to the client, so that the client displays the target data processing result based on the data processing page.

[0214] Among them, displaying the target data processing result based on the data processing page can be understood as displaying the target data processing result to the user based on the data processing page, or displaying the target data processing result to other objects based on the data processing page. The other object may be an object that needs to obtain the target data processing result, for example, the user may be a teacher, and the other object may be a student.

[0215] In the above embodiment, the target data processing result is displayed by using the client and the data processing page, thereby satisfying the user's need to use the data processing model to process the initial data.

[0216] In one or more embodiments provided in this specification, during the process of multi-stage data reasoning, it is necessary to verify whether to continue reasoning at each stage so as to determine in a timely manner whether the reasoning is completed. The specific implementation method is as follows.

[0217] After performing data reasoning on the update data using the data processing model to obtain an update reasoning result, the method further includes:

[0218] When it is determined that the updated data meets the data reasoning condition according to the update reasoning result, the updated data is updated using the data processing model according to the update reasoning result to obtain iterative update data, wherein the data quantity of the iterative update data is less than the data quantity of the update data;

[0219] The iterative update data is determined as the update data, and the step of performing data reasoning on the update data using the data processing model to obtain an update reasoning result is continued until it is determined, based on the update reasoning result, that the update data does not satisfy the data reasoning condition.

[0220] Continuing with the above example, the initial question may be “What is 1+1+2×3 equal to?”, the update question may be “What is 1+1+6 equal to?”, and the update question reasoning result is “1+1 equals 2”. Based on this, after the update question reasoning result “1+1 equals 2” is determined using the data processing model, the update question reasoning result and the update question are input into the large model again for verification to determine whether to continue the question reasoning (i.e., problem reduction). When it is determined that the update question contains multiple sub-problems “What is 1+1 equal to?” and “What is 1+6 equal to?”, and there is a sub-problem to be updated “What is 1+1 equals?” corresponding to “1+1 equals 2” in the multiple sub-problems, it is determined that the problem reduction can continue.

[0221] The data processing method provided in one or more embodiments of the present specification, in the process of using the data processing model to process data, first, the initial reference data associated with the initial data is determined from the data storage map associated with the initial data, and the data processing model is used to perform the first stage of data reasoning on the initial data and the initial reference data to obtain the initial reasoning result. Since the initial reference data is used in the process of data reasoning, the data processing model can perform data reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, the initial data is updated according to the initial reasoning result, thereby obtaining updated data with a smaller amount of data than the initial data; finally, the updated reference data associated with the updated data is determined from the data storage map, and the data processing model is used to perform the second stage of data reasoning on the updated data with a smaller amount of data and the updated reference data, thereby obtaining a more accurate updated reasoning result, thereby improving the data reasoning efficiency of the data processing model, and avoiding the problem of low reasoning efficiency of the neural network model in the process of processing more complex and large data.

[0222] Moreover, when it is determined based on the updated reasoning result that the updated data does not meet the data reasoning conditions, the data reasoning is terminated and the updated reasoning result is determined as the target data processing result, thereby meeting the need to use neural network models to process various types of data in practical applications.

[0223] The following combination Figure 5 , taking the application of the data processing method provided in this specification in the data reasoning scenario as an example, the data processing method is further explained. Figure 5 A flowchart of a data processing method provided by an embodiment of the present specification is shown. Figure 5 It can be seen that the data processing method provided in this specification can combine the Markov chain and CoT forms, transform the multi-step reasoning problem into a process of multiple independent problem reasoning and sub-problem generation, and complete the problem reasoning process through multiple rounds of iterations.

[0224] In addition, considering that large models usually perform better in dealing with relatively simple logical reasoning tasks, but encounter difficulties in dealing with more complex logical reasoning tasks, and their logical reasoning task capabilities are limited; this is because complex logical reasoning tasks often require higher-level reasoning capabilities, such as multi-step reasoning and the establishment of reasoning chains; based on this, by utilizing external knowledge graphs to enhance the logical reasoning capabilities of large language models (LLMs), the control of logical paths in the large model reasoning process is achieved, and the large model's ability to perform mathematical logical reasoning is enhanced, which specifically includes the following steps.

[0225] Step 502: Use the original question provided by the user as input to the LLM model, and query the knowledge graph for reference data associated with the original question.

[0226] The original question may be “given that s in a = 1 / 2, and a is a first quadrant angle, how to solve tan a?”.

[0227] It should be noted that in order to enhance the logical reasoning ability of the large language model (LLM), reference data associated with the original question can be queried from the knowledge graph in the retrieval stage before using the large model for question reasoning. The retrieval stage is implemented based on the RAG generation function (rag_generate), which is the core of RAG processing and is used to provide the ability to search the knowledge base.

[0228] In the retrieval stage: you can use the rich knowledge contained in the pre-training of the big model itself, and use the ability to decompose the content of the specified question. Use the big model to select the document / knowledge node in the knowledge base that is most relevant to the original question. In the process of searching the knowledge base (i.e., the knowledge graph), you can find the knowledge required to solve the given problem (i.e., the knowledge node) through the hierarchical selection method of the tree search of multi-layer content. Figure 5 Data A1, data BN).

[0229] Based on this, the junior high school mathematics knowledge graph constructed using junior high school mathematics knowledge can be determined from multiple knowledge graphs according to the problem type (junior high school mathematics type) of the original problem;

[0230] Then, the original question is used as a query statement, and based on the query statement, multiple knowledge points related to the original question are queried from the junior high school mathematics knowledge graph as reference data (i.e. Figure 5 Data A1, data BN).

[0231] It should be noted that before using the LLM model for problem reasoning, it is necessary to build a knowledge base with a clear hierarchical tree structure through manual or machine-assisted methods during the data preparation stage.

[0232] For example, the manually determined knowledge graph framework can be input into the big model, and the knowledge learned by the big model can be filled into the knowledge graph framework to complete the construction of the knowledge graph.

[0233] The manually determined knowledge graph framework and the corresponding knowledge data can be input into the big model, and the big model can be used to fill the knowledge data into the knowledge graph framework to complete the construction of the knowledge graph.

[0234] Step 504: Input the original problem and reference data into the LLM large model to perform the first stage of problem reasoning, obtain the first stage of reasoning results (ie, reasoning result 1), and perform problem reduction based on the reasoning results.

[0235] It should be noted that the data processing method provided in this manual can complete the reasoning process of the problem through a multi-stage iteration. During this process, the original large model LLM will use the information of the previous reasoning to perform the reasoning of the current step (ie, the current stage); thereby converting the problem reasoning steps for the original problem into a more streamlined process in which a large model only needs to complete one step of reasoning through the current problem to complete the entire task reasoning.

[0236] Among them, the one-step reasoning refers to simplifying / transforming and solving one or more sub-problems (i.e. known conditions) in the original problem in the form of large model + too ls or large model CoT.

[0237] Specifically, in the combination stage of the question reasoning process, the original question can be combined with the retrieved document / knowledge point content to form the input of the generation model;

[0238] In the generation phase of the question reasoning process, this combined input is passed to the generative model, which generates the answer to the question or a new sub-question.

[0239] That is to say, after the above combined data is input into the LLM big model, the LLM big model will perform the first stage of problem reasoning on the original question based on the retrieved document / knowledge point content, and obtain the first stage of reasoning result (i.e., step-wise solution);

[0240] Furthermore, by using the LLM large model, problem reduction is performed according to the original problem and the reasoning results to obtain a new problem (ie, problem 2), which has the same answer as the original problem.

[0241] Among them, the problem reduction refers to using the text generation ability of the large model to replace and rewrite the known conditions in the original problem based on the original problem and the simplified conditions in the one-step reasoning process (i.e., the reasoning result), so as to obtain a new problem that is simpler than the original problem;

[0242] For example, let's take the above problem 1 as an example to explain the problem reduction; the original problem is "Given that sina=1 / 2, and a is a first quadrant angle, find tan a?"; The reasoning result obtained by performing one-step reasoning on it is: "Because sina=1 / 2, and a is a first quadrant angle, we can get a=30 degrees"; that is to say, this one-step reasoning is to perform problem reasoning on the sub-problem of "Given that sina=1 / 2, and a is a first quadrant angle" in the original problem, so as to obtain the corresponding reasoning result.

[0243] Then, based on the reasoning result, the sub-problem "Given that s in a = 1 / 2, and a is a first quadrant angle" in the original problem "Given that s in a = 1 / 2, and a is a first quadrant angle, find tan a?" can be replaced and rewritten, thereby converting the original problem into a new problem "Given that a = 30 degrees, find tan a?".

[0244] In addition, it should be noted that in the process of multi-stage problem reasoning, the large model needs to verify whether to continue the problem reasoning at each stage, so as to determine whether the reasoning is completed in a timely manner to avoid wasting computing resources due to repeated reasoning; the way the large model verifies whether to continue the problem reasoning can be seen in the following example:

[0245] For example, the original question may be “What is 1+2×3?”, and the corresponding sub-questions may be “What is 1+2?” and “What is 2×3?”. After the first stage of one-step reasoning on the original question, the reasoning result is determined to be “2×3 is equal to 6.”

[0246] After using the large model to determine that the reasoning result of one step of reasoning is "2×3 equals 6", the original question and the current reasoning result are input into the large model to verify whether to continue the question reasoning.

[0247] In the process of verifying whether to continue problem reasoning, the large model can determine whether the original problem contains multiple sub-problems, and whether there are sub-problems among the multiple sub-problems that can be reduced based on the reasoning results; if the original problem contains multiple sub-problems and there are sub-problems that can be reduced, it is determined that the problem reasoning needs to continue.

[0248] For example, when the large model determines that the current original problem contains two sub-problems, "What is 1+2 equal to" and "What is 2×3 equal to", and that the sub-problem exists in the sub-problem "What is 2×3 equal to" which can be reduced using the reasoning result "2×3 equals 6", it determines that it is necessary to continue problem reasoning.

[0249] When it is determined that the problem reasoning needs to be continued, the original problem is reduced based on the reasoning result to obtain a new problem, and the reasoning operation is continued based on the new problem until the data reasoning is completed.

[0250] Step 506: Use the new question as input for the second-stage question reasoning of the LLM large model, and query the reference data associated with the new question from the knowledge graph.

[0251] The execution method of this step is the same as that of the above step 502, and will not be described in detail here.

[0252] Step 508: Input the new problem and reference data into the LLM large model to perform the second stage problem reasoning, obtain the second stage reasoning result, and perform problem reduction based on the reasoning result.

[0253] Specifically, the large model is used to perform a step-wise reasoning on the new problem. After obtaining the reasoning result (step-wise solution), the reasoning result is used again to reduce the new problem to obtain a problem that is consistent with the answer to the new problem but is simpler.

[0254] Repeat the above step of reasoning and problem reduction until the most concise problem N is obtained.

[0255] For example, when the original problem is “Given that s in a=1 / 2, and a is a first quadrant angle, find tan a?”, the most simplified problem N may be “Given that a=30 degrees, find tan a?”.

[0256] For another example, when the original question may be “What is 1+2×3 equal to?”, the most simplified question N may be “What is 1+6 equal to?”.

[0257] Step 510: Question N is used as the input of the final stage of question reasoning of the LLM large model, and reference data associated with question N is queried from the knowledge graph (i.e. Figure 5 Data M1, data BN).

[0258] Step 512: Input the question N and the reference data into the LLM large model for question reasoning to obtain the answer to the question (ie, output).

[0259] Specifically, the above-mentioned one-step reasoning and problem reduction process is repeated until the answer (ie, reasoning result 1) is obtained through problem solving, and it is determined that the problem reasoning is completed.

[0260] It should be noted that the judgment conditions of the problem reasoning results can be:

[0261] During the multi-stage problem reasoning process, the large model needs to verify whether to continue the problem reasoning at each stage. During the verification process of whether to continue the problem reasoning, if it is determined that the original problem contains only one sub-problem and the sub-problem cannot be reduced, then it can be determined that the problem reasoning is over.

[0262] For example, when the large model determines that the current original problem contains a sub-problem of "what is 1+6 equal to", and that the sub-problem cannot be further reduced, it determines that the problem reasoning is finished.

[0263] It should be noted that in the retrieval stage of each iteration, combinations can be made based on the existing retrieval content and known conditions to achieve limited and diverse tentative answers.

[0264] For example, considering that the large model may select inaccurate reference data for problem reasoning based on reference data; therefore, in order to ensure the accuracy of the reasoning result, the accuracy of the reasoning result can be tested after the problem reasoning.

[0265] Among them, accuracy detection can be understood as: using the current reasoning result to reversely reason the original problem, and determining whether the initial data can be inferred based on the current reasoning result. If so, it is determined to meet the result accuracy detection, otherwise, it is determined to not meet the result accuracy detection.

[0266] When it is determined that the answer does not meet the accuracy test, it will be combined based on the existing retrieved content and known conditions. For example, if the retrieved mathematical formulas are a, b, and c, and it is determined that the problem reasoning effect of formulas a and b (i.e., reference data) is inaccurate, you can reselect formulas b and c to combine with the original question and continue to reason about the problem, thereby obtaining a more accurate reasoning result.

[0267] Based on the above content, it can be known that the data processing method in one or more embodiments of the present specification provides a solution for enhancing the logical reasoning of large models by using knowledge graphs. The technical effects of the data processing method are: 1. Reasoning path planning: by converting complex reasoning into multiple one-step reasonings, and using the node information of the knowledge graph used in the historical path to control the logical reasoning direction of the large model, a traceable / plannable reasoning path is achieved; 2. Controllable text generation: using the content of specific nodes in the knowledge graph as prompts, combined with the content of known conditions, the controllable generation of one-step reasoning is achieved, thereby improving the detectability of one-step reasoning; 3. Effectively improve the reasoning ability of the large model through multiple rounds of attempts.

[0268] For details, please refer to Figure 6 , Figure 6 is a schematic diagram of the LLM function in a data processing method provided in an embodiment of this specification, based on Figure 6 It can be seen that, firstly, this solution decomposes the original reasoning task into multiple one-step reasoning tasks, and introduces the knowledge graph content for targeted prompts, which effectively improves the one-step reasoning accuracy of the large model;

[0269] Secondly, this solution uses the node information in the knowledge graph and the known information of the original problem, and stores the connection between the known conditions and the knowledge node information through short-term memory to determine the reference data, realize the detection of the current reasoning path and the planning of the future path;

[0270] Finally, this solution introduces a one-step verification function to detect the quality and accuracy of the current reasoning path, and introduces a rollback method to achieve multiple attempts at reasoning for large models, thereby improving the accuracy of logical reasoning of large models.

[0271] in, Figure 6 The "plan" part means that the data processing model will store the conditions, historical reasoning results, and target data processing results in the memory during the reasoning process, and select data as needed for efficient reasoning during the reasoning process.

[0272] Figure 6The “feedback” part means that the data processing model can check the inference results at each stage of the inference process to determine whether they are correct, as well as perform summary and rollback operations.

[0273] Figure 6 The “reasoning” part means that during the reasoning process, the data processing model can access relevant tools to assist in performing data reasoning and observe whether the corresponding reasoning results are correct.

[0274] In summary, the data processing method in one or more embodiments of this specification achieves the technical effects of knowledge base retrieval form, reasoning path planning, controllable text generation, and improved large model reasoning capabilities.

[0275] Among them, the knowledge base retrieval form refers to taking the retrieval task as a subtask (agent) of the big model, and autonomously selecting knowledge points through the big model. This is quite different from the existing text similarity retrieval, and actually expands the entire rag into a multi-agnet interactive form.

[0276] Among them, reasoning path planning refers to converting complex reasoning into multiple one-step reasoning, and using the node information of the knowledge graph used in the historical path to control the logical reasoning direction of the large model, so as to achieve a traceable / plannable reasoning path.

[0277] Among them, controllable text generation refers to using the content of specific nodes in the knowledge graph as prompts, combined with the content of known conditions, to achieve controllable generation of one-step reasoning and improve the detectability of one-step reasoning.

[0278] Among them, improving the reasoning ability of large models means effectively improving the reasoning ability of large models through multiple rounds of attempts.

[0279] See also Figure 7 , Figure 7 A flowchart of a problem handling method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0280] Step 702: Determine an initial question text and a data storage map associated with the initial question text, and determine initial reference data associated with the initial question text from the data storage map.

[0281] Step 704: Using the data processing model, perform question reasoning on the initial question text and the initial reference data to obtain an initial question reasoning result.

[0282] Step 706: When it is determined that the initial question text satisfies the question reasoning conditions based on the initial question reasoning result, the initial question text is updated using the data processing model based on the initial question reasoning result to obtain an updated question text, wherein the amount of data in the updated question text is less than the amount of data in the initial question text.

[0283] Step 708: Determine the updated reference data associated with the updated question text from the data storage map, and use the data processing model to perform question reasoning on the updated question text and the updated reference data to obtain an updated question reasoning result.

[0284] Step 710: When it is determined that the updated question text does not satisfy the question reasoning condition according to the updated question reasoning result, the updated question reasoning result is determined as the target question processing result.

[0285] In one or more embodiments provided in this specification, determining the initial question text includes:

[0286] Receiving the initial question text sent by the client, wherein the initial question text is generated by the client according to the problem handling operation performed by the user on the problem handling page;

[0287] After determining, based on the update question reasoning result, that the update question text does not satisfy the question reasoning condition, the update question reasoning result is determined as the target question processing result, the method further includes:

[0288] The target problem processing result is sent to the client, so that the client displays the target problem processing result to the user based on the problem processing page.

[0289] In one or more embodiments of the present specification, the problem handling method provided, in the process of using the problem handling model to handle the problem, first, the initial reference data associated with the initial question text is determined from the data storage map associated with the initial question text, and the problem handling model is used to perform the first stage of problem reasoning on the initial question text and the initial reference data to obtain the initial problem reasoning result; because the initial reference data is used in the problem reasoning process, the problem handling model can perform problem reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial problem reasoning result that the initial question text meets the problem reasoning condition, the initial question text is updated according to the initial problem reasoning result, thereby obtaining an updated question text with a smaller amount of data than the initial question text; finally, the updated reference data associated with the updated question text is determined from the data storage map, and the problem handling model is used to perform the second stage of problem reasoning on the updated question text and the updated reference data with a smaller amount of data to obtain a more accurate updated problem reasoning result, thereby improving the problem reasoning efficiency of the problem handling model, and avoiding the problem of low reasoning efficiency of the neural network model in the process of processing more complex data with a larger amount of data;

[0290] Moreover, when it is determined according to the update question reasoning result that the update question text does not meet the question reasoning conditions, the question reasoning is terminated and the update question reasoning result is determined as the target question processing result, thereby meeting the need to use neural network models to process various types of problems in practical applications.

[0291] The above is an illustrative solution of a problem solving method of this embodiment. It should be noted that the technical solution of the problem solving method and the technical solution of the above data processing method belong to the same concept, and the details not described in detail in the technical solution of the problem solving method can be referred to the description of the technical solution of the above data processing method.

[0292] In one or more embodiments provided in this specification, another data processing method is provided, which specifically includes the following steps.

[0293] The other data processing method is applied to the server, comprising:

[0294] Receive initial data sent by a client, determine a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map;

[0295] Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain an initial reasoning result;

[0296] When it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, the initial data is updated using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data;

[0297] Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result;

[0298] When it is determined according to the update reasoning result that the update data does not satisfy the data reasoning condition, determining the update reasoning result as the target data processing result;

[0299] The target data processing result is sent to the client.

[0300] Among them, the server is a server applied by another data processing method in one or more embodiments of this specification, and the server may include but is not limited to a server, a server cluster, a data center, a cloud server, etc.; the server can be connected to one or more clients through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The clients here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc.; the client can interact with the user through a user interaction interface to realize the call of the large model, thereby realizing the method provided in the embodiments of this specification.

[0301] For example, the user can send a question (i.e., initial data) to the server through the client. After receiving the question, the server queries the reference data associated with the question (i.e., initial reference data) from the knowledge graph (i.e., data storage graph), inputs the question and reference data into the LLM large model (i.e., data processing model), and uses the large model to perform one-step reasoning on the current question. After one-step reasoning, the reasoning result of the first stage (i.e., initial reasoning result) is obtained, and then a new question (i.e., updated data) is obtained using the question and reasoning result; the operation of querying the reference data associated with the new question from the knowledge graph is continued, and the large model is used to perform one-step reasoning on the new question and the reference data to obtain the reasoning result of the second stage, and again based on the reasoning result and the new question, a question that is consistent with the answer to the new question but simpler is obtained; this process is repeated until the answer can be obtained through question solving, and the answer corresponding to the user question (i.e., target data processing result) is obtained; and then the answer is sent to the client. In the above steps, by converting the long sequence reasoning process in the original complex problem into multiple short sequence reasoning processes, the reasoning cost of the large model is significantly reduced; and, combined with the structured knowledge of the knowledge graph, the large model can better plan the reasoning path and improve the reasoning efficiency of the large model.

[0302] Another data processing method provided by one or more embodiments of the present specification is that in the process of using a data processing model to process data, first, the initial data sent by the client is received, the initial reference data associated with the initial data is determined from the data storage map associated with the initial data, and the data processing model is used to perform the first stage of data reasoning on the initial data and the initial reference data to obtain the initial reasoning result. Since the initial reference data is used in the process of data reasoning, the data processing model can perform data reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, the initial data is updated according to the initial reasoning result, thereby obtaining updated data with a smaller amount of data than the initial data; finally, the updated reference data associated with the updated data is determined from the data storage map, and the data processing model is used to perform the second stage of data reasoning on the updated data with a smaller amount of data and the updated reference data, thereby obtaining a more accurate updated reasoning result, thereby improving the data reasoning efficiency of the data processing model, and avoiding the problem of low reasoning efficiency in the process of processing more complex and large-volume data by the neural network model.

[0303] Moreover, when it is determined based on the updated reasoning result that the updated data does not meet the data reasoning conditions, the data reasoning is terminated, the updated reasoning result is determined as the target data processing result, and the target data processing result is sent to the client, thereby meeting the need to use neural network models to process various types of data in practical applications.

[0304] The above is a schematic scheme of another data processing method of this embodiment. It should be noted that the technical scheme of the other data processing method and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the other data processing method that are not described in detail can all be referred to the description of the technical scheme of the above data processing method.

[0305] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 8 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 8 As shown, the device comprises:

[0306] A data determination module 802 is configured to determine initial data and a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map;

[0307] A first result determination module 804 is configured to perform data reasoning on the initial data and the initial reference data using a data processing model to obtain an initial reasoning result;

[0308] The data updating module 806 is configured to update the initial data using the data processing model according to the initial reasoning result to obtain updated data when it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, wherein the data quantity of the updated data is less than the data quantity of the initial data;

[0309] A second result determination module 808 is configured to determine the update reference data associated with the update data from the data storage map, and use the data processing model to perform data reasoning on the update data and the update reference data to obtain an update reasoning result;

[0310] The third result determination module 810 is configured to determine the updated reasoning result as the target data processing result when it is determined that the updated data does not satisfy the data reasoning condition according to the updated reasoning result.

[0311] Optionally, the data determination module 802 is further configured to:

[0312] Determine initial data and multiple candidate data storage maps;

[0313] Based on the data type information of the initial data, a data storage map associated with the initial data is determined from the multiple candidate data storage maps.

[0314] Optionally, the data determination module 802 is further configured to:

[0315] Determine the initial data as a data query statement;

[0316] Based on the data query statement, query and obtain query data corresponding to the data query statement from the data storage graph;

[0317] The query data is determined as the initial reference data associated with the initial data.

[0318] Optionally, the query data is multiple;

[0319] The data determination module 802 is further configured to:

[0320] determining the plurality of query data as a plurality of candidate reference data;

[0321] The initial reference data associated with the initial data is determined from the multiple candidate reference data using the data processing model, wherein the data processing model is a machine learning model.

[0322] Optionally, the first result determination module 804 is further configured to:

[0323] Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain a reasoning result to be detected;

[0324] The data processing model is used to perform a result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested passes the result accuracy test, the inference result to be tested is determined as the initial inference result.

[0325] Optionally, the first result determination module 804 is further configured to:

[0326] Using the data processing model, performing a result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested does not pass the result accuracy test, determining a plurality of candidate reference data associated with the initial data, wherein the initial reference data is determined from the plurality of candidate reference data;

[0327] Determining, using the data processing model, updated initial reference data associated with the initial data from the plurality of candidate reference data, wherein the updated initial reference data is different from the initial reference data;

[0328] The updated initial reference data is determined as the initial reference data, and the step of using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be tested is continued until it is determined that the reasoning result to be tested meets the result accuracy test.

[0329] Optionally, the data processing device further includes a graph construction module configured to:

[0330] Determine the graph structure to be constructed and / or the graph data to be constructed;

[0331] The graph construction model is utilized to construct the graph according to the graph structure to be constructed and / or the graph data to be constructed, so as to obtain the data storage graph.

[0332] Optionally, the second result determination module 808 is further configured to:

[0333] inputting the initial inference result, the initial reference data, and the initial data into the data processing model;

[0334] Using the data processing model, determining a plurality of initial sub-data contained in the initial data, and determining, from the plurality of initial sub-data, sub-data to be updated that is associated with the initial reasoning result and the initial reference data;

[0335] The data processing model is used to update the sub-data to be updated in the initial data based on the initial reasoning result and the initial reference data to obtain the updated data.

[0336] Optionally, the first result determination module 804 is further configured to:

[0337] Inputting the initial data and the initial reference data into the data processing model, performing data segmentation on the initial data using the data processing model, and determining a plurality of initial sub-data contained in the initial data;

[0338] The sub-data to be inferred is determined from the multiple initial sub-data, and semantic reasoning is performed on the sub-data to be inferred according to the initial reference data to obtain the initial reasoning result.

[0339] Optionally, the initial data is an initial question text, and the initial reasoning result is an initial question reasoning result;

[0340] Optionally, the data updating module 806 is further configured to:

[0341] When it is determined that the initial question text meets the question reasoning condition according to the initial question reasoning result, the initial question text is updated by using the data processing model according to the initial question reasoning result to obtain an updated question text.

[0342] Optionally, the data determination module 802 is further configured to:

[0343] Receiving the initial data sent by the client, wherein the initial data is generated by the client according to a data processing operation performed by the user on the data processing page;

[0344] The data processing device further includes a result sending module, which is configured to:

[0345] The target data processing result is sent to the client, so that the client displays the target data processing result to the user based on the data processing page.

[0346] The data processing device provided in one or more embodiments of the present specification, in the process of using the data processing model to perform data processing, first, determines the initial reference data associated with the initial data from the data storage map associated with the initial data, and uses the data processing model to perform the first stage of data reasoning on the initial data and the initial reference data to obtain the initial reasoning result. Since the initial reference data is used in the process of data reasoning, the data processing model can perform data reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, the initial data is updated according to the initial reasoning result, thereby obtaining updated data with a smaller amount of data than the initial data; finally, the updated reference data associated with the updated data is determined from the data storage map, and the data processing model is used to perform the second stage of data reasoning on the updated data with a smaller amount of data and the updated reference data, thereby obtaining a more accurate updated reasoning result, thereby improving the data reasoning efficiency of the data processing model, and avoiding the problem of low reasoning efficiency in the process of the neural network model processing more complex and large data.

[0347] Moreover, when it is determined based on the updated reasoning result that the updated data does not meet the data reasoning conditions, the data reasoning is terminated and the updated reasoning result is determined as the target data processing result, thereby meeting the need to use neural network models to process various types of data in practical applications.

[0348] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0349] Corresponding to the above method embodiment, this specification also provides a problem handling device embodiment, Fig. 9 FIG. 1 is a schematic diagram showing a structure of a problem handling device provided by an embodiment of the present specification. Fig. 9 As shown, the device comprises:

[0350] The data determination module 902 is configured to determine an initial question text and a data storage map associated with the initial question text, and determine initial reference data associated with the initial question text from the data storage map;

[0351] The first result determination module 904 is configured to use a data processing model to perform question reasoning on the initial question text and the initial reference data to obtain an initial question reasoning result;

[0352] The question updating module 906 is configured to, when it is determined that the initial question text meets the question reasoning condition according to the initial question reasoning result, update the initial question text according to the initial question reasoning result using the data processing model to obtain an updated question text, wherein the data amount of the updated question text is less than the data amount of the initial question text;

[0353] The second result determination module 908 is configured to determine the update reference data associated with the update question text from the data storage map, and use the data processing model to perform question reasoning on the update question text and the update reference data to obtain an update question reasoning result;

[0354] The third result determination module 910 is configured to determine the update question reasoning result as the target question processing result when it is determined that the update question text does not meet the question reasoning condition according to the update question reasoning result.

[0355] Optionally, the data determination module 902 is further configured to:

[0356] Receiving the initial question text sent by the client, wherein the initial question text is generated by the client according to the problem handling operation performed by the user on the problem handling page;

[0357] The problem handling device further includes a result sending module, which is configured to:

[0358] The target problem processing result is sent to the client, so that the client displays the target problem processing result to the user based on the problem processing page.

[0359] In one or more embodiments of the present specification, the problem processing device provided, in the process of using the problem processing model to process the problem, first, determines the initial reference data associated with the initial problem text from the data storage map associated with the initial problem text, and uses the problem processing model to perform the first stage of problem reasoning on the initial problem text and the initial reference data to obtain the initial problem reasoning result; because the initial reference data is used in the problem reasoning process, the problem processing model can perform problem reasoning based on rich data, thereby improving the accuracy of the reasoning result; secondly, when it is determined according to the initial problem reasoning result that the initial problem text meets the problem reasoning condition, the initial problem text is updated according to the initial problem reasoning result, thereby obtaining an updated problem text with a smaller amount of data than the initial problem text; finally, determines the updated reference data associated with the updated problem text from the data storage map, and uses the problem processing model to perform the second stage of problem reasoning on the updated problem text and the updated reference data with a smaller amount of data, thereby obtaining a more accurate updated problem reasoning result, thereby improving the problem reasoning efficiency of the problem processing model, and avoiding the problem of low reasoning efficiency existing in the process of the neural network model processing more complex data with a larger amount of data;

[0360] Moreover, when it is determined according to the update question reasoning result that the update question text does not meet the question reasoning conditions, the question reasoning is terminated and the update question reasoning result is determined as the target question processing result, thereby meeting the need to use neural network models to process various types of problems in practical applications.

[0361] The above is a schematic scheme of a problem solving device of this embodiment. It should be noted that the technical scheme of the problem solving device and the technical scheme of the problem solving method described above are of the same concept, and the details not described in detail in the technical scheme of the problem solving device can be found in the description of the technical scheme of the problem solving method described above.

[0362] Fig.10 The block diagram of a computing device 1000 according to an embodiment of the present specification is shown. The components of the computing device 1000 include but are not limited to a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and the database 1050 is used to store data.

[0363] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface, wired or wireless (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0364] In one embodiment of the present specification, the above components of the computing device 1000 and Fig.10 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.10 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0365] The computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1000 may also be a mobile or stationary server.

[0366] The processor 1020 is used to execute the following computer program / instruction, which implements the steps of the above two data processing methods or problem processing methods when executed by the processor.

[0367] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the above two data processing methods or problem processing method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the above two data processing methods or problem processing method embodiments.

[0368] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above two data processing methods or problem processing methods when executed by a processor.

[0369] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the above two data processing methods or problem processing method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the above two data processing methods or problem processing method embodiments.

[0370] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which implements the steps of the above two data processing methods or problem processing methods when executed by a processor.

[0371] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned data processing method or problem processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical schemes of the above-mentioned two data processing methods or problem processing methods.

[0372] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0373] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0374] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0375] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0376] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Determine initial data and a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map; Using a data processing model, performing data reasoning on the initial data and the initial reference data to obtain an initial reasoning result; When it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, the initial data is updated using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data; Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result; When it is determined according to the update reasoning result that the update data does not satisfy the data reasoning condition, the update reasoning result is determined as the target data processing result.

2. The data processing method according to claim 1, wherein the determining of the initial data and the data storage map associated with the initial data comprises: Determine the initial data and a plurality of candidate data storage maps; Based on the data type information of the initial data, a data storage map associated with the initial data is determined from the multiple candidate data storage maps.

3. The data processing method according to claim 1, wherein determining the initial reference data associated with the initial data from the data storage map comprises: Determine the initial data as a data query statement; Based on the data query statement, query and obtain query data corresponding to the data query statement from the data storage graph; The query data is determined as the initial reference data associated with the initial data.

4. The data processing method according to claim 3, wherein the query data is multiple; The determining the query data as the initial reference data associated with the initial data comprises: determining the plurality of query data as a plurality of candidate reference data; The initial reference data associated with the initial data is determined from the multiple candidate reference data using the data processing model, wherein the data processing model is a machine learning model.

5. The data processing method according to claim 1, wherein the using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain an initial reasoning result comprises: Using the data processing model, performing data reasoning on the initial data and the initial reference data to obtain a reasoning result to be detected; The data processing model is used to perform a result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested passes the result accuracy test, the inference result to be tested is determined as the initial inference result.

6. The data processing method according to claim 5, wherein after using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be detected, the method further comprises: Using the data processing model, performing the result accuracy test on the inference result to be tested, and when it is determined that the inference result to be tested does not pass the result accuracy test, determining a plurality of candidate reference data associated with the initial data, wherein the initial reference data is determined from the plurality of candidate reference data; Determining, using the data processing model, updated initial reference data associated with the initial data from the plurality of candidate reference data, wherein the updated initial reference data is different from the initial reference data; The updated initial reference data is determined as the initial reference data, and the step of using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain the reasoning result to be tested is continued until it is determined that the reasoning result to be tested meets the result accuracy test.

7. The data processing method according to claim 1, before determining the initial data and the data storage map associated with the initial data, further comprising: Determine the graph structure to be constructed and / or the graph data to be constructed; The graph construction model is utilized to construct the graph according to the graph structure to be constructed and / or the graph data to be constructed, so as to obtain the data storage graph.

8. The data processing method according to claim 1, wherein the updating of the initial data using the data processing model according to the initial inference result to obtain updated data comprises: inputting the initial inference result, the initial reference data, and the initial data into the data processing model; Using the data processing model, determining a plurality of initial sub-data contained in the initial data, and determining, from the plurality of initial sub-data, sub-data to be updated that is associated with the initial reasoning result and the initial reference data; The data processing model is used to update the sub-data to be updated in the initial data based on the initial reasoning result and the initial reference data to obtain the updated data.

9. The data processing method according to claim 1, wherein the using the data processing model to perform data reasoning on the initial data and the initial reference data to obtain an initial reasoning result comprises: Inputting the initial data and the initial reference data into the data processing model, performing data segmentation on the initial data using the data processing model, and determining a plurality of initial sub-data contained in the initial data; The sub-data to be inferred is determined from the multiple initial sub-data, and semantic reasoning is performed on the sub-data to be inferred according to the initial reference data to obtain the initial reasoning result.

10. The data processing method according to any one of claims 1 to 9, wherein the initial data is an initial question text, and the initial reasoning result is an initial question reasoning result; When it is determined according to the initial reasoning result that the initial data meets the data reasoning condition, updating the initial data using the data processing model according to the initial reasoning result to obtain updated data includes: When it is determined that the initial question text meets the question reasoning condition according to the initial question reasoning result, the initial question text is updated by using the data processing model according to the initial question reasoning result to obtain an updated question text.

11. A problem handling method, comprising: Determine an initial question text and a data storage map associated with the initial question text, and determine initial reference data associated with the initial question text from the data storage map; Using the data processing model, performing question reasoning on the initial question text and the initial reference data to obtain an initial question reasoning result; When it is determined that the initial question text satisfies the question reasoning condition according to the initial question reasoning result, the initial question text is updated by using the data processing model according to the initial question reasoning result to obtain an updated question text, wherein the amount of data in the updated question text is less than the amount of data in the initial question text; Determine the update reference data associated with the update question text from the data storage map, and use the data processing model to perform question reasoning on the update question text and the update reference data to obtain an update question reasoning result; When it is determined according to the update question reasoning result that the update question text does not satisfy the question reasoning condition, the update question reasoning result is determined as the target question processing result.

12. The problem handling method according to claim 11, wherein determining the initial problem text comprises: Receiving the initial question text sent by the client, wherein the initial question text is generated by the client according to the problem handling operation performed by the user on the problem handling page; After determining, based on the update question reasoning result, that the update question text does not satisfy the question reasoning condition, the update question reasoning result is determined as the target question processing result, the method further includes: The target problem processing result is sent to the client, so that the client displays the target problem processing result to the user based on the problem processing page.

13. A data processing method, applied to a server, comprising: Receive initial data sent by a client, determine a data storage map associated with the initial data, and determine initial reference data associated with the initial data from the data storage map; Using a data processing model, performing data reasoning on the initial data and the initial reference data to obtain an initial reasoning result; When it is determined that the initial data meets the data reasoning condition according to the initial reasoning result, the initial data is updated using the data processing model according to the initial reasoning result to obtain updated data, wherein the data quantity of the updated data is less than the data quantity of the initial data; Determine updated reference data associated with the updated data from the data storage map, and use the data processing model to perform data reasoning on the updated data and the updated reference data to obtain an updated reasoning result; When it is determined according to the update reasoning result that the update data does not satisfy the data reasoning condition, determining the update reasoning result as the target data processing result; The target data processing result is sent to the client.

14. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the data processing method described in any one of claims 1 to 10, the problem processing method described in any one of claims 11 to 12, or the data processing method described in claim 13 are implemented.

15. A computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the data processing method described in any one of claims 1 to 10, the problem processing method described in any one of claims 11 to 12, or the data processing method described in claim 13.

16. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the data processing method according to any one of claims 1 to 10, the problem processing method according to any one of claims 11 to 12, or the data processing method according to claim 13.

Citation Information

Patent Citations

  • Method and device for obtaining result, electronic equipment and storage medium

    CN115203366A

  • Fine adjustment method, system and equipment based on large language model and medium

    CN117290480A

  • Prompt text generation method and device, electronic equipment and storage medium

    CN117689027A

  • Crowdsource reasoning process to facilitate question answering

    US9373086B1