Data processing method and device and electronic equipment

Abstract

The invention relates to the technical field of big data, in particular to a data processing method and device and electronic equipment, and the method comprises the steps: obtaining a to-be-answered target question; screening out a target data set for querying the answer of the target question and an auxiliary information set for assisting in querying the answer of the target question from a pre-constructed knowledge base, wherein the auxiliary information set comprises a business information set related to the target question and / or a statement construction information set related to a query statement of the answer of the target question; splicing information extracted from the target data set and the auxiliary information set to obtain a cue word, and determining a target query statement based on the target problem and the cue word; and executing the target query statement in the target data set, and taking an obtained result as an answer to the target question. The accuracy is high.

Description

Technical Field

[0001] The present disclosure relates to the field of big data technology, and in particular to a data processing method, device and electronic device. Background Art

[0002] With the development of big data technology, it is possible to analyze massive amounts of data to extract valuable information. However, with the exponential growth of data and increasingly complex business scenarios, many data analysis methods are still lacking in the analysis of existing data, resulting in the need to improve the accuracy and reliability of the extracted information. Summary of the invention

[0003] In view of this, the present disclosure provides a data processing method, a data processing device, an electronic device, a computer-readable storage medium, and a computer program product with high accuracy and reliability.

[0004] In a first aspect, the present disclosure provides a data processing method, the method comprising:

[0005] Get the target question to be answered;

[0006] Filtering out a target data set for querying an answer to a target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to a query statement of the answer to the target question;

[0007] splicing the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determining a target query sentence based on the target question and the prompt words;

[0008] The target query statement is executed in the target data set, and the obtained result is used as the answer to the target question.

[0009] In a second aspect, the present disclosure provides a data processing system, the device comprising:

[0010] A question acquisition module is used to obtain the target question to be answered;

[0011] An information acquisition module, used to filter out a target data set for querying an answer to a target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to a query statement for the answer to the target question;

[0012] A sentence generation module, used for splicing the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determining a target query sentence based on the target question and the prompt words;

[0013] The statement execution module is used to execute the target query statement in the target data set and use the obtained result as the answer to the target question.

[0014] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the above method.

[0015] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the above method.

[0016] In the technical solutions of some embodiments of the present disclosure, a target data set for querying the answer to the target question and an auxiliary information set for assisting the query of the answer to the target question are screened in a pre-built knowledge base, and then the information extracted from the target data set and the auxiliary information set are spliced ​​into prompt words. In this way, the prompt words can include business information related to the target question, or sentence construction information related to the query sentence of the answer to the target question. Therefore, with the prompt words as a guide, the generated target query sentence can be more accurate and in line with the actual business scenario. Furthermore, the answer to the target question obtained based on the target query sentence query is correspondingly more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flowchart of intelligent data analysis methods in some technologies;

[0019] Figure 2 is a module schematic diagram of a data processing system provided by an embodiment of the present disclosure;

[0020] Figure 3 is a flowchart of a data processing method provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of a question answering process provided by an embodiment of the present disclosure

[0022] Figure 5 is a schematic diagram of a process for screening a target data set provided by an embodiment of the present disclosure;

[0023] Figure 6 is a schematic diagram of a process for screening a business information set provided by an embodiment of the present disclosure;

[0024] Figure 7 It is a schematic diagram of a process of filtering a statement construction information set from a statement construction knowledge base provided by an embodiment of the present disclosure;

[0025] Figure 8 It is a schematic diagram of a process of filtering statements from an example knowledge base to build an information set provided by an embodiment of the present disclosure;

[0026] Fig. 9 is a module schematic diagram of a data processing device provided by some embodiments of the present disclosure;

[0027] Fig.10 It is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0031] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0032] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0033] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the types, scopes of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0034] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0036] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] Combined with reference Figure 1 In the intelligent data analysis method based on the sentence generation model, the user's question can be input into the trained sentence generation model, and the sentence generation model outputs the query statement (such as SQL statement) used to query the answer to the question. Then, the query statement output by the sentence generation model is executed in the business data storage system (such as business database, business data warehouse, etc.) to get the answer to the question. In this method, the user only needs to input the question, without having to master the relevant knowledge of the query statement, and can also query the data he needs from the business data storage system, which is more applicable.

[0038] For ease of understanding, let's take an example. During subway operations, the number of people arriving and leaving a station is a key operational data indicator. When analyzing subway data, users may analyze the number of people arriving and leaving a station on a specific day or during a specific period of time every day. However, the business data storage system stores the number of people arriving and leaving a station every minute. Therefore, it is necessary to execute specific query statements in the business data storage system to obtain the number of people arriving and leaving a station on a specific day or during a specific period of time every day. In conventional data analysis methods, users manually enter query statements to obtain corresponding data query results. This method requires too high skills from users and is not very applicable. In view of this, an intelligent data analysis method based on a statement generation model can be used to solve this problem. Specifically, in the intelligent data analysis method, users only need to enter questions similar to the following:

[0039] - "Zhongyuan Station, July 22, 2024, what is the volume of incoming traffic?"

[0040] - "What is the number of trains entering Zhongyuan Station at 8 a.m. on July 22, 2024?"

[0041] - "What is the number of trains entering Changning Station from 8:11am to 9:26am on July 22, 2024?"

[0042] - "Changning Station, what is the number of passengers entering the station during the morning peak (7:00 to 9:00) on July 22, 2024?"

[0043] - "What will be the passenger intensity of Line 2 on July 22, 2024?"

[0044] - "What will be the number of passengers on Line 2 on July 22, 2024?"

[0045] …

[0046] The sentence generation model can output query statements for finding answers to questions based on the questions entered by the user. Based on the query statements output by the sentence generation model, the corresponding query statements can be executed in the business data storage system to get the answers to the user's questions. This greatly reduces the skill threshold requirements for data analysis and makes it more applicable.

[0047] At present, in the case of relatively simple business scenarios, the statement generation model can output relatively accurate query statements. However, with the exponential growth of data and increasingly complex business scenarios, the query statements output by the statement generation model are not accurate enough, resulting in the accuracy and reliability of the query data needing to be improved.

[0048] In view of this, the present disclosure provides a data processing method, which can improve the accuracy of query statements, and further improve the accuracy and reliability of the data obtained by the query. Specifically, the data processing method can be applied to a data processing system, or an electronic device running the data processing system. The electronic device may include but is not limited to a tablet computer, a laptop computer, a desktop computer, etc. Figure 2 , which is a module diagram of a data processing system provided by an embodiment of the present disclosure. Figure 2 In the data processing system, a question receiving interface for receiving user questions, a prompt word generation module for generating prompt words according to user questions, a sentence generation model for outputting query sentences according to user questions and prompt words, a data query module for executing data query operations in a target data set according to the query sentences output by the sentence generation model, and an answer display interface for displaying data query results. The target data set is extracted from a business data storage system. For details, please refer to the specific description of the data processing method. Figure 3 , which is a flow chart of a data processing method provided by an embodiment of the present disclosure. Figure 3 In the data processing method, the data processing method comprises the following steps:

[0049] Step S301, obtaining a target question to be answered.

[0050] The target question is the user question received through the question receiving interface.

[0051] Step S302, screen out a target data set for querying the answer to the target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to the query statement of the answer to the target question.

[0052] In this embodiment, the knowledge base may include a metadata knowledge base, an industry knowledge base, a statement construction knowledge base, and an example knowledge base.

[0053] The metadata knowledge base may store one or more metadata items and description information of the metadata items. Metadata items may refer to data items obtained from statistics in the business data storage system. For example, suppose that the business data storage system stores the number of people entering and leaving each station every minute. Then, using hours and stations as statistical dimensions, the number of people entering and leaving each station per hour can be statistically obtained, and this statistical result can be used as a metadata item A. For another example, taking all stations as a whole and using days as statistical dimensions, the sum of the number of people entering all stations per day can be statistically obtained, and this statistical result can be used as a metadata item B. Each metadata item may correspond to a database table. For example, the database table corresponding to the above metadata item A may be shown in Table 1.

[0054] Table 1 Metadata items corresponding to the database table

[0055] station time Station volume Outbound volume Changning Station September 10, 2024 15:00 300 500 …… …… …… …… Zhongyuan Station September 13, 2024 14:00 50 60

[0056] The description information of the metadata item is used to provide a general description of the metadata item. For example, the description information of the metadata item A can be the number of passengers entering and leaving each station per hour. In this way, based on the description information of the metadata item, the characteristics of the metadata item can be known.

[0057] The answer to the target question can be obtained based on at least some metadata items. For example, assuming the target question is "How many people entered Changning Station from 15:00 to 17:00 on September 10, 2024?" Then a query operation can be performed in the database table corresponding to the above metadata item A to obtain the number of people entering Changning Station from 15:00 to 17:00 on September 10, 2024. In some embodiments of the present disclosure, one or more metadata items are first obtained from the business data storage system, and a metadata knowledge base is constructed based on the metadata items obtained by statistics, which can reduce data query operations and improve data query speed in subsequent question answering.

[0058] Based on the above description, in some embodiments, before executing the data processing method of the present disclosure, a metadata knowledge base may be constructed based on the following method:

[0059] 1) Collecting business data from a business data storage system and performing pre-processing operations such as data cleaning, standardization and formatting on the business data. The business data may include, for example, business indicators reflecting the business activities or operating conditions of the enterprise;

[0060] 2) Extract metadata items from the preprocessed business data according to actual business needs;

[0061] 3) Processing metadata items using language models, etc., to generate description information for each metadata item;

[0062] 4) Based on each metadata item and the description information of the metadata item, a metadata knowledge base is constructed.

[0063] The industry knowledge base can store business logic, standard operating procedures, etc. corresponding to each business scenario. Based on the data in the industry knowledge base, the accuracy of the query statement can be improved. For example, the industry knowledge base can define the time period of the morning peak as 7 am to 10 am. Then, when the statement generation model receives a target question including the morning peak (such as "What is the number of passengers entering Changning Station during the morning peak?"), it can convert the morning peak to 7 am to 10 am, and generate the corresponding query statement to obtain the query result related to the morning peak. It is understandable that if there is no definition of the morning peak, it is difficult for the statement generation model to determine the specific time period of the morning peak, resulting in inaccurate generated query statements. For example, the statement generation model may use 7 am to 9 am as the time period of the morning peak, so that the generated query statement is to query the data from 7 am to 9 am. Obviously, the answer to the target question obtained based on such a query statement is definitely inaccurate. In summary, based on the information in the industry knowledge base, the probability of errors in query statements can be reduced, and the accuracy of query statements can be improved.

[0064] Based on the above description, in some embodiments, before executing the data processing method of the present disclosure, an industry knowledge base may be constructed based on the following method:

[0065] 1) Combine the knowledge of industry experts and actual business data to extract knowledge such as business logic, business operation process and business rules;

[0066] 2) Convert the extracted knowledge into structured information, such as saving the extracted knowledge in the form of knowledge graph or semantic network;

[0067] 3) Use language models to understand the above structured information and reason about semantic relationships, and generate business logic, standard operating procedures, etc. corresponding to each business scenario;

[0068] 4) Build an industry knowledge base based on the business logic, standard operating procedures, etc. corresponding to each business scenario.

[0069] The statement construction knowledge base can store statement construction information corresponding to different business scenarios. Among them, the statement construction information can also be called statement construction logic, which is used to specify the rules that need to be followed when generating query statements. For example, assuming that the business data storage system of type A does not support the XX field, the following statement construction information can be set in the statement construction knowledge base: "When performing data query in the business data storage system of type A, the query statement cannot include the XX field." In this way, the problem of failure to execute query statements in the business data storage system of type A can be prevented. For another example, assuming that when querying the amount of incoming stations, generating query statements according to rule B can have a higher query efficiency, the following statement construction information can be set in the statement construction knowledge base: "Generate a query statement for querying the amount of incoming stations according to rule B." In this way, it is ensured that the generated query basis can have a higher query efficiency. In summary, based on the information in the statement construction knowledge base, the probability of errors in query statements can be reduced, and the accuracy and execution efficiency of query statements can be improved.

[0070] Based on the above description, in some embodiments, before executing the data processing method of the present disclosure, a statement construction knowledge base may be constructed based on the following method:

[0071] 1) Collect a large number of common query statements, which can come from historical data analysis tasks, expert experience summaries, and industry standards;

[0072] 2) Based on the collected query statements, determine the statement construction information corresponding to different business scenarios. In addition to the conventional basic query statements, the statement construction information may also include multi-table joins, complex condition screening, aggregation operations, etc.;

[0073] 3) After verifying the statement construction information in each business scenario, a statement construction knowledge base is constructed based on the verified statement construction information.

[0074] The example knowledge base may store historical questions, historical query statements used when querying answers to historical questions, and feedback information for historical query statements. Among them, the feedback information may be input by the user through the answer display interface, which is used to characterize whether the historical query statement is accurate. The data in the example knowledge base can be used as a reference for constructing query statements. For example, assume that the target question A1 is similar to the historical question A2. In the example knowledge base, if the historical query statement corresponding to the historical question A2 is determined to be relatively accurate based on the feedback information, then when constructing the query statement for the target question A1, the historical query statement corresponding to the historical question A2 can be referred to. On the contrary, if the historical query statement corresponding to the historical question A2 is determined to be inaccurate based on the feedback information, then when constructing the query statement for the target question A1, the problems in the historical query statement corresponding to the historical question A2 can be avoided. In this way, the probability of errors in the query statement is reduced, and the accuracy and execution efficiency of the query statement are improved.

[0075] In this embodiment, the feedback information can be input by the user through the answer display interface according to the matching degree between the answer and the question. Specifically, in the answer display interface, a first button representing an accurate answer and a second button representing an inaccurate answer can be set. When the user queries the answer to a question A3, if the answer obtained matches the question A3, the user can click the first button. In response to the first button being triggered by the user, it can be considered that the query statement M corresponding to the question A3 is relatively accurate. Therefore, the question A3 can be taken as a historical question, and the question A3, the query statement M and the feedback information for the query statement M can be saved in the example knowledge base, so that when a question similar to the question A3 is received later, the query statement M can be used as a reference. On the contrary, when the user queries the answer to a question A3, if the answer obtained does not match the question A3, the user can click the second button. In response to the second button being triggered by the user, it can be considered that the query statement M corresponding to question A3 is inaccurate. Therefore, question A3 can be treated as a historical question, and question A3, query statement M and feedback information for query statement M can be saved in the example knowledge base, so that when a question similar to question A3 is received in the future, the problem in the query statement M can be avoided.

[0076] Based on the above description, the example knowledge base can be constructed in the process of executing the data processing method of the present invention. For example, during the operation of the system, the user's feedback on the query results (such as satisfaction with the query results, correction suggestions, etc.) and the success or failure of the system to execute the query statement and other information can be recorded, and then this information can be marked as positive or negative examples and stored in the example knowledge base. In some embodiments, based on the constructed example knowledge base, the data in the example knowledge base can also be analyzed and classified through language models, etc., so as to extract the key factors that affect the success or failure of the answer query. These key factors may include but are not limited to the structure of the query statement, the setting of the query conditions, etc. These key factors can be saved in the statement construction knowledge base for use as a reference for subsequent construction of query statements.

[0077] The sample knowledge base not only records successful and failed query instances, but also contains optimization suggestions and improvement strategies derived from the system’s analysis. In future query tasks of the system, the sample knowledge base can provide references to help the system automatically correct potential query errors and optimize query strategies based on past experience, thereby improving the success rate of queries and the accuracy of results.

[0078] To sum up, after obtaining the target problem, you can perform the following operations according to the target problem:

[0079] 1) Filter the target data set for querying the answer to the target question from the metadata knowledge base. The target data set may include target metadata items related to the target question in the metadata knowledge base. For example, assuming that there are metadata items a, b, and c in the metadata knowledge base, when answering the target question, it is necessary to query data in the database table corresponding to metadata item a and metadata item b, and it is not necessary to query data in the database table corresponding to metadata item c. Then, the target data set can be constructed based on metadata items a and b. The constructed target data set may not include metadata item c.

[0080] 2) Filter the business information set related to the target question from the industry knowledge base. For example, if the target question is "What is the number of outbound and inbound passengers at Changning Station during the morning peak?" Then the business information set can be constructed based on the definition of the morning peak in the industry knowledge base. The constructed business information set may not include the definition of the evening peak in the industry knowledge base.

[0081] 3) Filter the sentence construction information set related to the query sentence of the target question answer from the sentence construction knowledge base and the sample knowledge base. For example, suppose the target question is "What is the number of outbound and inbound passengers at Changning Station during the morning rush hour?" At the same time, the sample knowledge base includes the following historical questions:

[0082] Historical Question 1: "During the morning rush hour, how many people leave Changning Station?"

[0083] Historical Question 2: "What is the number of passengers entering Changning Station during the morning rush hour?"

[0084] History Question 3: "What is the passenger volume at station 3 during the evening peak?"

[0085] Then, a sentence construction information set may be constructed based on historical question 1, historical query statement 1 used when querying the answer to historical question 1, feedback information on historical query statement 1, historical question 2, historical query statement 2 used when querying the answer to historical question 2, and feedback information on historical query statement 2. The constructed sentence construction information set may not include historical question 3, historical query statement 3 used when querying the answer to historical question 3, and feedback information on historical query statement 3.

[0086] Step S303, splicing the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determining the target query sentence based on the target question and the prompt words.

[0087] Specifically, the identifiers and data structures of the database tables corresponding to the metadata items in the target data set can be extracted to obtain the first information. At the same time, the information in the business information set and the statement construction information set can be respectively refined through the language model to obtain the second information and the third information. Then, the first information, the second information and the third information can be spliced ​​to obtain the prompt word.

[0088] After obtaining the prompt word, the prompt word and the target question can be input into the sentence generation model, and the sentence generation model uses the prompt word as a guide to generate a target query sentence for querying the answer to the target question.

[0089] Step S304, executing the target query statement in the target data set, and using the obtained result as the answer to the target question.

[0090] In this embodiment, if the display method of the answer to the target question is specified in the target question, the answer to the target question can be displayed in the display method specified in the target question. For example, assuming that the target question is "Please display the daily entry volume of Changning Station from May to August in a table", then the answer to the target question can be displayed in a table in the answer display interface. If the display method of the answer to the target question is not specified in the target question, the display method of the answer to the target question can be determined according to the data characteristics of the answer to the target question. For example, assuming that the target question is "What is the daily entry volume of Changning Station from May to August?" Since the answer to the target question obtained includes multiple records, it can be selected to display the answer to the target question in the form of a table, chart, etc. For another example, assuming that the target question is "What is the total entry volume of Changning Station from May to August?" Since the answer to the target question is only a numerical value, the answer to the target question can be directly displayed in the designated area of ​​the answer display interface, without the need to display the answer to the target question in the form of a table or chart.

[0091] In summary, in the technical solutions of some embodiments of the present disclosure, a target data set for querying the answer to the target question and an auxiliary information set for assisting in querying the answer to the target question are screened in a pre-built knowledge base, and then the information extracted from the target data set and the auxiliary information set are spliced ​​into prompt words. In this way, the prompt words can include business information related to the target question, or statement construction information related to the query statement of the answer to the target question. Therefore, with the prompt words as a guide, the generated target query statement can be more accurate and in line with the actual business scenario. Furthermore, the answer to the target question obtained based on the target query statement query is correspondingly more accurate and reliable.

[0092] The scheme for screening the target data set is further explained below.

[0093] In some embodiments, filtering out a target data set for querying an answer to a target question from a pre-built knowledge base may include:

[0094] The first semantic vector of the target question is compared with the second semantic vector of the description information of each metadata item for similarity. If the similarity between the first semantic vector and the second semantic vector exceeds a similarity threshold, the metadata item corresponding to the second semantic vector is used as the first metadata item;

[0095] Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the first metadata items, and a target data set is constructed based on the target metadata items.

[0096] Specifically, the relevance between the answer to the target question and the metadata item is used to represent the probability of querying data in the database table corresponding to the metadata item when querying the answer to the target question. For any metadata item, if the probability of querying data in the database table corresponding to the metadata item when querying the answer to the target question is greater, it means that the relevance between the target question and the metadata item is higher; conversely, if the probability of querying data in the database table corresponding to the metadata item when querying the answer to the target question is smaller, it means that the relevance between the target question and the metadata item is lower.

[0097] Furthermore, if the answer to the target question is related to a metadata item, then there may be content with similar semantic information in the description information of the target question and the metadata item. For example, suppose the description information of metadata item A is the number of passengers arriving and departing at each station per hour. When the target question is "What is the number of passengers arriving at Changning Station during the morning rush hour?", since both the description information of metadata item A and the target question include text information such as "number of passengers arriving", it can be considered that when querying the answer to the target question, it may be necessary to query the data in the database table corresponding to metadata item A, that is, the answer to the target question is related to metadata item A.

[0098] In summary, when the first semantic vector of the target question is compared with the second semantic vector of the description information of each metadata item for similarity, if the similarity between the first semantic vector and the second semantic vector exceeds the threshold, it can be considered that the target question is related to the metadata item; if the similarity between the first semantic vector and the second semantic vector does not exceed the threshold, it can be considered that the target question is not related to the metadata item. In this way, each metadata item can be quickly preliminarily screened to obtain the first metadata item related to the answer to the target question. At the same time, metadata items that are not related to the answer to the target question can be eliminated to reduce the amount of subsequent data queries.

[0099] In the first metadata item, some metadata items may have a high relevance to the answer to the target question, while some metadata items may have a low relevance to the answer to the target question. In order to further reduce the amount of subsequent data queries, further screening can be performed in the first metadata item, that is, the target metadata items whose relevance to the answer to the target question meets the relevance requirement can be further screened from the first metadata item.

[0100] Specifically, in some embodiments, the first metadata items may be sorted using a rerank algorithm. After sorting, the first metadata items with a higher relevance to the answer to the target question may be ranked in the front, and the first metadata items with a lower relevance to the answer to the target question may be ranked in the back. Then, the first metadata items ranked in the back may be eliminated.

[0101] In some embodiments, based on the first metadata item after the elimination operation is performed, filtering out the target metadata item whose relevance to the answer to the target question meets the relevance requirement from the first metadata item may include:

[0102] Clustering the first metadata item to obtain a clustering result;

[0103] Determining the degree of aggregation between different first metadata items according to the clustering result;

[0104] Filtering out second metadata items whose aggregation degree is higher than an aggregation threshold from the first metadata items;

[0105] Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the second metadata items.

[0106] Specifically, in the clustering result, if one of the first metadata items B is far away from the other first metadata items, it can be considered that the relevance of the first metadata item B to the answer to the target question does not meet the relevance requirement. On the contrary, for multiple first metadata items with a degree of aggregation higher than the aggregation threshold, it can be considered that the relevance of these first metadata items to the answer to the target question is high.

[0107] Through the clustering operation, metadata items with low relevance to the answer to the target question can be further eliminated from the first metadata items, thereby obtaining second metadata items with high relevance to the answer to the target question.

[0108] In some embodiments, selecting the target metadata item whose relevance to the answer to the target question meets the relevance requirement from the second metadata item may include:

[0109] Determine a graph structure relationship of the second metadata items, where the graph structure relationship is used to represent the association between different second metadata items;

[0110] Based on the graph structure relationship, metadata items with relevance higher than a relevance threshold are screened out from the second metadata items as target metadata items.

[0111] Specifically, the graph structure relationship may include nodes and lines. Among them, the nodes may represent the second metadata items, and the lines between the nodes may represent the correlation between the second metadata items. For example, when the line between two nodes is a line segment with an arrow, it can be indicated that the two nodes are associated; when the line between the two nodes is a line segment without an arrow, it can be indicated that the two nodes are not associated. Through the analysis of the graph structure relationship, metadata items with low relevance to the answer to the target question can be further eliminated from the second metadata items.

[0112] In the above embodiment, by comparing the similarity of semantic vectors, sorting by the rerank algorithm, clustering and graph structure relationships, metadata items can be screened layer by layer, and metadata items with low relevance to the answer to the target question can be gradually eliminated, so that the target metadata items finally obtained are metadata items that are highly relevant to the answer to the target question. In other words, the target data set constructed based on the target metadata items can be more refined.

[0113] In some embodiments, before constructing the target dataset based on the target metadata item, the method of the present disclosure may further include:

[0114] If the similarity between the first target metadata item and the second target metadata item in the target metadata item is higher than the similarity threshold, one of the first target metadata item and the second target metadata item is deleted, or the first target metadata item and the second target metadata item are merged into a third target metadata item, and the first target metadata item and the second target metadata item are deleted.

[0115] In this way, redundant data in the target metadata items can be eliminated, further improving the refinement of the target data set.

[0116] The following is a further explanation of the scheme for constructing information sets using filtering statements.

[0117] In some embodiments, as in step S302, the knowledge base may store statement construction information corresponding to different business scenarios; when the auxiliary information set includes a statement construction information set, the auxiliary information set for assisting in querying the answer to the target question is selected from the pre-built knowledge base, which may include:

[0118] Determine the target business scenario to which the target problem belongs;

[0119] Filter out target sentence construction information corresponding to the target business scenario from the knowledge base;

[0120] Information is constructed based on the target sentence, and the constructed sentence constructs the information set.

[0121] and, filtering out target historical questions matching the target questions in the knowledge base;

[0122] A statement construction information set is constructed based on the target historical question, the target historical query sentence used when querying the answer to the target historical question, and the feedback information for the target historical query sentence.

[0123] The relevant principles can be found in the description of step S302 and will not be elaborated here.

[0124] The scheme for filtering the business information set is further described below.

[0125] In some embodiments, the knowledge base stores business information corresponding to different business scenarios;

[0126] When the auxiliary information set includes a business information set, an auxiliary information set for assisting in querying the target question is selected from a pre-built knowledge base, including:

[0127] Determine the target business scenario to which the target problem belongs;

[0128] Filter out target business information corresponding to the target business scenario from the knowledge base;

[0129] Based on the target business information, a business information set is constructed.

[0130] The relevant principles can be found in the description of step S302 and will not be elaborated here.

[0131] The following further explains how to splice the information extracted from the target dataset and the auxiliary information set.

[0132] In some embodiments, when splicing the first information extracted from the target data set, the second information extracted from the business information set, and the third information extracted from the statement construction information set, there may be no order restrictions among the first information, the second information, and the third information. For example, the first information may be spliced ​​before the second information, and the second information may be spliced ​​before the first information. However, there may be order restrictions for different information extracted from the same data set. For example, assume that the first information extracted from the target data set includes information A, information B, and information C. In the first information, information A, information B, and information C may have order restrictions.

[0133] Specifically, in some embodiments, stitching the information extracted from the target data set and the auxiliary information set may include:

[0134] According to the importance of each target sentence construction information in the sentence construction information set, the information extracted from each target sentence construction information is sequentially spliced.

[0135] Specifically, information extracted from target sentence construction information with high importance can be arranged in front, and information extracted from target sentence construction information with low importance can be arranged in the back. In this way, the sentence generation model can first read the information extracted from target sentence construction information with high importance.

[0136] In some embodiments, the importance of each target sentence construction information is determined based on the matching degree between the target sentence construction information and the answer to the target question, and / or the probability of the target sentence construction information being used. Specifically, if the matching degree between the target sentence construction information and the answer to the target question is high, or the probability of the target sentence construction information being used is high, the target sentence construction information can be considered to be more important.

[0137] Similarly, in some embodiments, stitching the information extracted from the target data set and the auxiliary information set may include:

[0138] According to the importance of each target historical question in the statement construction information set, the information extracted from each target historical question, the historical query statement corresponding to the target historical question and the feedback information are sequentially spliced.

[0139] In some embodiments, stitching the information extracted from the target dataset and the auxiliary information set may include:

[0140] According to the matching degree between each target business information and the target problem, the information extracted from each target business information is sequentially spliced.

[0141] In some embodiments, the target business information is described in natural language. Before extracting information for splicing prompt words from the target business information, the method of the present disclosure may further include:

[0142] According to the specified optimization strategy, the natural language expression used to describe the target business information is optimized.

[0143] Specifically, the natural language expression method for describing the target business information may be optimized through a language model, etc., so that the information extracted from the target business information for splicing prompt words may be more accurate.

[0144] In some embodiments, after executing the target query statement on the target data set, the method of the present disclosure may further include:

[0145] Obtain feedback information for the target query statement;

[0146] The target question, the target query statement and the feedback information for the target query statement are saved in the knowledge base.

[0147] In this way, the target question can be used as a historical question, and the target query statement can be used as a historical query statement to serve as a reference for subsequent questions received.

[0148] For easier understanding, please refer to Figure 4 , which is a schematic diagram of a question answering process provided by an embodiment of the present disclosure. Figure 4In the question answering process, the specific steps include:

[0149] Step S401: receiving a target question.

[0150] Specifically, the data processing system can receive the target questions to be answered input by the user through the question receiving interface, such as "What is the number of passengers entering Changning Station and Zhongyuan Station from 15:00 to 17:00 on September 10, 2024?".

[0151] Step S402, data screening.

[0152] That is, to select the target dataset from the metadata knowledge base for querying the answer to the target question. For details, see Figure 5 Related description.

[0153] Step S403: perform knowledge retrieval.

[0154] That is, the business information set related to the target question is selected from the industry knowledge base, and the statement construction information set related to the query statement of the answer to the target question is selected from the statement construction knowledge base and the example knowledge base. Figures 6 to 8 Related description.

[0155] Step S404: Generate a query statement.

[0156] That is, based on the knowledge retrieved from the industry knowledge base, a query statement for the target question is generated.

[0157] Step S405: execute the query statement.

[0158] That is, the query statement obtained in step S404 is executed in the target data set.

[0159] Step S406, check whether the query statement is executed successfully. If the query statement is executed successfully, execute step S407; if the query statement fails to execute, re-execute step S404.

[0160] Specifically, the situation where the query fails to execute includes but is not limited to the problem of an error being reported during the query statement execution. Generally, the query statement execution fails because the structure of the query statement is incorrect. Therefore, the query statement can be regenerated by returning to step S404 to adjust the structure of the query statement, so as to obtain a query statement that can be successfully executed.

[0161] Step S407, generating answer description information.

[0162] That is, the execution result of the query statement is converted into a natural description language. For example, according to the target question in step S401, if the execution results are 50 and 57, then the answer description information corresponding to 50 can be: the number of passengers entering Changning Station from 15:00 to 17:00 on September 10, 2024, and the answer description information corresponding to 57 can be: the number of passengers entering Zhongyuan Station from 15:00 to 17:00 on September 10, 2024.

[0163] Step S408: Generate answer visualization information.

[0164] That is, the answer to be displayed is generated according to the specified display format. For example, based on the query basis execution result in step S406 and the answer description information in step S407, the answer visualization information can be shown in Table 2:

[0165] Table 2 Answer Visualization Information

[0166] The number of passengers arriving at Changning Station from 15:00 to 17:00 on September 10, 2024 50 The number of passengers arriving at Zhongyuan Station from 15:00 to 17:00 on September 10, 2024 57

[0167] Step S409: display the answer to the question.

[0168] This completes the overall description of the question answering process.

[0169] Combined with reference Figure 5 In some embodiments, the data screening in step S402 may specifically include the following steps:

[0170] Step S501, searching the metadata knowledge base.

[0171] Specifically, the first semantic vector of the target question is compared with the second semantic vector of the description information of each metadata item for similarity. If the similarity between the first semantic vector and the second semantic vector exceeds a similarity threshold, the metadata item corresponding to the second semantic vector is used as the first metadata item.

[0172] Step S502, re-rank using a rerank algorithm.

[0173] Specifically, the rerank algorithm is used to sort the first metadata items. After sorting, the first metadata items with higher relevance to the answer to the target question can be ranked in front, and the first metadata items with lower relevance to the answer to the target question can be ranked in the back. Based on the sorting result, the first metadata items ranked in the back are eliminated.

[0174] Step S503: Perform secondary filtering using a community algorithm.

[0175] Specifically, based on the first metadata items after the elimination operation is performed, the first metadata items are clustered to obtain clustering results, and according to the clustering results, second metadata items with a degree of aggregation higher than an aggregation threshold are screened out.

[0176] Step S504, performing three filterings based on the target graph structure relationship.

[0177] Specifically, the graph structure relationship of the second metadata items can be determined, the graph structure relationship is used to characterize the correlation between different second metadata items, and based on the graph structure relationship, metadata items with correlation higher than the correlation threshold are screened out from the second metadata items as target metadata items.

[0178] Step S505: Use the language model to perform another screening.

[0179] Specifically, based on the powerful computing power and deep learning algorithms of the language model, further refined data screening is performed on the target metadata items to eliminate redundant or irrelevant data items, ensuring that the target data items that are ultimately retained are data items that are highly relevant to the user's questions.

[0180] Combined with reference Figure 6 In some embodiments, the filtering of the business information set related to the target problem from the industry knowledge base in step S403 may specifically include the following steps:

[0181] Step S601, searching the industry knowledge base.

[0182] That is, based on the target business scenario to which the target problem belongs, the target business information corresponding to the target business scenario is screened out from the industry knowledge base to obtain a business information set.

[0183] Step S602, re-rank using a rerank algorithm.

[0184] That is, the target business information is sorted using the rerank algorithm according to the degree of match between each target business information and the target problem.

[0185] Step S603: repair the natural language describing the industry knowledge.

[0186] That is, based on the sorted target business information, the natural language expression method describing the target business information is optimized through language models and the like.

[0187] Step S604: insert the prompt word with reference information.

[0188] That is, based on the sorting result in step S602, information is extracted from each target business information in turn according to the matching degree between each target business information and the target question, and the extracted information is incorporated into the prompt word.

[0189] Combined with reference Figure 7In some embodiments, the step S403 of selecting a statement construction information set related to the query statement of the answer to the target question from the statement construction knowledge base may specifically include the following steps:

[0190] Step S701, constructing a knowledge base based on search statements.

[0191] That is, according to the target business scenario to which the target problem belongs, the target sentence construction information corresponding to the target business scenario is screened out from the sentence construction knowledge base to obtain a sentence construction information set.

[0192] Step S702, re-rank using a rerank algorithm.

[0193] That is, according to the importance of each target sentence construction information in the sentence construction information set, the target sentence construction information is sorted using the rerank algorithm.

[0194] Step S703, inserting the prompt word with reference information.

[0195] That is, based on the sorting result in step S702, information is extracted from each target sentence construction information in turn according to the importance of each target sentence construction information, and the extracted information is pieced into the prompt word.

[0196] Combined with reference Figure 8 In some embodiments, the step S403 of filtering the query statements related to the target question answer from the example knowledge base to construct the information set may specifically include the following steps:

[0197] Step S801, searching the example knowledge base.

[0198] That is, according to the target business scenario to which the target question belongs, target historical questions matching the target question are screened out from the example knowledge base, and a statement construction information set is constructed based on the target historical question, the target historical query statement used when querying the answer to the target historical question, and the feedback information for the target historical query statement.

[0199] Step S802, re-rank using a rerank algorithm.

[0200] That is, according to the importance of each target history question in the sentence construction information set, the target history is sorted using the rerank algorithm.

[0201] Step S803, insert the prompt word in a few shot manner.

[0202] That is, based on the sorting result in step S803, information extracted from at least part of the target historical questions, the historical query statements corresponding to the target historical questions, and the feedback information are spliced ​​into the prompt word.

[0203] Few-shot technology can effectively guide the data processing system to generate expected query statements by citing a small number of high-quality examples.

[0204] Corresponding to the data processing method, the present disclosure also provides a data processing device. Fig. 9 , which is a module schematic diagram of a data processing device provided in some embodiments of the present disclosure. Fig. 9 In the data processing device, the data processing device comprises:

[0205] The question acquisition module 901 is used to acquire the target question to be answered;

[0206] An information acquisition module 902 is used to filter out a target data set for querying the answer to the target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to the query statement of the answer to the target question;

[0207] The sentence generation module 903 is used to combine the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determine the target query sentence based on the target question and the prompt words;

[0208] The statement execution module 904 is used to execute the target query statement in the target data set and use the obtained result as the answer to the target question.

[0209] In some embodiments, the knowledge base stores one or more metadata items and description information of the metadata items, and the answer to the target question is obtained based on at least some of the metadata items; the information acquisition module 902 is specifically used to:

[0210] The first semantic vector of the target question is compared with the second semantic vector of the description information of each metadata item for similarity. If the similarity between the first semantic vector and the second semantic vector exceeds a similarity threshold, the metadata item corresponding to the second semantic vector is used as the first metadata item;

[0211] Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the first metadata items, and a target data set is constructed based on the target metadata items.

[0212] In some embodiments, the information acquisition module 902 is specifically used to:

[0213] Clustering the first metadata item to obtain a clustering result;

[0214] Determining the degree of aggregation between different first metadata items according to the clustering result;

[0215] Filtering out second metadata items whose aggregation degree is higher than an aggregation threshold from the first metadata items;

[0216] Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the second metadata items.

[0217] In some embodiments, the information acquisition module 902 is specifically used to:

[0218] Determine a graph structure relationship of the second metadata items, where the graph structure relationship is used to represent the association between different second metadata items;

[0219] Based on the graph structure relationship, metadata items with relevance higher than a relevance threshold are screened out from the second metadata items as target metadata items.

[0220] In some embodiments, before constructing the target dataset based on the target metadata item, the information acquisition module 902 is further configured to:

[0221] If the similarity between the first target metadata item and the second target metadata item in the target metadata item is higher than the similarity threshold, one of the first target metadata item and the second target metadata item is deleted, or the first target metadata item and the second target metadata item are merged into a third target metadata item, and the first target metadata item and the second target metadata item are deleted.

[0222] In some embodiments, the knowledge base stores statement construction information corresponding to different business scenarios; when the auxiliary information set includes a statement construction information set, the information acquisition module 902 is specifically used to:

[0223] Determine the target business scenario to which the target problem belongs;

[0224] Filter out target sentence construction information corresponding to the target business scenario from the knowledge base;

[0225] Information is constructed based on the target sentence, and the constructed sentence constructs the information set.

[0226] In some embodiments, the statement generation module 903 is specifically used to:

[0227] According to the importance of each target sentence construction information in the sentence construction information set, the information extracted from each target sentence construction information is spliced ​​in sequence; wherein the importance of each target sentence construction information is determined based on the degree of matching between the target sentence construction information and the answer to the target question, and / or the probability of the target sentence construction information being used.

[0228] In some embodiments, the knowledge base stores historical questions, historical query statements used when searching for answers to historical questions, and feedback information for historical query statements; when the auxiliary information set includes a statement construction information set, the information acquisition module 902 is specifically used to:

[0229] Filter out target historical questions matching the target questions in the knowledge base;

[0230] A statement construction information set is constructed based on the target historical question, the target historical query sentence used when querying the answer to the target historical question, and the feedback information for the target historical query sentence.

[0231] In some embodiments, the statement generation module 903 is specifically used to:

[0232] According to the importance of each target historical question in the statement construction information set, the information extracted from each target historical question, the historical query statement corresponding to the target historical question and the feedback information are sequentially spliced.

[0233] In some embodiments, the knowledge base stores business information corresponding to different business scenarios; when the auxiliary information set includes a business information set, the information acquisition module 902 is specifically used to:

[0234] Determine the target business scenario to which the target problem belongs;

[0235] Filter out target business information corresponding to the target business scenario from the knowledge base;

[0236] Based on the target business information, a business information set is constructed.

[0237] In some embodiments, the statement generation module 903 is specifically used to:

[0238] According to the matching degree between each target business information and the target problem, the information extracted from each target business information is sequentially spliced.

[0239] The data processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0240] The device disclosed in the present invention has the same beneficial effects as the above method, which will not be described in detail here.

[0241] The present disclosure also provides an electronic device having the above Fig. 9 The data processing device shown.

[0242] Combined with reference Fig.10 , is a schematic diagram of the structure of an electronic device provided by some embodiments of the present disclosure. Fig.10 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0243] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0244] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0245] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0246] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0247] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0248] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0249] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0250] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get the target question to be answered; Filtering out a target data set for querying an answer to a target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to a query statement of the answer to the target question; splicing the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determining a target query sentence based on the target question and the prompt words; The target query statement is executed in the target data set, and the obtained result is used as the answer to the target question.

2. The method according to claim 1, characterized in that The knowledge base stores one or more metadata items and description information of the metadata items, and the answer to the target question is obtained based on at least part of the metadata items; The step of selecting a target dataset for querying answers to target questions from a pre-built knowledge base includes: Performing a similarity comparison between the first semantic vector of the target question and the second semantic vector of the description information of each metadata item, if the similarity between the first semantic vector and the second semantic vector exceeds a similarity threshold, taking the metadata item corresponding to the second semantic vector as the first metadata item; Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the first metadata items, and the target data set is constructed based on the target metadata items.

3. The method according to claim 2, characterized in that The step of selecting, from the first metadata items, target metadata items whose relevance to the answer to the target question meets the relevance requirement, includes: Clustering the first metadata items to obtain a clustering result; Determining, according to the clustering result, the degree of aggregation between different first metadata items; Filtering out second metadata items whose aggregation degree is higher than an aggregation threshold from the first metadata items; Target metadata items whose relevance to the answer to the target question meets the relevance requirement are screened out from the second metadata items.

4. The method according to claim 3, characterized in that: The step of selecting, from the second metadata items, a target metadata item whose relevance to the answer to the target question meets a relevance requirement, includes: Determine a graph structure relationship of the second metadata items, where the graph structure relationship is used to represent the association between different second metadata items; Based on the graph structure relationship, metadata items with relevance higher than a relevance threshold are screened out from the second metadata items as the target metadata items.

5. The method according to claim 2, characterized in that: Before constructing the target dataset based on the target metadata item, the method further includes: If the similarity between the first target metadata item and the second target metadata item in the target metadata items is higher than a similarity threshold, one of the first target metadata item and the second target metadata item is deleted, or the first target metadata item and the second target metadata item are merged into a third target metadata item, and the first target metadata item and the second target metadata item are deleted.

6. The method according to claim 1, characterized in that The knowledge base stores statement construction information corresponding to different business scenarios; In the case where the auxiliary information set includes the sentence construction information set, selecting the auxiliary information set for assisting in querying the answer to the target question from the pre-built knowledge base includes: Determine the target business scenario to which the target problem belongs; Filtering target sentence construction information corresponding to the target business scenario from the knowledge base; The sentence construction information set is constructed based on the target sentence construction information.

7. The method according to claim 6, characterized in that The splicing of information extracted from the target data set and the auxiliary information set includes: According to the importance of each of the target sentence construction information in the sentence construction information set, sequentially splicing the information extracted from each of the target sentence construction information; The importance of each of the target sentence construction information is determined based on the matching degree between the target sentence construction information and the answer to the target question, and / or the probability of the target sentence construction information being used.

8. The method according to claim 1, characterized in that: The knowledge base stores historical questions, historical query statements used when searching for answers to the historical questions, and feedback information for the historical query statements; In the case where the auxiliary information set includes the sentence construction information set, selecting the auxiliary information set for assisting in querying the answer to the target question from the pre-built knowledge base includes: Screening out a target historical question matching the target question in the knowledge base; The statement construction information set is constructed based on the target historical question, the target historical query sentence used when querying the answer to the target historical question, and the feedback information for the target historical query sentence.

9. The method according to claim 8, characterized in that The splicing of information extracted from the target data set and the auxiliary information set includes: According to the importance of each of the target historical questions in the statement construction information set, information extracted from each of the target historical questions, the historical query statements corresponding to the target historical questions, and the feedback information are sequentially spliced.

10. The method according to claim 1, characterized in that The knowledge base stores business information corresponding to different business scenarios; In the case where the auxiliary information set includes the business information set, the auxiliary information set for assisting in querying the target question is selected from a pre-built knowledge base, including: Determine the target business scenario to which the target problem belongs; Filtering target business information corresponding to the target business scenario from the knowledge base; Based on the target business information, the business information set is constructed.

11. The method according to claim 10, characterized in that The splicing of information extracted from the target data set and the auxiliary information set includes: According to the matching degree between each target business information and the target question, the information extracted from each target business information is sequentially spliced.

12. A data processing device, characterized in that: The device comprises: A question acquisition module is used to obtain the target question to be answered; An information acquisition module, used to filter out a target data set for querying an answer to a target question and an auxiliary information set for assisting in querying the answer to the target question from a pre-built knowledge base, wherein the auxiliary information set includes a business information set related to the target question and / or a statement construction information set related to a query statement for the answer to the target question; A sentence generation module, used for splicing the information extracted from the target data set and the auxiliary information set to obtain prompt words, and determining a target query sentence based on the target question and the prompt words; The statement execution module is used to execute the target query statement in the target data set and use the obtained result as the answer to the target question.

13. An electronic device, characterized in that: include: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method of any one of claims 1 to 11 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the data processing method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The method comprises computer instructions for causing a computer to perform the data processing method according to any one of claims 1 to 11.

Images