A data processing method, apparatus, device, and storage medium

Abstract

The present application discloses a data processing method, apparatus, device, and storage medium. The method includes: when a target user needs to obtain target data, a statement indicating the acquisition of the target data can be input, and the statement can be in natural language. Then, natural language processing is performed on the statement input by the user, including obtaining the intent type of the statement, annotating the slots of the statement based on the intent type of the statement, and then mapping the slots to obtain first processed data. Data query, analysis, etc. can be performed on the data in the database based on the first processed data, so as to obtain the target data required by the user. Through the data processing method provided by the present application, the natural language of the user can be obtained for processing, and thus the target data required by the user can be obtained according to the processed data, improving the efficiency of the user in obtaining data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data processing method, apparatus, device, and storage medium. Background Art

[0002] As an important tool for a company's data-driven operations, a data dashboard can be used to call business data tables, display data, monitor, and analyze key business metrics. Users can obtain the required data information through the data content of the data dashboard to make business decisions.

[0003] With the development of business, business data tables also exhibit the characteristics of high dimensions and multiple metrics. When using a data dashboard to search for and analyze data, due to the large number of data dimensions and metrics, it may lead to problems such as long data search time and complex data analysis process, affecting the efficiency of users in obtaining data. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data processing method, apparatus, device, and storage medium to improve the efficiency of users in obtaining data.

[0005] To achieve the above object, the technical solutions provided in this application are as follows:

[0006] In the first aspect of this application, a data processing method is provided. The method includes:

[0007] Obtain a statement input by a target user, where the statement is used to indicate obtaining target data required by the target user;

[0008] Obtain the intent type of the statement;

[0009] Based on the intent type of the statement, label the slots in the statement;

[0010] Map the slots in the statement to obtain first processed data;

[0011] Perform data processing based on the first processed data to obtain the target data.

[0012] In the second aspect of this application, a data processing apparatus is provided. The apparatus includes:

[0013] A first acquisition unit, configured to obtain a statement input by a target user, where the statement is used to indicate obtaining target data required by the target user;

[0014] A second acquisition unit, configured to obtain the intent type of the statement;

[0015] A labeling unit, configured to label the slots in the statement based on the intent type of the statement;

[0016] A mapping unit, configured to map the slots in the statement to obtain first processed data;

[0017] A processing unit, configured to perform data processing based on the first processed data to obtain the target data.

[0018] In a third aspect of the present application, an electronic device is provided, and the device includes: a processor and a memory;

[0019] The memory is configured to store instructions or computer programs;

[0020] The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the method described in the first aspect above.

[0021] In a fourth aspect of the present application, a computer-readable storage medium is provided, and instructions are stored in the computer-readable storage medium. When the instructions run on a device, the device is caused to execute the method described in the first aspect above.

[0022] In a fifth aspect of the present application, a computer program product is provided, and the computer program product includes a computer program / instructions. When the computer program / instructions are executed by a processor, the method described in the first aspect above is implemented.

[0023] It can be seen that the present application has the following beneficial effects:

[0024] In the above implementation manner of the present application, when a target user needs to obtain target data, a statement indicating the acquisition of the target data can be input, and the statement can be a natural language. Then, natural language processing is performed on the statement input by the user, including obtaining the intent type of the statement and annotating the slots of the statement based on the intent type of the statement, and then mapping the slots to obtain first processed data. Data query, analysis, etc. can be performed on the data in the database based on the first processed data, so as to obtain the target data required by the user. Through the data processing method provided by the present application, the natural language of the user can be obtained for processing, so as to obtain the target data required by the user according to the processed data, and the efficiency of the user obtaining data is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1Flow chart of a data processing method provided by an embodiment of the present application;

[0027] Figure 2 Flow chart of another data processing method provided by an embodiment of the present application;

[0028] Figure 3 Schematic diagram of a data processing device provided by an embodiment of the present application;

[0029] Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0031] With the development of business, business data tables show the characteristics of high dimension and multiple indicators. Currently, when using a data dashboard to search for and analyze data, due to the large number of data dimensions and indicators, it will lead to problems such as long data search time and complex data analysis process, affecting the efficiency of users to obtain data.

[0032] Based on this, the embodiments of the present application provide a data processing method to improve the efficiency of users to obtain data. Specifically, when a target user needs to obtain target data, a statement indicating the acquisition of the target data can be input, and this statement can be a natural language. Then, natural language processing is performed on the statement input by the user, including obtaining the intent type of the statement and annotating the slots of the statement based on the intent type of the statement, and then mapping the slots to obtain the first processed data. Data query, analysis, etc. can be performed on the data in the database based on the first processed data, so as to obtain the target data required by the user. Through the data processing method provided by the present application, natural language of the user can be obtained and processed based on the form of a dialogue, so that the target data required by the user can be obtained from the database according to the processed data, improving the efficiency of the user to obtain data.

[0033] To facilitate understanding of the technical method provided by the embodiments of the present application, the following will be specifically introduced with reference to the accompanying drawings.

[0034] See Figure 1 , Figure 1 Flow chart of a data processing method provided by an embodiment of the present application.

[0035] Among them, this method can be executed by a data processing device, which can be an electronic device or other devices. The data processing device can be applied to the PC side or the mobile side. This method may include the following steps:

[0036] S101: Obtain the statement input by the target user, which is used to indicate obtaining the target data required by the target user.

[0037] In order to obtain the required target data, the target user can input a statement on the data processing device, which is used to indicate obtaining the required target data. Among them, this statement can be a natural language, that is, the language commonly used by people in daily life. That is to say, the data processing device can obtain the natural language input by the target user and process the natural language to provide the target data required by the target user. Optionally, the target user can input a statement on the text input interface of the data processing device so that the data processing device obtains the statement. Or, the target user can also input a statement in the form of voice through the voice input device of the data processing device so that the data processing device obtains the statement. This statement can be used to indicate the target data that the target user needs to obtain. For example, when the statement input by the target user is "the turnover of the target store", the actual value of the turnover of the target store is the target data required by the target user. The data processing device can process this statement to return the actual value required by the target user.

[0038] S102: Obtain the intent type of the statement.

[0039] After obtaining the statement input by the user, natural language processing can be performed on this statement, including determining the intent type, annotating slots, and slot mapping, etc., so as to obtain the first processed data after processing. Specifically, the intent of the statement can be recognized based on a text classification model to obtain the intent type corresponding to this statement.

[0040] S103: Based on the intent type of the statement, annotate the slots in the statement.

[0041] When the intent types expressed by the statements are different, the slots annotated in the statements will also be different. Optionally, after determining the intent type of this statement, use a sequence annotation model to annotate the slots in the statement. In the statement used by the target user to express the intent, the slot represents an identifier for the key information used to accurately express this intent. The slot is a variable that can help the data processing device understand the intent of the target user from a semantic perspective. For example, the types of slots can include tenant, data table, metric, dimension, dimension member, metric filter condition, dimension filter condition, date, sorting, limit (limiting condition), etc.

[0042] In the above embodiments, an intent recognition can be performed using a text classification model, and a slot labeling can be performed using a sequence labeling model. Among them, both the text classification model and the sequence labeling model are models trained according to training samples, and the training samples for training the text classification model and the sequence labeling model can be obtained simultaneously. To facilitate the understanding of the working principles of the text classification model and the sequence labeling model, the training processes of the text classification model and the sequence labeling model will be introduced below.

[0043] To train the text classification model and the sequence labeling model, it is first necessary to obtain training samples. Optionally, the training samples can be obtained in the following way. The data processing device can obtain the historical statements input by each user, and then determine the slots of a preset type, the set of statement templates, and the set of intent types corresponding to the set of statement templates according to the historical statements. Among them, the slots of the preset type can refer to the above embodiments, including tenant, data table, indicator, dimension, dimension member, indicator filtering condition, dimension filtering condition, date, sorting, limit (limiting condition), etc. The set of statement templates can be determined according to different forms of questions included in the historical statements. Among them, the intent type corresponding to the set of statement templates can indicate the way for the user to obtain the target data, such as data query, data anomaly detection, etc. As shown in Table 1, it is a schematic diagram of a set of statement templates and intent types. Among them, [] represents the slots in the statement. According to Table 1, it can be seen that the standard statement includes required information, which is the information that needs to be provided in the statement of the target user, so that this statement can be used as a complete statement to obtain the data required by the target user.

[0044] Table 1 Set of statement templates and intent types

[0045]

[0046]

[0047]

[0048] After determining the set of statement templates, the set of statement templates can be filled with historical statements to obtain the corresponding set of statements. For any first statement in the set of statements, based on the various types of slots determined above, the first statement is annotated using BIO to obtain annotation information. That is, the slot types included in the first statement are annotated. When using BIO for annotation, each word can be annotated as "B-X", "I-X", or "O". Among them, "B-X" means that the segment where the word is located belongs to type X and the word is at the beginning of this segment, "I-X" means that the segment where the word is located belongs to type X and the word is in the middle position of this segment, and "O" means that it does not belong to any type. For example, when the first statement is "Yesterday's e-commerce daily report", the annotation information obtained using BIO can be expressed as "B-Date, I-Date, O, B-Dim-value, I-Dim-value, B-Dim, I-Dim". "Yesterday" represents the date, "e-commerce" represents the dimension, and "daily report" represents the keyword. Since the set of intent types corresponding to the set of statement templates has been determined based on the historical statements of each user, the intent type corresponding to each statement template in the set of statement templates can also be determined, that is, the intent type corresponding to each statement in the filled set of statements can be determined. Then, using all the first statements in the set of statements, the annotation information corresponding to each first statement, and the intent type corresponding to each first statement, training samples are determined.

[0049] Optionally, when training a text classification model based on the training samples, multiple first statements in the training samples can be input into the initial classification model to obtain the first encoded vector. The initial classification model is used to extract features from the first encoded vector to obtain the intent type output value corresponding to the training samples. Then, based on the intent types corresponding to the multiple first statements respectively and the intent type output value, the first loss function is determined. Among them, the first loss function can represent the degree of difference between the actual intent type corresponding to the training samples and the intent type output value output by the initial classification model. The larger the first loss function, the greater the difference between the two, that is, the less accurate the initial classification model. When the first loss function is greater than or equal to the first threshold, it indicates that the matching degree between the intent type output value output by the initial classification model and the actual intent types corresponding to the multiple first statements does not meet the requirements. The parameters of the initial classification model can be adjusted based on the first loss function, and the process of inputting the multiple first statements into the initial classification model with adjusted parameters and subsequent training is re-executed until the determined first loss function is less than the first threshold to obtain the trained text classification model.

[0050] Optionally, the text classification model can be composed of an ALBERT model and a convolutional structure. Then, the structure and implementation process of the text classification model can include the following steps:

[0051] A1. Embedding layer. First, multiple first statements can be concatenated into a matrix, and the ALBERT model can be used to encode the concatenated matrix into a first encoded vector.

[0052] A2. Convolutional layer to extract features. The size of the first encoded vector is denoted as |d|, and the size of the convolutional kernel can be set to n * |d|, where n is the length of the convolutional kernel and |d| is the width of the convolutional kernel. The width of the convolutional kernel is the same as the dimension of the vector.

[0053] A3. Max pooling layer. Take the maximum value of several one-dimensional vectors obtained after convolution and then concatenate them together as the output value of this layer.

[0054] A4. Fully connected layer. Concatenate another layer after the max pooling layer to obtain the intent type output value.

[0055] When training the sequence labeling model using training samples, it can be achieved in the following way: Input multiple first statements into the initial labeling model to obtain a second encoded vector. Use the initial labeling model to decode the second encoded vector to obtain the labeled information output value. Then, based on the labeled information corresponding to multiple first statements and the labeled information output value, determine the second loss function. The second loss function can represent the degree of difference between the actual labeled information corresponding to the training sample and the labeled information output value output by the initial labeling model. When the second loss function is greater than or equal to the second threshold, it indicates that the accuracy of the initial labeling model does not meet the requirements. The parameters of the initial labeling model can be adjusted based on the second loss function, and the process of inputting multiple first statements into the initial labeling model with adjusted parameters and subsequent training can be re-executed until the second loss function is less than the second threshold to obtain the trained sequence labeling model. Among them, the sequence labeling model can be composed of the ALBERT model and the conditional random field CRF. After the ALBERT model encodes the input first statement, a second encoded vector can be obtained. Then, use the CRF to decode the second encoded vector to output the optimal labeled sequence, that is, the labeled information corresponding to the training sample.

[0056] It should be noted that the text classification model and the sequence labeling model provided in the above embodiments are only an exemplary description and are not limited to the above implementation manners. Other implementable manners are also within the protection scope of this application.

[0057] S104: Map the slots in the statement to obtain the first processed data.

[0058] After the slots in the statement are annotated, the slots can be mapped to the real values in the database through entity linking, and the corresponding slots are replaced with the real values, so as to obtain the first processed data according to the original statement. For example, when the statement fragment representing the date slot is "the last week", the specific time corresponding to "the last week" can be determined based on the time of the user input statement, that is, mapped to the real value. The process of mapping the slot to the real value through entity linking will be introduced below in combination with the specific types of slots.

[0059] For the slots representing dates, taking the current time corresponding to the target user input statement as the benchmark, the extracted date slots are mapped into standard date real values. For example, when the time of the user input statement is October 1, 2022, the date slot "last Monday" can be mapped to "September 19, 2022", and the date slot "this week" can be mapped to "September 26, 2022 to October 2, 2022".

[0060] For the slots representing dimension members, the entity linking of dimension members can be divided into two processes, including a pre-execution process and an online process. The pre-execution process is executed regularly, which can be executed daily. The online process is triggered by the statement input by the target user. In the pre-execution process, for the dimensions corresponding to each data in the database, the content of the dimension members corresponding to different dimensions is counted, and an index is built by importing Elasticsearch (ES). And during the running process, the dimensions and dimension members used by the user in the product can also be imported into ES. In the online process, after the slots of the target user input statement are annotated, the statement fragments with the slot type of dimension member can be filtered out. Perform an ES search on the statement fragments representing dimension members, and recall several candidates. Then the text similarity algorithm can be used to calculate the similarity score between the statement fragment and each candidate, and sort according to each similarity score, and select the candidate with the highest score as the real value corresponding to the dimension member. Among them, the text similarity algorithm can be implemented through the edit distance. The edit distance refers to the minimum number of edit operations required to edit one character into another between two characters. If the edit distance is larger, it means that the similarity between the two characters is lower. For example, when the statement fragment representing the dimension member in the statement is "target store", the mapped real value can be "target official flagship store".

[0061] For the slot representing an indicator, a sequence labeling model can be used to label and filter out the statement fragments with the slot type of indicator from the statement input by the target user. Obtain the indicator data set that the target user has viewing permission from the database, and then use the text similarity algorithm to calculate the similarity score between the statement fragment and the indicators in the indicator data set. Sort according to each similarity score, and select the indicator corresponding to the highest score as the true value corresponding to the indicator slot. It should be noted that for the entity linking process of the slot type of dimension, refer to the entity linking process of the indicator, which will not be elaborated here. Among them, in different application scenarios, different users can have the permission to view different data, so the indicator data set that the target user has viewing permission can be obtained, and the true value corresponding to the indicator slot of the statement fragment can be determined in this indicator data set.

[0062] For the slot representing the dimension filtering condition, the true value mapped by this slot can be expressed as [dimension name] + [operator] + [dimension member]. After using the sequence labeling model to label the slots in the statement input by the target user, the statement fragments with the slot type of dimension filtering condition can be filtered out. Then, based on the preset operator combination, determine the position of the operator in this statement fragment. Among them, the preset operator set can include "is", "equals", etc. According to the position of the operator, the part representing the dimension and the part identifying the dimension member can be determined, and then the entity linking process of the dimension and the entity linking process of the dimension member can be respectively executed according to the above embodiments.

[0063] S105: Perform data processing based on the first processed data to obtain the target data.

[0064] After obtaining the first processed data through natural language processing, the data processing device can process the data in the database according to the first processed data, so as to obtain the target data required by the target user. Optionally, the first processed data may be text data. To facilitate the data processing by the data processing device and the database, the first processed data can be processed based on the target data structure to obtain the second processed data. That is, the second processed data and the first processed data are data with the same semantics but different data definition forms, and the data processing device can process the data in the database based on the second processed data. For example, the target data structure can be a parameter structure for the database defined using JSON. As shown in Table 2, it is a schematic diagram of a target data structure.

[0065] Table 2 Target Data Structure

[0066] Parameter Name Meaning userInfo User Information filters Dynamic Filter Conditions measures Measures schemaId Data Table sorts Sorting analysisMeasures Analysis Measures dimensions Dimensions

[0067] Specifically, when the target data required by the target user can be directly obtained from the database, the data processing device can perform a data query based on the second processed data to obtain the target data. When the target data required by the target user needs to be further calculated and processed, data query can be performed first to obtain the basic data, and then the basic data can be analyzed and calculated to obtain the target data. When performing a data query, since the data in the database may have different data sources, the data processing device can define multiple data query interfaces and use multiple execution engines for data query, including: SQL engine, HBO engine, NOSQL engine, etc. Among them, the metrics that need to be analyzed and calculated can include: year-on-year and month-on-month, proportion, ranking, etc. For example, when the target data required by the target user is the growth rate of the store's turnover this week compared to last week, the data that can be queried from the database includes the store's turnover this week and last week, and then it is necessary to further calculate based on the turnover this week and last week to obtain the growth rate. When calculating and analyzing metrics, multiple metrics in the intermediate process may be involved. The data processing device can use a directed acyclic graph to determine the dependency relationship and calculation order between each metric, so that each metric can be obtained in sequence after obtaining the basic data. A specific application scenario will be introduced below. When the target user's statement is "query the week-on-week change in the conversion rate of each store", the analysis metric is "the week-on-week change in the conversion rate", and the week-on-week change in the conversion rate = (conversion rate this week - conversion rate last week) / conversion rate last week, and the conversion rate = number of transactions / number of clicks. Therefore, the basic data that the data processing device needs to obtain is the number of transactions and clicks of each store this week / last week, and then calculate the conversion rate and the week-on-week change in the conversion rate in sequence.

[0068] After obtaining the target data, the obtained target data can be sent to the target user so that the target user can obtain the required data in a timely manner. Optionally, when the target data that the user needs to obtain is relatively large, in order to facilitate the user to view the target data more clearly, the target data can be filled into the corresponding data template, and then the filled target data template can be sent to the user. Among them, the target data template is a template in the data template set, and the data template set can be determined in advance according to the historical statements input by each user. To facilitate the understanding of the technical solution of this application, compared with the first keyword, the second keyword will be introduced first below. Specifically, when implementing, multiple second keywords and the data fields corresponding to the second keywords can be determined based on the historical statements input by the user. Among them, the keyword is an invariant and is associated with the user's intention to obtain data. For example, in "the daily report of the store in October" and "the daily report of the store in November", "daily report" is the keyword. The data fields corresponding to the keyword can include static data and dynamic data. Among them, the static data is fixed and stores a fixed piece of text, link, picture, etc. The data processing device does not need to dynamically query when replying to the user's data intention. The dynamic data can be understood as a variable, such as the true value corresponding to the slot. When the chart stores a query condition, the query condition can be composed of dimensions, metrics, filtering conditions, sorting, etc. The data processing device needs to dynamically generate data when replying to the user's data intention. As shown in Table 3, it is a schematic diagram of a data field. For each second keyword, the second keyword and the data fields corresponding to the second keyword can be edited by using an editor to determine a data template. That is, each keyword can correspond to a data template, so that a data template set can be determined.

[0069] Table 3 Data Fields

[0070]

[0071]

[0072]

[0073] Based on this, the target data template corresponding to the statement of the target user can be obtained in the following manner. According to the above embodiments, after using the sequence annotation model to annotate the slots in the statement, the first keyword in the annotated statement can be determined. For example, after removing the slots of the statement and the statement fragments without types, the first keyword of the statement can be determined. Then, based on the first keyword, the corresponding target data template is obtained from the data template set. Specifically, the similarity between the first keyword and each second keyword corresponding to the data template set can be determined, and the second keyword corresponding to the maximum value among all similarities is determined as the target keyword, and the data template corresponding to the target keyword is the target data template. Subsequently, when the data processing device obtains the target data, it can match the type of the target data with the data fields in the target data template. When the match is successful, the target data can be filled into the matching data fields, so that the filled target data template can be obtained.

[0074] In the above embodiments of the present application, the statement input by the target user includes the necessary information in the standard question sentence, that is, a semantically complete statement. When the statement input by the target user is an incomplete statement, the data intention to be expressed by the target user can be predicted and inferred according to the statement input by the target user. Specifically, after performing natural language processing on the statement, the third processed data is first obtained. Then, the transfer slots corresponding to the slots in the third processed data can be predicted based on the data combination framework. The data combination framework includes preset types of slots, defines the information that the target user needs to provide, and also defines the types of slots that the current slot can be transferred to. As shown in Table 4, it is a schematic diagram of slot transfer. When the target user confirms the transfer slot, the first processed data can be determined based on the transfer slot and the third processed data. That is, a semantically complete processed data. In addition, the data processing device can also obtain the previous conversation of the target user, that is, the previous statement input, and ask questions about the types of slots that the target user may be interested in in combination with the context information, so as to provide more data for the target user.

[0075] Table 4 Slot Transfer

[0076]

[0077]

[0078] Through the data processing method provided by the present application, the natural language of the user can be obtained for processing, so as to obtain the target data required by the user according to the processed data, and the efficiency of the user obtaining data is improved.

[0079] Based on the above method embodiments, the embodiments of the present application further provide a data processing method. See Figure 2 , Figure 2Flowchart of another data processing method provided by an embodiment of this application.

[0080] This method may include the following steps:

[0081] S201: Obtain a statement input by a target user, where the statement is used to indicate obtaining target data required by the target user;

[0082] S202: Based on a text classification model, perform intent recognition on the statement to obtain the intent type of the statement;

[0083] S203: Based on the intent type of the statement, use a sequence labeling model to label the slots in the statement;

[0084] S204: Through entity linking, map the slots to real values in the database, and use the real values to replace the slots to obtain first processed data;

[0085] S205: Based on the first processed data, perform data querying and / or analytical calculations to obtain target data.

[0086] For the beneficial effects of the data processing method provided by the embodiment of this application, refer to the above method embodiment, which will not be elaborated here.

[0087] Based on the above method embodiment, the embodiment of this application further provides a data processing device. Refer to Figure 3 , Figure 3 Schematic diagram of a data processing device provided by the embodiment of this application.

[0088] This device 300 may include:

[0089] A first obtaining unit 301, configured to obtain a statement input by a target user, where the statement is used to indicate obtaining target data required by the target user;

[0090] A second obtaining unit 302, configured to obtain the intent type of the statement;

[0091] A labeling unit 303, configured to label the slots in the statement based on the intent type of the statement;

[0092] A mapping unit 304, configured to map the slots in the statement to obtain first processed data;

[0093] A processing unit 305, configured to perform data processing based on the first processed data to obtain the target data.

[0094] In a possible implementation manner, the second obtaining unit 302 is specifically configured to perform intent recognition on the statement based on a text classification model to obtain the intent type of the statement, and the text classification model is obtained by training based on training samples.

[0095] In a possible implementation, the annotation unit 303 is specifically configured to label the slots in the statement based on the intent type of the statement by using a sequence annotation model, and the sequence annotation model is trained based on the training samples.

[0096] In a possible implementation, the mapping unit 304 is specifically configured to map the slots to real values in the database through entity linking, replace the slots with the real values, and obtain the first processed data.

[0097] In a possible implementation, the processing unit 305 is specifically configured to process the first processed data based on the target data structure to obtain second processed data; perform data processing based on the second processed data to obtain the target data.

[0098] In a possible implementation, the device 300 further includes: a determination unit; after labeling the slots in the statement by using the sequence annotation model, the determination unit is configured to determine a first keyword in the labeled statement; obtain a corresponding target data template from a data template set based on the first keyword, and the data template set is pre-determined based on historical statements input by users.

[0099] The device 300 further includes: a filling unit; after obtaining the target data, the filling unit is configured to fill the target data into the target data template and send the filled target data template to the target user.

[0100] In a possible implementation, the process of obtaining the training samples includes:

[0101] Based on historical statements input by users, determine slots of a preset type, a statement template set, and an intent type set corresponding to the statement template set;

[0102] Use the historical statements to fill the statement template set to obtain a statement set;

[0103] For any first statement in the statement set, label the first statement by using BIO based on the slots of the preset type to obtain annotation information;

[0104] Determine the training samples based on multiple first statements, annotation information respectively corresponding to the multiple first statements, and intent types respectively corresponding to the multiple first statements.

[0105] In a possible implementation, the training process of the text classification model includes:

[0106] Input multiple of the first statements into an initial classification model to obtain a first encoded vector;

[0107] Use the initial classification model to extract features from the first encoded vector to obtain an intent type output value;

[0108] Based on the intent types respectively corresponding to multiple of the first statements and the intent type output value, determine a first loss function, and based on the first loss function, adjust the parameters of the initial classification model, and re - execute the process of inputting multiple of the first statements into the initial classification model and subsequent training processes until the first loss function is less than a first threshold to obtain the text classification model.

[0109] In a possible implementation manner, the training process of the sequence annotation model includes:

[0110] Input multiple of the first statements into an initial annotation model to obtain a second encoded vector;

[0111] Use the initial annotation model to decode the second encoded vector to obtain an annotation information output value;

[0112] Based on the annotation information respectively corresponding to multiple of the first statements and the annotation information output value, determine a second loss function, and based on the second loss function, adjust the parameters of the initial annotation model, and re - execute the process of inputting multiple of the first statements into the initial annotation model and subsequent training processes until the second loss function is less than a second threshold to obtain the sequence annotation model.

[0113] In a possible implementation manner, when the statement is an incomplete statement, the mapping unit 304 is specifically configured to map the slots in the statement to obtain third - processed data; predict the transfer slots corresponding to the slots in the third - processed data based on a data combination framework, where the data combination framework includes preset types of slots; in response to an operation by the target user to confirm the transfer slots, determine the first - processed data based on the transfer slots and the third - processed data.

[0114] In a possible implementation manner, each data template in the data template set includes a second keyword and data fields of different types, and the determination process of the preset data template set includes:

[0115] Determine multiple second keywords and the data fields corresponding to the multiple second keywords based on historical statements input by a user; for any one of the multiple second keywords, use an editor to edit the second keyword and the data fields corresponding to the second keyword to determine a data template;

[0116] The determining unit is specifically configured to: determine the similarity between the first keyword and the multiple second keywords; determine the second keyword corresponding to the maximum value in the similarities as the target keyword; determine the data template corresponding to the target keyword as the target data template;

[0117] The filling unit is specifically configured to: match the field type of the target data with the data fields in the target data template; when the matching is successful, fill the target data into the data fields in the target data template that match.

[0118] In a possible implementation manner, the mapping unit 304 is specifically configured to, when the type of the slot is a dimension filtering condition, determine the operator position in the statement based on a preset operator set; determine the dimension and / or dimension member in the statement based on the position of the operator; map the dimension and / or the dimension member to the real value in the database.

[0119] In a possible implementation manner, the processing unit 305 is specifically configured to perform a data query based on the second processed data to obtain the target data.

[0120] In a possible implementation manner, the processing unit 305 is specifically configured to perform a data query based on the second processed data to obtain the basic data; perform analysis and calculation on the basic data to obtain the target data.

[0121] For the beneficial effects of the data processing device provided in the embodiments of the present application, reference can be made to the above method embodiments, which will not be elaborated here.

[0122] It should be noted that the specific implementation of each unit in this embodiment can refer to the relevant descriptions in the above method embodiments. The division of units in the embodiments of the present application is illustrative, and is only a logical function division. In actual implementation, there may be other division methods. Each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. For example, in the above embodiments, the processing unit and the sending unit can be the same unit or different units. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0123] See Figure 4, which shows a schematic structural diagram of an electronic device 400 suitable for implementing the embodiments of the present application. The terminal devices in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0124] As Figure 4 shown, the electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 401, which may perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 402 or the programs loaded from the storage device 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0125] Generally, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 shows the electronic device 400 having various devices, it should be understood that it is not required to implement or include all the shown devices. Instead, more or fewer devices may be implemented or included.

[0126] Particularly, according to the embodiments of the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the methods of the embodiments of the present application are executed.

[0127] The electronic device provided in the embodiments of the present application and the method provided in the above embodiments belong to the same inventive concept. For technical details not described in detail in the present embodiment, reference may be made to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0128] The embodiments of the present application provide a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the method provided in the above embodiments is implemented.

[0129] It should be noted that the computer-readable medium in the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0130] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0131] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device.

[0132] The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device can execute the above method.

[0133] Computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by connecting through the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this application may be implemented in software or in hardware. Among them, the name of the unit / module does not constitute a limitation on the unit itself in some cases.

[0136] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and so on.

[0137] In the context of this application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM or Flash Memory), an optical fiber, a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] According to one or more embodiments of this application, a data processing method is provided, which can include:

[0139] Obtain a statement input by a target user, where the statement is used to indicate obtaining target data required by the target user;

[0140] Obtain the intent type of the statement;

[0141] Based on the intent type of the statement, label the slots in the statement;

[0142] Map the slots in the statement to obtain first processed data;

[0143] Based on the first processed data, perform data processing to obtain the target data.

[0144] According to one or more embodiments of this application, the obtaining the intent type of the statement includes:

[0145] Based on a text classification model, perform intent recognition on the statement to obtain the intent type of the statement, where the text classification model is trained based on training samples.

[0146] According to one or more embodiments of this application, the labeling the slots in the statement based on the intent type of the statement includes:

[0147] Based on the intention type of the statement, use a sequence labeling model to label the slots in the statement, and the sequence labeling model is trained based on the training samples.

[0148] According to one or more embodiments of the present application, the mapping of the slots in the statement to obtain the first processed data includes:

[0149] Map the slots to the real values in the database through entity linking, and use the real values to replace the slots to obtain the first processed data.

[0150] According to one or more embodiments of the present application, the data processing based on the first processed data to obtain the target data includes:

[0151] Process the first processed data based on the target data structure to obtain the second processed data;

[0152] Perform data processing based on the second processed data to obtain the target data.

[0153] According to one or more embodiments of the present application, after using the sequence labeling model to label the slots in the statement, the method further includes:

[0154] Determine the first keyword in the labeled statement;

[0155] Obtain the corresponding target data template from the data template set based on the first keyword, and the data template set is pre-determined based on the historical statements input by the user;

[0156] After obtaining the target data, the method further includes:

[0157] Fill the target data into the target data template and send the filled target data template to the target user.

[0158] According to one or more embodiments of the present application, the process of obtaining the training samples includes:

[0159] Based on the historical statements input by the user, determine the slots of the preset type, the statement template set, and the intention type set corresponding to the statement template set;

[0160] Use the historical statements to fill the statement template set to obtain a statement set;

[0161] For any first statement in the statement set, based on the slots of the preset type, use BIO to label the first statement to obtain labeling information;

[0162] Determine the training sample based on the multiple first statements, the annotation information respectively corresponding to the multiple first statements, and the intent types respectively corresponding to the multiple first statements.

[0163] According to one or more embodiments of the present application, the training process of the text classification model includes:

[0164] Input the multiple first statements into the initial classification model to obtain a first encoded vector;

[0165] Use the initial classification model to extract features from the first encoded vector to obtain an intent type output value;

[0166] Based on the intent types respectively corresponding to the multiple first statements and the intent type output value, determine a first loss function, and adjust the parameters of the initial classification model based on the first loss function. Re-execute the process of inputting the multiple first statements into the initial classification model and subsequent training processes until the first loss function is less than a first threshold to obtain the text classification model.

[0167] According to one or more embodiments of the present application, the training process of the sequence annotation model includes:

[0168] Input the multiple first statements into the initial annotation model to obtain a second encoded vector;

[0169] Use the initial annotation model to decode the second encoded vector to obtain an annotation information output value;

[0170] Based on the annotation information respectively corresponding to the multiple first statements and the annotation information output value, determine a second loss function, and adjust the parameters of the initial annotation model based on the second loss function. Re-execute the process of inputting the multiple first statements into the initial annotation model and subsequent training processes until the second loss function is less than a second threshold to obtain the sequence annotation model.

[0171] According to one or more embodiments of the present application, when the statement is an incomplete statement, the mapping of the slots in the statement to obtain the first processed data includes:

[0172] Map the slots in the statement to obtain third processed data;

[0173] Predict the transfer slots corresponding to the slots in the third processed data based on a data combination framework, where the data combination framework includes slots of a preset type;

[0174] In response to the operation of the target user confirming the transfer slots, determine the first processed data based on the transfer slots and the third processed data.

[0175] According to one or more embodiments of the present application, each data template in the set of data templates includes a second keyword and data fields of different types. The process of determining the preset set of data templates includes:

[0176] Determining a plurality of second keywords and the data fields corresponding to the plurality of second keywords based on the historical statements input by the user;

[0177] For any second keyword among the plurality of second keywords, using an editor to edit the second keyword and the data fields corresponding to the second keyword to determine a data template;

[0178] The obtaining of the corresponding target data template from the set of data templates based on the first keyword includes:

[0179] Determining the similarity between the first keyword and the plurality of second keywords;

[0180] Determining the second keyword corresponding to the maximum value in the similarities as the target keyword;

[0181] Determining the data template corresponding to the target keyword as the target data template;

[0182] The filling of the target data into the target data template includes:

[0183] Matching the field type of the target data with the data fields in the target data template;

[0184] When the matching is successful, filling the target data into the matching data fields in the target data template.

[0185] According to one or more embodiments of the present application, the mapping of the slot to the real value in the database through entity linking includes:

[0186] When the type of the slot is a dimension filtering condition, determining the operator position in the statement based on a preset set of operators;

[0187] Determining the dimension and / or dimension member in the statement based on the position of the operator;

[0188] Mapping the dimension and / or the dimension member to the real value in the database.

[0189] According to one or more embodiments of the present application, the data processing based on the second processed data to obtain the target data includes:

[0190] Performing a data query based on the second processed data to obtain the target data.

[0191] According to one or more embodiments of the present application, the data processing based on the second processed data to obtain the target data includes:

[0192] Performing a data query based on the second processed data to obtain basic data;

[0193] Performing analysis and calculation on the basic data to obtain the target data.

[0194] According to one or more embodiments of the present application, a data processing device is provided, and the device may include:

[0195] A first acquisition unit, configured to acquire a statement input by a target user, where the statement is used to indicate acquiring target data required by the target user;

[0196] A second acquisition unit, configured to acquire the intent type of the statement;

[0197] A labeling unit, configured to label slots in the statement based on the intent type of the statement;

[0198] A mapping unit, configured to map the slots in the statement to obtain first processed data;

[0199] A processing unit, configured to perform data processing based on the first processed data to obtain the target data.

[0200] In one or more embodiments of the present application, the second acquisition unit is specifically configured to perform intent recognition on the statement based on a text classification model to obtain the intent type of the statement, and the text classification model is trained based on training samples.

[0201] In one or more embodiments of the present application, the labeling unit is specifically configured to label slots in the statement by using a sequence labeling model based on the intent type of the statement, and the sequence labeling model is trained based on the training samples.

[0202] In one or more embodiments of the present application, the mapping unit is specifically configured to map the slots to real values in a database through entity linking, and use the real values to replace the slots to obtain the first processed data.

[0203] In one or more embodiments of the present application, the processing unit is specifically configured to process the first processed data based on a target data structure to obtain second processed data; perform data processing based on the second processed data to obtain the target data.

[0204] In one or more embodiments of the present application, the device further includes: a determination unit; after using a sequence annotation model to annotate the slots in the statement, the determination unit is configured to determine a first keyword in the annotated statement; and obtain a corresponding target data template from a data template set based on the first keyword, where the data template set is pre-determined based on historical statements input by users.

[0205] The device further includes: a filling unit; after obtaining the target data, the filling unit is configured to fill the target data into the target data template and send the filled target data template to the target user.

[0206] In one or more embodiments of the present application, the process of obtaining the training samples includes:

[0207] Based on historical statements input by users, determine slots of a preset type, a statement template set, and an intent type set corresponding to the statement template set.

[0208] Use the historical statements to fill the statement template set to obtain a statement set.

[0209] For any first statement in the statement set, based on the slots of the preset type, use BIO to annotate the first statement to obtain annotation information.

[0210] Based on multiple first statements, the annotation information respectively corresponding to the multiple first statements, and the intent types respectively corresponding to the multiple first statements, determine the training samples.

[0211] In one or more embodiments of the present application, the training process of the text classification model includes:

[0212] Input multiple first statements into an initial classification model to obtain a first encoded vector.

[0213] Use the initial classification model to perform feature extraction on the first encoded vector to obtain an intent type output value.

[0214] Based on the intent types respectively corresponding to the multiple first statements and the intent type output value, determine a first loss function, and based on the first loss function, adjust the parameters of the initial classification model, and re-execute the process of inputting the multiple first statements into the initial classification model and subsequent training processes until the first loss function is less than a first threshold to obtain the text classification model.

[0215] In one or more embodiments of the present application, the training process of the sequence annotation model includes:

[0216] Input multiple of the first statements into the initial annotation model to obtain second encoding vectors;

[0217] Use the initial annotation model to decode the second encoding vectors to obtain annotation information output values;

[0218] Based on the annotation information corresponding to multiple of the first statements and the annotation information output values, determine a second loss function, and based on the second loss function, adjust the parameters of the initial annotation model, and re - execute the process of inputting multiple of the first statements into the initial annotation model and subsequent training processes until the second loss function is less than a second threshold to obtain the sequence annotation model.

[0219] In one or more embodiments of the present application, when the statement is an incomplete statement, the mapping unit is specifically configured to map the slots in the statement to obtain third processing data; predict the transfer slots corresponding to the slots in the third processing data based on a data combination framework, where the data combination framework includes preset types of slots; in response to an operation in which the target user confirms the transfer slots, determine the first processing data based on the transfer slots and the third processing data.

[0220] In one or more embodiments of the present application, each data template in the data template set includes a second keyword and data fields of different types, and the determination process of the preset data template set includes:

[0221] Determine multiple second keywords and the data fields corresponding to the multiple second keywords based on the historical statements input by the user; for any one of the multiple second keywords, use an editor to edit the second keyword and the data fields corresponding to the second keyword to determine a data template;

[0222] The determination unit is specifically configured to: determine the similarity between the first keyword and the multiple second keywords; determine the second keyword corresponding to the maximum value in the similarities as the target keyword; determine the data template corresponding to the target keyword as the target data template;

[0223] The filling unit is specifically configured to: match the field type of the target data with the data fields in the target data template; when the match is successful, fill the target data into the data fields in the target data template that match.

[0224] In one or more embodiments of the present application, the mapping unit is specifically configured to, when the type of the slot is a dimension filtering condition, determine the operator position in the statement based on a preset operator set; determine the dimension and / or dimension member in the statement based on the position of the operator; and map the dimension and / or the dimension member to the real value in the database.

[0225] In one or more embodiments of the present application, the processing unit is specifically configured to perform a data query based on the second processed data to obtain the target data.

[0226] In one or more embodiments of the present application, the processing unit is specifically configured to perform a data query based on the second processed data to obtain the basic data; and perform analysis and calculation on the basic data to obtain the target data.

[0227] According to one or more embodiments of the present application, there is provided an electronic device, which includes: a processor and a memory;

[0228] The memory is used to store instructions or computer programs;

[0229] The processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes the data processing method.

[0230] According to one or more embodiments of the present application, there is provided a computer-readable storage medium, in which instructions are stored, and when the instructions run on a device, the device is enabled to execute the data processing method.

[0231] It should be noted that the various embodiments in this specification are described in a progressive manner, and the key points of each embodiment are the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions in the method part.

[0232] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item) of the following" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0233] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0234] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the technical field.

[0235] The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, characterized in that, The method comprises: Acquire a statement input by a target user, wherein the statement is used to instruct acquisition of target data required by the target user; Performing intent recognition on the sentence based on a text classification model to obtain the intent type of the sentence, wherein the text classification model is obtained by training based on training samples; Based on the intent type of the sentence, label the slots in the sentence using a sequence labeling model, where the sequence labeling model is trained based on the training sample; Mapping the slots in the statement to obtain first processing data; Performing data processing based on the first processed data to obtain the target data; Wherein, when the statement is an incomplete statement, mapping the slots in the statement to obtain the first processing data includes: Mapping the slots in the statement to obtain third processing data; Predicting a transfer slot corresponding to a slot in the third processed data based on a data combination framework, wherein the data combination framework includes slots of a preset type; In response to the target user confirming an operation of the transfer slot, the first processing data is determined based on the transfer slot and the third processing data.

2. The method according to claim 1, wherein Mapping the slots in the statement to obtain first processing data includes: The slot is mapped to a real value in a database through entity linking, and the slot is replaced with the real value to obtain the first processed data.

3. The method according to claim 1, wherein The performing data processing based on the first processed data to obtain the target data includes: Process the first processed data based on the target data structure to obtain second processed data; Data processing is performed based on the second processed data to obtain the target data.

4. The method according to claim 1, characterized in that, After labeling the slots in the sentence using the sequence labeling model, the method further includes: Determining the first keyword in the annotated sentence; Acquire a corresponding target data template from a data template set based on the first keyword, wherein the data template set is predetermined based on a historical sentence input by a user; After acquiring the target data, the method further includes: The target data is filled into the target data template, and the filled target data template is sent to the target user.

5. The method according to claim 1, characterized in that, The process of obtaining the training samples includes: Based on the historical sentences input by the user, determine a preset type of slot, a sentence template set, and an intention type set corresponding to the sentence template set; Filling the statement template set with the historical statements to obtain a statement set; For any first statement in the statement set, based on the slot of the preset type, the first statement is annotated by using BIO to obtain annotation information; The training sample is determined based on the plurality of first sentences, the annotation information respectively corresponding to the plurality of first sentences, and the intent types respectively corresponding to the plurality of first sentences.

6. The method according to claim 5, characterized in that The training process of the text classification model includes: Inputting a plurality of the first sentences into an initial classification model to obtain a first encoding vector; Using the initial classification model to perform feature extraction on the first encoding vector to obtain an intent type output value; Based on the intent types respectively corresponding to the multiple first statements and the output values of the intent types, determine a first loss function, and based on the first loss function, adjust the parameters of the initial classification model, and re - execute the process of inputting the multiple first statements into the initial classification model and subsequent training processes until the first loss function is less than a first threshold to obtain the text classification model.

7. The method according to claim 5, wherein The training process of the sequence labeling model includes: Input the multiple first statements into an initial labeling model to obtain second encoded vectors; Use the initial labeling model to decode the second encoded vectors to obtain annotation information output values; Based on the annotation information respectively corresponding to the multiple first statements and the annotation information output values, determine a second loss function, and based on the second loss function, adjust the parameters of the initial labeling model, and re - execute the process of inputting the multiple first statements into the initial labeling model and subsequent training processes until the second loss function is less than a second threshold to obtain the sequence labeling model.

8. The method according to claim 4, characterized in that, Each data template in the data template set includes a second keyword and data fields of different types. The determination process of the data template set includes: Determine multiple second keywords and the data fields corresponding to the multiple second keywords based on the historical statements input by the user; For any second keyword among the multiple second keywords, use an editor to edit the second keyword and the data fields corresponding to the second keyword to determine a data template; The obtaining of the corresponding target data template from the data template set based on the first keyword includes: Determine the similarity between the first keyword and the multiple second keywords; Determine the second keyword corresponding to the maximum value in the similarities as the target keyword; Determine the data template corresponding to the target keyword as the target data template; The filling of the target data into the target data template includes: Match the field type of the target data with the data fields in the target data template; When the match is successful, fill the target data into the data fields in the target data template that match.

9. The method according to claim 2, wherein The mapping of the slot to the real value in the database through entity linking includes: When the type of the slot is a dimension filtering condition, determine the operator position in the statement based on a preset operator set; Determine the dimension and / or dimension members in the statement based on the position of the operator; Map the dimension and / or the dimension members to the real value in the database.

10. The method according to claim 3, wherein The data processing based on the second processed data to obtain the target data includes: Perform data query based on the second processed data to obtain the target data.

11. The method according to claim 3, wherein The data processing based on the second processed data to obtain the target data includes: Perform data query based on the second processed data to obtain basic data; Analyze and calculate the basic data to obtain the target data.

12. A data processing device, characterized in that, The device includes: A first acquisition unit, configured to acquire a statement input by a target user, where the statement is used to indicate acquiring target data required by the target user; A second acquisition unit, configured to perform intent recognition on the statement based on a text classification model to obtain an intent type of the statement, where the text classification model is trained based on training samples; A labeling unit, configured to label slots in the statement based on the intent type of the statement by using a sequence labeling model, where the sequence labeling model is trained based on the training samples; A mapping unit, configured to map the slots in the statement to obtain first processed data; A processing unit, configured to perform data processing based on the first processed data to obtain the target data; Wherein, when the statement is an incomplete statement, the mapping unit is specifically configured to map the slots in the statement to obtain third processed data; predict a transfer slot corresponding to the slot in the third processed data based on a data combination framework, where the data combination framework includes preset types of slots; in response to an operation by the target user to confirm the transfer slot, determine the first processed data based on the transfer slot and the third processed data.

13. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is configured to store instructions or computer programs; The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium, and when the instructions run on the device, the device executes the method according to any one of claims 1-11.

Images