Data processing method and device, storage medium and electronic equipment

Abstract

This application discloses a data processing method, apparatus, storage medium, and electronic device. Relating to the field of artificial intelligence, the method includes: receiving a data query request from a business user; extracting the business type of the data to be queried from the query request; determining the job type of the business user; inputting the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained from multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; determining the data source associated with each data label in the set of data labels to obtain at least one target data source; retrieving the data to be queried from the target data table of each target data source based on the set of data labels, and displaying the data to be queried. This application solves the problem of low efficiency when querying data from multiple data sources in related technologies.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a data processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] In related technologies, with the advancement of information technology, data resources are accumulating more and more. Business personnel in companies have numerous frequent and personalized data acquisition needs when handling daily business. Since business personnel lack data processing skills, they submit data acquisition requirements to technical personnel in a request-based manner. Technical personnel then customize the data source to generate the data needed by the business personnel. For example, business personnel A's data acquisition requirement is a combination of columns a, b, and c in a table; business personnel B's requirement is a combination of columns b, c, and d; and business personnel C's requirement is a combination of columns a, b, c, and d. When technical personnel customize and generate the data needed by the business personnel, the content of the data acquisition cannot be flexibly combined.

[0003] There is currently no effective solution to the problem of low efficiency when querying data from multiple data sources in related technologies. Summary of the Invention

[0004] The main objective of this application is to provide a data processing method, apparatus, storage medium, and electronic device to solve the problem of low efficiency when querying data from multiple data sources in related technologies.

[0005] To achieve the above objectives, according to one aspect of this application, a data processing method is provided. The method includes: receiving a data query request from a business user; extracting the business type of the data to be queried from the data query request; determining the job type of the business user; inputting the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained from multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; determining the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; retrieving the data to be queried from the target data table of each target data source based on the set of data labels, and displaying the data to be queried.

[0006] Optionally, obtaining the data to be queried from the target data table of each target data source based on a set of data tags includes: determining the target data tags associated with the target data source, determining the target data table to which each target data tag belongs in the target data source; extracting the initial data corresponding to the target data tags from the target data table; and performing data filtering processing on the initial data to obtain the data to be queried.

[0007] Optionally, the initial data is filtered to obtain the data to be queried, including: extracting query conditions and filtering conditions from the data query request, wherein the query conditions or filtering conditions include at least one of the following: time range, data type, and transaction channel; extracting data that meets the query conditions from each target data source to obtain target data; combining the target data from all target data sources to obtain a target data set; and removing preset data from the target data set based on the filtering conditions to obtain the data to be queried, wherein the preset data is data that does not meet the data requirements of the business type.

[0008] Optionally, the data labels are obtained by: identifying at least one data source, wherein each data source contains multiple data tables; generating a first data label based on the common data characteristics of all data in the Nth column of each data table, wherein N is a positive integer; generating a second data label based on the common data characteristics of all data in the Mth row of each data table, wherein M is a positive integer; and combining the first data label and the second data label to form the data label of the data table.

[0009] Optionally, the target model is obtained by: acquiring historical data query records, and extracting from the historical data query records the historical job type of the queryer, the historical business type of the data application, and a set of historical data tags associated with the queried historical data for each data query record; determining the historical business type, historical job type, and a set of historical data tags for each data query record as a set of training samples to obtain multiple sets of training samples; training the neural network model through multiple sets of training samples to obtain the trained target model.

[0010] Optionally, after displaying the data to be queried, the method further includes: extracting data application requirements from the data query request, determining whether the data to be queried meets the data application requirements; and updating the data tags associated with the data source based on the data application requirements if the data to be queried does not meet the data application requirements of the business personnel.

[0011] Optionally, displaying the data to be queried includes: extracting the data display type from the data query request, wherein the data display type includes at least one of the following: charts and reports; and displaying the data to be queried according to the data display type.

[0012] To achieve the above objectives, according to another aspect of this application, a data processing apparatus is provided. The apparatus includes: a receiving unit, configured to receive a data query request from a business user, extract the business type of the data to be queried from the data query request, and determine the job type of the business user; an input unit, configured to input the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained from multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; a first determining unit, configured to determine the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; and an obtaining unit, configured to obtain the data to be queried from the target data table of each target data source based on the set of data labels, and display the data to be queried.

[0013] This application employs the following steps: receiving data query requests from business personnel; extracting the business type of the data to be queried from the data query requests; determining the job type of the business personnel; inputting the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained from multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; determining the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; retrieving the data to be queried from the target data tables of each target data source based on the set of data labels, and displaying the data to be queried, thus solving the problem of low efficiency when querying data from multiple data sources in related technologies. By establishing data labels for all data sources, and automatically obtaining a set of data labels based on the job type and business type of the business personnel input into the target model, the efficiency and flexibility of retrieving data from multiple data sources are improved. Attached Figure Description

[0014] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0015] Figure 1 This is a flowchart of a data processing method provided according to an embodiment of this application;

[0016] Figure 2 This is a schematic diagram of an optional data processing system provided according to an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of a data processing apparatus provided according to an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0023] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a data processing method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0024] Step S101: Receive data query requests from business personnel, extract the business type of the data to be queried from the data query requests, and determine the job type of the business personnel.

[0025] Specifically, business personnel can be employees of financial institutions. In their daily business operations, they need to access data from various data sources within the financial institution to complete tasks. The business type to which the queried data is applied refers to the type of business the business personnel will apply the retrieved data to. When processing business, the business personnel initiate a data query request to the financial institution's business system to retrieve the required data. Upon receiving the data query request, the business system extracts the business type to which the data will be applied. Different business types use different data; therefore, the system can recommend queried data that matches the business type in the data query request. Furthermore, it can also recommend data related to the business personnel's job title.

[0026] Step S102: Input the business type and job type into the target model to obtain a set of data labels. The target model is trained by multiple sets of training samples. Each set of training samples includes historical business types, historical job types and a set of historical data labels.

[0027] Specifically, by inputting the job types of the acquired business personnel and the business types to which the queried data is applied into the target model, a set of data labels matching the business type and job type is obtained and recommended to the business personnel based on the target model. The target model can be a model trained using a machine learning clustering algorithm. For example, if business personnel A's historical job type is "general employee" and the historical business type to which the data is applied is "business X", a set of corresponding historical data labels is generated based on the data frequently used by general employees and business X.

[0028] It should be noted that the data tags can be the content of a column in a database, the result of operations on a data table using aggregation statements (SQL statements), or the results retrieved from Elasticsearch (a full-text search and analysis engine) based on a certain condition. The data tags in this embodiment have many advantages, including cross-database compatibility, flexible combination, and convenient scalability.

[0029] Step S103: Determine the data source associated with each data tag in a set of data tags to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data tags.

[0030] Specifically, different data labels in a set of data labels can be labels from the same data source or labels from different data sources, such as databases of financial institutions and full-text search analysis engines. After obtaining a set of data labels output by the target model, in order to obtain the data to be queried, it is necessary to first determine the target data source associated with each data label, and then extract the data to be queried from the target data table to which the data label belongs in the target data source.

[0031] Step S104: Obtain the data to be queried from the target data table of each target data source based on a set of data labels, and display the data to be queried.

[0032] Specifically, each target data source contains multiple data tables, and the data tags belong to the target data tables in multiple data tables. After the target data source is determined, the data associated with the data tags is extracted from the target data tables. After all the data in the target data tables of all target data sources are extracted, the extracted data is processed to obtain the data to be queried, and the data to be queried is summarized into charts or reports for business personnel to use.

[0033] The data processing method provided in this application embodiment receives data query requests from business personnel, extracts the business type of the data to be queried from the data query request, and determines the job type of the business personnel. The business type and job type are input into a target model to obtain a set of data labels. The target model is trained from multiple sets of training samples, each set including historical business types, historical job types, and a set of historical data labels. The data source associated with each data label in the set of data labels is determined, resulting in at least one target data source. Each data source contains multiple data tables, and each data table contains multiple data labels. Based on the set of data labels, the data to be queried is retrieved from the target data tables of each target data source, and the queried data is displayed. This solves the problem of low efficiency when querying data from multiple data sources in related technologies. By establishing data labels for all data sources, and automatically obtaining a set of data labels based on the job type and business type of the business personnel input into the target model, the queried data from each data source is retrieved using this set of data labels, thereby improving the efficiency and flexibility of retrieving data from multiple data sources.

[0034] The data extracted from the target data source needs to be filtered before the data to be queried can be obtained. Optionally, in the data processing method provided in this application embodiment, obtaining the data to be queried from the target data table of each target data source based on a set of data tags includes: determining the target data tags associated with the target data source, determining the target data table to which each target data tag belongs in the target data source; extracting the initial data corresponding to the target data tag from the target data table; and performing data filtering processing on the initial data to obtain the data to be queried.

[0035] Specifically, a target data tag is a data tag from a target data source within a set of data tags. Each target data source contains multiple data tables, and each data table also contains multiple data tags. Therefore, when retrieving query data from the target data table of each target data source based on a set of data tags, it is necessary to determine the target data table of each target data tag in the target data source one by one. Then, the initial data corresponding to the target data tag is extracted from the target data table. Only after filtering and processing the initial data can the query data be obtained. Retrieving query data based on data tags can help business personnel query and process large amounts of data more quickly. In business analysis scenarios and basic operation and maintenance scenarios, it is convenient for non-technical personnel to obtain customized data that meets business needs in a simple way.

[0036] For example, a set of data labels contains m, n, and s tags. The m and n tags originate from database C, and the s tag originates from database O. The m tag is located in column m of table c in database C, the n tag in column n of table c in database C, and the s tag in column s of table o in database O. By extracting data from columns m and n of table c in database C and column s of table o in database O, initial data is obtained. After filtering this initial data, the data to be queried is obtained.

[0037] After obtaining the initial data, the initial data is filtered using query and filtering conditions. Optionally, in the data processing method provided in this application embodiment, filtering the initial data to obtain the data to be queried includes: extracting query and filtering conditions from the data query request, wherein the query or filtering conditions include at least one of the following: time range, data type, and transaction channel; extracting data that meets the query conditions from each target data source to obtain target data; combining the target data from all target data sources to obtain a target data set; and removing preset data from the target data set based on the filtering conditions to obtain the data to be queried, wherein the preset data is data that does not meet the data requirements of the business type.

[0038] Specifically, to facilitate data retrieval for business personnel, the business system also includes a tag query and filtering module. This module allows for data querying and filtering based on data tags, enabling advanced queries and filtering through combinations of multiple tags. Query conditions can be defined by data tags, or by time range, data type, and transaction channel. For example, if the target data tag is transaction volume, initial data related to transaction volume can be extracted using this tag. By specifying various query conditions such as time range, transaction type, and transaction channel, target data that meets the query criteria can be found. The more query conditions, the higher the accuracy of the retrieved data. The target data retrieved from each target data source is combined to obtain a target data set. Business personnel can also filter out unwanted pre-defined data from the target data set based on filtering conditions, which can also be defined by data tags, or by time range, data type, and transaction channel. After removing pre-received data, the final data required by the business personnel can be obtained. By filtering the initial data, the accuracy of the retrieved data is significantly improved.

[0039] For example, the target data label is transaction volume, and the query condition is the transaction volume of K-type goods traded online in the past month. The initial data related to transaction volume is extracted from the target data table of the target data source by querying the target data to obtain the target data. The filtering condition is to remove the transaction volume data in the past week and the transaction volume data that is less than or equal to the transaction volume threshold. After removing the preset data that meets the above filtering conditions from the target data, the data to be queried is obtained.

[0040] It should be noted that business personnel can fill in query or filter conditions according to their business needs. They do not need to fill in all conditions to extract data within a suitable range. If the target data tag does not meet the business personnel's query or filter conditions, technical personnel only need to optimize the target data tag and add search conditions for that tag.

[0041] It should be noted that the following is a code example, using a database as an example. The field queried (chg_value as transaction volume) is the tag. <if>The content inside contains the selection criteria for the tag. Technical personnel can further expand the applicable scenarios for this tag by extending the query logic and conditions.

[0042]

[0043] Data tags are generated from the data characteristics of each row or column in a data table. Optionally, in the data processing method provided in this application embodiment, data tags are obtained in the following manner: determining at least one data source, wherein each data source contains multiple data tables; generating a first data tag based on the common data characteristics of all data in the Nth column of each data table, wherein N is a positive integer; generating a second data tag based on the common data characteristics of all data in the Mth row of each data table, wherein M is a positive integer; and combining the first data tag and the second data tag into a data tag for the data table.

[0044] Specifically, the business system provides automated label generation and management functions, reducing the workload of manually labeling data. The label generation and classification module is responsible for automatically generating categories based on the content and characteristics of each data table in the data source, performing pre-classification management of labels. For example, data table c in data source C records transaction data between institution L and institution P. In data table c, the first row records the transaction object, the second row records the transaction time, the third row records the transaction frequency, the first column records the transaction volume, and the second column records the transaction method. Data labels are generated based on the data characteristics of each row and column in the data table. The data in the first column of data table c generates the first data label "transaction volume," the data in the second column generates the first data label "transaction method," the data in the first row generates the second data label "transaction object," the data in the second row generates the second data label "transaction time," and the data in the third row generates the second data label "transaction frequency." These first and second data labels together constitute the data labels for data table c. Automated label generation and management can reduce the error rate of manually labeling data and improve the efficiency and accuracy of data management.

[0045] Before obtaining a set of data labels, a target model needs to be trained. Optionally, in the data processing method provided in this application embodiment, the target model is obtained in the following way: obtaining historical data query records, and extracting from the historical data query records the historical job type of the queryer, the historical business type of the data application, and a set of historical data labels associated with the queried historical data of each data query record; determining the historical business type, historical job type, and a set of historical data labels of each data query record as a set of training samples to obtain multiple sets of training samples; training the neural network model through multiple sets of training samples to obtain the trained target model.

[0046] Specifically, historical data query records can be records of data queries by business personnel recorded in the business system. The neural network model can be a backpropagation (BP) neural network model or a convolutional neural network model. For example, if a department manager queries data related to business m, the historical business type would be business m, the historical job type would be department manager, and the historical data tags would be the tags associated with the queried data. The neural network model is trained using multiple sets of training samples. That is, based on the job type of the querying personnel, the combination of tags in the queried data, and the applicable business scenario (business type), the neural network model is trained to obtain the target model. The target model then uses a recommendation algorithm to recommend combined tags—that is, a set of data tags—to business personnel, helping them efficiently obtain accurate data reports.

[0047] Data tags can also be optimized based on feedback from business personnel. Optionally, in the data processing method provided in this application embodiment, after displaying the data to be queried, the method further includes: extracting data application requirements from the data query request, determining whether the data to be queried meets the data application requirements; and updating the data tags associated with the data source based on the data application requirements if the data to be queried does not meet the data application requirements of the business personnel.

[0048] Specifically, when business users initiate a data query request, they can add data application requirements, which are the conditions that the queried data must meet when applied to the business. After retrieving the data from various data sources using a set of data tags, it is determined whether the data can serve the business users' data application requirements. If data that meets the requirements exists, the required data report or chart is generated. If no data that meets the requirements exists, the development team updates the data tags and adds tags that match the application requirements. By optimizing data tags, more accurate data is provided to business users, improving business efficiency.

[0049] For example, the data application requirement is set to ensure that the data to be queried is compatible with version W of the Q software. If all the data to be queried retrieved from multiple data sources using a set of data tags is not compatible with version W of the Q software, then the data tags are optimized by adding data tags of types compatible with version W of the Q software, so that data that meets the data application requirements can be retrieved the next time data is queried.

[0050] The business system also provides diverse display methods for the data to be queried, such as charts or reports. Optionally, in the data processing method provided in this application embodiment, displaying the data to be queried includes: extracting the data display type from the data query request, wherein the data display type includes at least one of the following: charts and reports; and displaying the data to be queried according to the data display type.

[0051] Specifically, the business system also includes a data visualization and analysis module, used to present the data to be queried as charts or reports to help business personnel better understand and analyze the data. Data visualization and analysis capabilities can also improve business personnel's ability to understand and analyze data, further enhancing the quality and efficiency of data processing.

[0052] According to another embodiment of this application, a data processing system is also provided. Figure 2 This is a schematic diagram of an optional data processing system provided according to an embodiment of this application. For example... Figure 2 As shown, the system includes:

[0053] The tag generation and classification module 201 is used to automatically generate classifications based on the content and characteristics of the data table involved in the tag, and to perform pre-classification management of the tag;

[0054] For example, data table c in data source C records transaction data between institution L and institution P. The first row of table c records the transaction parties, the second row records the transaction time, the third row records the transaction frequency, the first column records the transaction volume, and the second column records the transaction method. Labels are generated based on the data characteristics of each row and column in the data table. The data in the first column of table c generates the first label "Transaction Volume," the data in the second column generates the first label "Transaction Method," the data in the first row generates the second label "Transaction Parties," the data in the second row generates the second label "Transaction Time," and the data in the third row generates the second label "Transaction Frequency." These first and second labels together constitute the labels for data table c.

[0055] The tag query and filtering module 202 is used to query and filter data based on tags. Users can perform advanced queries by combining multiple tags.

[0056] Specifically, users can perform advanced queries and filters using combinations of multiple tags. Query conditions can be defined by data tags, or based on time range, data type, and transaction channel. For example, if the target data tag is transaction volume, initial data related to transaction volume can be extracted using this tag. By specifying various query conditions such as time range, transaction type, and transaction channel, target data that meets the query conditions can be found. The more query conditions, the higher the accuracy of the retrieved data. The target data retrieved from each target data source is combined to obtain a target data set. Users can also filter out unwanted pre-defined data from the target data set based on filtering conditions, which can also be defined by data tags, or based on time range, data type, and transaction channel. After removing pre-received data, the final query data required by the user is obtained.

[0057] The data visualization and analysis module 203 is used to present data as charts or reports to help users better understand and analyze the data.

[0058] Specifically, the retrieved data is automatically analyzed and processed to generate charts or reports according to the user's needs.

[0059] The data processing system provided in this application helps users more easily query and process large amounts of data. The system uses tags to organize data, allowing users to query and filter based on custom tags. For example, tags a, b, c, and d can be displayed separately to meet the personalized combinations of business personnel and ultimately generate the desired data retrieval reports.

[0060] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0061] This application also provides a data processing apparatus. It should be noted that the data processing apparatus of this application can be used to execute the data processing method provided in this application. The data processing apparatus provided in this application will be described below.

[0062] Figure 3 This is a schematic diagram of a data processing apparatus provided according to an embodiment of this application. For example... Figure 3 As shown, the device includes:

[0063] The receiving unit 10 is used to receive data query requests from business personnel, extract the business type of the data to be queried from the data query request, and determine the job type of the business personnel.

[0064] Input unit 20 is used to input business type and job type into target model to obtain a set of data labels. The target model is trained by multiple sets of training samples. Each set of training samples includes historical business type, historical job type and a set of historical data labels.

[0065] The first determining unit 30 is used to determine the data source associated with each data tag in a set of data tags, and to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data tags.

[0066] The acquisition unit 40 is used to retrieve the data to be queried from the target data table of each target data source based on a set of data labels, and to display the data to be queried.

[0067] The data processing apparatus provided in this application embodiment receives data query requests from business personnel through a receiving unit 10, extracts the business type of the data to be queried from the data query request, and determines the job type of the business personnel; an input unit 20 inputs the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained by multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; a first determining unit 30 determines the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; an obtaining unit 40 obtains the data to be queried from the target data table of each target data source based on a set of data labels and displays the data to be queried. This solves the problem of low efficiency when querying data from multiple data sources in related technologies. By establishing data labels for all data sources, a set of data labels is automatically obtained based on the job type and business type of the business personnel input into the target model, and the data to be queried in each data source is obtained through a set of data labels, thereby improving the efficiency and flexibility of obtaining data from multiple data sources.

[0068] Optionally, in the data processing apparatus provided in this application embodiment, the acquisition unit 40 includes: a first determining module, used to determine the target data tag associated with the target data source, and to determine the target data table to which each target data tag belongs in the target data source; a first extraction module, used to extract the initial data corresponding to the target data tag from the target data table; and a processing module, used to perform data filtering processing on the initial data to obtain the data to be queried.

[0069] Optionally, in the data processing apparatus provided in this application embodiment, the processing module includes: a first extraction submodule, used to extract query conditions and filtering conditions from a data query request, wherein the query conditions or filtering conditions include at least one of the following: time range, data type, and transaction channel; a second extraction submodule, used to extract data that meets the query conditions from each target data source to obtain target data; a combination submodule, used to combine the target data from all target data sources to obtain a target data set; and a removal submodule, used to remove preset data from the target data set based on filtering conditions to obtain data to be queried, wherein the preset data is data that does not meet the data requirements of the business type.

[0070] Optionally, in the data processing apparatus provided in this application embodiment, the data tag is obtained by: determining at least one data source, wherein each data source contains multiple data tables; generating a first data tag based on the common data characteristics of all data in the Nth column of each data table, wherein N is a positive integer; generating a second data tag based on the common data characteristics of all data in the Mth row of each data table, wherein M is a positive integer; and combining the first data tag and the second data tag into a data tag for the data table.

[0071] Optionally, in the data processing apparatus provided in this application embodiment, the target model is obtained in the following manner: acquiring historical data query records, and extracting from the historical data query records the historical job type of the queryer, the historical business type of the data application, and a set of historical data tags associated with the queried historical data for each data query record; determining the historical business type, historical job type, and a set of historical data tags for each data query record as a set of training samples to obtain multiple sets of training samples; training the neural network model through multiple sets of training samples to obtain the trained target model.

[0072] Optionally, in the data processing apparatus provided in this application embodiment, the apparatus further includes: an extraction unit, used to extract data application requirements from a data query request and determine whether the data to be queried meets the data application requirements; and an update unit, used to update the data tags associated with the data source based on the data application requirements when the data to be queried does not meet the data application requirements of business personnel.

[0073] Optionally, in the data processing apparatus provided in this application embodiment, the acquisition unit 40 includes: a second extraction module, used to extract the data display type from the data query request, wherein the data display type includes at least one of the following: charts and reports; and a display module, used to display the data to be queried according to the data display type.

[0074] The data processing device includes a processor and a memory. The receiving unit 10, the input unit 20, the first determining unit 30, and the acquiring unit 40 are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0075] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve efficiency and flexibility when acquiring data from multiple data sources.

[0076] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0077] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a data processing method.

[0078] This invention provides a processor for running a program, wherein the program executes a data processing method during runtime.

[0079] Figure 4 This is a schematic diagram of an electronic device provided according to an embodiment of this application. For example... Figure 4 As shown, electronic device 401 includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: receiving a data query request from a business user; extracting the business type of the data to be queried from the data query request; determining the job type of the business user; inputting the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained from multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; determining the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; retrieving the data to be queried from the target data table of each target data source based on the set of data labels, and displaying the data to be queried. The device in this document can be a server, PC, PAD, mobile phone, etc.

[0080] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: receiving a data query request from a business person; extracting the business type of the data to be queried from the data query request; determining the job type of the business person; inputting the business type and job type into a target model to obtain a set of data labels, wherein the target model is trained by multiple sets of training samples, each set of training samples including historical business types, historical job types, and a set of historical data labels; determining the data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source contains multiple data tables, and each data table contains multiple data labels; retrieving the data to be queried from the target data table of each target data source based on the set of data labels, and displaying the data to be queried.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0087] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0089] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.< / if>

Claims

1. A data processing method, characterized by, The method comprises: receiving a data query request of a service personnel, extracting a service type of a data application to be queried from the data query request, and determining a position type of the service personnel; inputting the service type and the position type into a target model to obtain a set of data labels, wherein the target model is trained by a plurality of training samples, and each training sample comprises a historical service type, a historical position type, and a set of historical data labels; determining a data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source comprises a plurality of data tables, and each data table comprises a plurality of data labels; obtaining the data to be queried from a target data table of each target data source based on the set of data labels, and displaying the data to be queried; wherein the target model is obtained by: obtaining historical data query records, and extracting a historical position type of a query personnel, a historical service type of a data application, and a set of historical data labels associated with historical data queried each time from the historical data query records; determining the historical service type, the historical position type, and the set of historical data labels of each data query record as a set of training samples to obtain the plurality of training samples; and training a neural network model by using the plurality of training samples to obtain the trained target model.

2. The method of claim 1, wherein, Obtaining the data to be queried from a target data table of each target data source based on the set of data labels comprises: determining a target data label associated with the target data source, and determining a target data table to which each target data label belongs in the target data source; extracting initial data corresponding to the target data label from the target data table; performing data filtering processing on the initial data to obtain the data to be queried.

3. The method of claim 2, wherein, Performing data filtering processing on the initial data to obtain the data to be queried comprises: extracting a query condition and a filtering condition from the data query request, wherein the query condition or the filtering condition comprises at least one of the following: a time range, a data type, and a transaction channel; extracting data satisfying the query condition from each target data source to obtain target data; combining the target data of all target data sources to obtain a target data set; based on the filtering condition, eliminating preset data from the target data set to obtain the data to be queried, wherein the preset data is data that does not meet the data requirement of the service type.

4. The method of claim 1, wherein, The data label is obtained by: determining at least one data source, wherein each data source comprises a plurality of data tables; generating a first data label based on the same data features of all data in the Nth column of each data table, wherein N is a positive integer; generating a second data label based on the same data features of all data in the Mth row of each data table, wherein M is a positive integer; combining the first data label and the second data label as the data label of the data table.

5. The method of claim 1, wherein, After displaying the data to be queried, the method further comprises: extract a data application requirement from the data query request, and determine whether the to-be-queried data meets the data application requirement; in a case where the to-be-queried data does not meet the data application requirement of the business personnel, updating a data label associated with the data source based on the data application requirement.

6. The method of claim 1, wherein, displaying the to-be-queried data comprises: extracting a data display type from the data query request, wherein the data display type at least comprises one of a chart and a report; displaying the to-be-queried data according to the data display type.

7. A data processing apparatus, characterized by, comprises: a receiving unit configured to receive a data query request of a business personnel, extract a business type of a to-be-queried data application from the data query request, and determine a position type of the business personnel; an input unit configured to input the business type and the position type into a target model to obtain a set of data labels, wherein the target model is trained by a plurality of sets of training samples, and each set of training samples comprises a historical business type, a historical position type, and a set of historical data labels; a first determining unit configured to determine a data source associated with each data label in the set of data labels to obtain at least one target data source, wherein each data source comprises a plurality of data tables, and each data table comprises a plurality of data labels; an obtaining unit configured to obtain the to-be-queried data from a target data table of each target data source based on the set of data labels, and display the to-be-queried data; wherein the target model is obtained by: obtaining historical data query records, and extracting a historical position type of a query personnel, a historical business type of a data application, and a set of historical data labels associated with queried historical data from each data query record in the historical data query records; determining the historical business type, the historical position type, and the set of historical data labels of each data query record as a set of training samples to obtain the plurality of sets of training samples; and training a neural network model by using the plurality of sets of training samples to obtain the trained target model.

8. A non-volatile storage medium, comprising: The non-volatile storage medium comprises a stored program, wherein the program controls a device in which the non-volatile storage medium is located to perform the data processing method of any one of claims 1 to 6 when the program is executed.

9. An electronic device, comprising: comprises one or more processors and a memory, and the memory is configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the data processing method of any one of claims 1 to 6.

Images