Data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. In the application, in response to a data processing request initiated by a target object for a target data source, identification information of the target data source and a data processing demand of the target object are acquired from the data processing request; then, selecting a target data source matched with the identification information from a preset data source set, and screening out a target data processing strategy associated with the data processing demand from a preset data processing strategy set; and finally, based on the target data processing strategy, performing data processing on the target data source to obtain a corresponding data processing result. By adopting the mode, data processing can be performed on the target data source according to the data processing request of the target object for the target data source, so that the personalized multi-dimensional statistical summary requirement of a user is met, and the flexibility of data processing is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of information technology, a large amount of business data continues to emerge, which makes the reasonable analysis and aggregation of data a common demand in all walks of life.

[0003] For example, see Figure 1 As shown, the existing data statistical measurement system includes three parts: a data source, an extraction module and an analysis and storage module; wherein the data source is used to access the original data sources to which multiple data to be measured belong, the extraction module is used to extract the data to be measured from each original data source included in the data source as the data to be measured, and the analysis and storage module is used to analyze the data to be measured to obtain corresponding measurement reports, and display the obtained measurement reports; it can be seen that the above-mentioned data statistical measurement system can aggregate data from multiple original data sources, thereby automatically realizing multi-dimensional analysis of data in multiple original data sources.

[0004] However, when the above method is used to perform statistical aggregation on data, the data is usually screened on the designated original data source, and then the original data source is statistically aggregated according to fixed dimensions, resulting in the statistical aggregation results being unable to meet the user's personalized multi-dimensional statistical aggregation needs. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device, electronic device and storage medium to meet the user's personalized multi-dimensional statistical aggregation needs, that is, to improve the flexibility of data processing.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising:

[0007] Responding to a data processing request initiated by a target object for a target data source, and obtaining identification information of the target data source and a data processing requirement of the target object from the data processing request; wherein the data processing requirement represents a data processing method of the target object for the target data source;

[0008] Selecting a target data source that matches the identification information from a preset data source set, and selecting a target data processing strategy associated with the data processing requirement from a preset data processing strategy set; wherein each data processing strategy includes: corresponding data statistical aggregation and analysis methods;

[0009] Based on the target data processing strategy, the target data source is processed to obtain the corresponding data processing results.

[0010] In a second aspect, an embodiment of the present application further provides a data processing device, the device comprising:

[0011] An acquisition module is used to respond to a data processing request initiated by a target object for a target data source, and acquire identification information of the target data source and a data processing requirement of the target object from the data processing request; wherein the data processing requirement represents a data processing method of the target object for the target data source;

[0012] A screening module is used to select a target data source that matches the identification information from a preset data source set, and to screen a target data processing strategy associated with the data processing requirement from a preset data processing strategy set; wherein each data processing strategy includes: a corresponding data statistical summary and analysis method;

[0013] The processing module is used to process the target data source based on the target data processing strategy to obtain the corresponding data processing results.

[0014] In a possible embodiment, before responding to the data processing request initiated by the target object for the target data source, the processing module is further configured to:

[0015] For each original data source, perform the following operations:

[0016] Obtaining each piece of raw data contained in an original data source, and cleaning each piece of raw data to obtain multiple pieces of raw data; wherein an original data source is a business data set;

[0017] Generate candidate data sources based on multiple raw data, and save the candidate data sources into a data source collection.

[0018] In a possible embodiment, when filtering out a target data processing strategy associated with a data processing requirement from a preset data processing strategy set, the filtering module is specifically used to:

[0019] Parse the data processing requirements to obtain at least one data indicator of the target object for the target data source; wherein each data indicator includes: at least one statistical dimension and one metric;

[0020] From the data processing strategy set, a data processing strategy including at least one data indicator is screened out, and the data processing strategy is used as a target data processing strategy.

[0021] In a possible embodiment, when performing data processing on a target data source based on a target data processing strategy to obtain a corresponding data processing result, the processing module is specifically used to:

[0022] Based on at least one data indicator included in the target data processing strategy, perform data segmentation processing on the target data source to obtain the target data source after the data segmentation processing;

[0023] Performing statistical aggregation on the target data source after the data segmentation processing to obtain statistical aggregation results corresponding to at least one data indicator;

[0024] Based on the data analysis method included in the target data processing strategy, data analysis is performed on at least one statistical summary result to obtain a data processing result.

[0025] In a possible embodiment, after performing data processing on the target data source based on the target data processing strategy to obtain corresponding data processing results, the device further includes a display module, and the display module is specifically used to:

[0026] In response to a display operation of a target object on a data processing result, a target data display template matching the display operation is selected from a preset data display template set;

[0027] Add each data analysis result included in the data processing result to the target data display template to generate a data display interface for the data processing result.

[0028] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the data processing method described in the first aspect above.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, which includes a program code. When the program code is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the data processing method described in the first aspect above.

[0030] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the data processing method as described in the first aspect.

[0031] The beneficial effects of this application are as follows:

[0032] In the data processing method provided in the present application, a data processing request initiated by a target object for a target data source is responded to, and identification information of the target data source and data processing requirements of the target object are obtained from the data processing request; then, a target data source matching the identification information is selected from a preset data source set, and a target data processing strategy associated with the data processing requirement is screened out from a preset data processing strategy set; finally, based on the target data processing strategy, data processing is performed on the target data source to obtain corresponding data processing results; in this way, data processing can be performed on the target data source according to the data processing request of the target object for the target data source, thereby meeting the user's personalized multi-dimensional statistical summary requirements and improving the flexibility of data processing.

[0033] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0035] Figure 1 A schematic diagram of the structure of a multi-dimensional data statistical measurement system provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of an optional system architecture applicable to the embodiments of the present application;

[0037] Figure 3 A schematic diagram of an implementation flow of a data processing method provided in an embodiment of the present application;

[0038] Figure 4 A logical schematic diagram for obtaining a candidate data source provided in an embodiment of the present application;

[0039] Figure 5 A logical schematic diagram of obtaining data processing results provided in an embodiment of the present application;

[0040] Figure 6 A schematic diagram of the structure of a multi-dimensional data statistics system provided in an embodiment of the present application;

[0041] Figure 7A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0042] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0044] It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. A and B are connected, which can represent two situations: A and B are directly connected and A and B are connected through C. In addition, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0045] In addition, the collection, dissemination, and use of data in the technical solution of this application comply with the requirements of relevant national laws and regulations.

[0046] The following is a brief introduction to the design concept of the embodiment of the present application:

[0047] At present, with the rapid development of information technology, a large amount of business data continues to emerge, and reasonable analysis and statistical summary of data has become a common demand in all walks of life.

[0048] In the related art, when statistically summarizing data, data is usually screened on a designated data table (ie, data source), and then statistically summarized according to fixed dimensions to obtain corresponding analysis results.

[0049] It can be seen that the filtering conditions for data screening and the associated indicator items (such as the dimensions and / or metrics of statistical summary, and the thresholds of each dimension and / or metric) cannot be flexibly configured, and the results of statistical summary cannot meet the user's personalized multi-dimensional summary analysis.

[0050] In view of this, in order to efficiently and accurately count and analyze large-scale data, provide important decision-making support and business optimization means for all walks of life, that is, to meet the user's personalized multi-dimensional statistical summary needs and improve the flexibility of data processing, the embodiment of the present application proposes a data processing method, which specifically includes: responding to a data processing request initiated by a target object for a target data source, and obtaining identification information of the target data source and the data processing requirements of the target object from the data processing request; wherein the data processing requirements characterize the data processing method of the target object for the target data source; then, from a preset data source set, select a target data source that matches the identification information, and from a preset data processing strategy set, screen out a target data processing strategy associated with the data processing requirement; wherein each data processing strategy includes: corresponding data statistical summary and analysis methods; finally, based on the target data processing strategy, perform data processing on the target data source to obtain corresponding data processing results.

[0051] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0052] See also Figure 2 As shown, it is a schematic diagram of a system architecture applicable to an embodiment of the present application, and the system architecture includes: a target terminal 201 and a server 202. The target terminal 201 and the server 202 can exchange information through a communication network, wherein the communication mode adopted by the communication network may include: a wireless communication mode and a wired communication mode.

[0053] Exemplarily, the target terminal 201 can access the network through cellular mobile communication technology and communicate with the server 202, wherein the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0054] Optionally, the target terminal 201 may access the network and communicate with the server 202 via a short-range wireless communication method, wherein the short-range wireless communication method, for example, includes Wireless Fidelity (Wi-Fi) technology.

[0055] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, there may be more target terminals, or no target terminals, or other network devices may be included, such as Figure 2As shown, only the target terminal 201 and the server 202 are described as examples, and the above-mentioned devices and their respective functions are briefly introduced below.

[0056] The target terminal 201 is a device that can provide voice and / or data connectivity to the user, and can be a device that supports wired and / or wireless connection.

[0057] Exemplarily, the target terminal 201 includes, but is not limited to: mobile phones, tablet computers, laptop computers, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0058] In addition, a related client may be installed on the target terminal 201, and the client may be software, for example, an application (APP), a browser, a short video software, etc., or a web page, a mini-program, etc.; it should be noted that in an embodiment of the present application, the target terminal 201 may enable the above-mentioned client related to data processing to send a data processing request initiated by the target object and a display operation for the data processing results to the server 202, etc., so as to perform subsequent data processing and display method steps.

[0059] Server 202 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0060] It is worth mentioning that in the embodiment of the present application, the server 202 is used to respond to the data processing request initiated by the target object for the target data source, and obtain the identification information of the target data source and the data processing requirements of the target object from the data processing request; then, from the preset data source set, select the target data source that matches the identification information, and from the preset data processing strategy set, filter out the target data processing strategy associated with the data processing requirement; finally, based on the target data processing strategy, perform data processing on the target data source to obtain the corresponding data processing result.

[0061] The data processing method provided by the exemplary embodiment of the present application is described below in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0062] See also Figure 3 As shown, it is a schematic diagram of an implementation process of a data processing method provided in an embodiment of the present application. The execution subject takes a server as an example. The specific implementation process of the method is as follows:

[0063] S301: Respond to a data processing request initiated by a target object for a target data source, and obtain identification information of the target data source and data processing requirements of the target object from the data processing request.

[0064] The data processing requirements represent the data processing method of the target object for the target data source, that is, the target object (ie, the user)'s personalized multi-dimensional statistical aggregation and analysis method for the target data source.

[0065] It should be noted that the target data source can be a legitimate business data set in any industry / field, and specifically can be internal data and external data of a relevant institution / organization.

[0066] Exemplarily, internal data includes: various business databases and systems collect various types of business data, such as sales data, customer data, supply chain data, etc., and also use sensors, monitoring equipment or IoT devices within the organization to collect real-time data; external data includes: third-party data, such as social media data, survey data, news and other data.

[0067] In an alternative implementation, see Figure 4 As shown, before responding to the data processing request initiated by the target object for the target data source, the server performs the following operations for each original data source: obtains each original data contained in an original data source, cleans each original data, obtains multiple original data, and generates candidate data sources based on the multiple original data, and saves the candidate data sources to a data source set; wherein an original data source is a business data set.

[0068] It should be noted that when the server cleans each piece of raw data, it removes duplicate data and erroneous data in the raw data, that is, it deduplicates and corrects errors in the raw data; optionally, the server can also interpolate or remove missing data to ensure the subsequent analysis and integration of standardized data formats.

[0069] Based on the above method, the server obtains a set of data sources in advance, and performs preprocessing and data cleaning operations on each original data source, which ensures the accuracy and completeness of the data, so that when a data processing request is subsequently initiated for the target data source according to the target object, the target data source can be accurately and quickly obtained from the data source set (i.e., multiple candidate data sources).

[0070] S302: Select a target data source that matches the identification information from a preset data source set, and select a target data processing strategy associated with the data processing requirement from a preset data processing strategy set.

[0071] Among them, each data processing strategy includes: corresponding data statistical aggregation and analysis methods; it should be noted that each data processing strategy is set according to corresponding statistical requirements and business rules, and contains multiple data indicators (i.e. dimensions and metrics) for describing various aspects of the data.

[0072] Exemplarily, when executing step S302, it is assumed that the preset data source set includes 5 candidate data sources, and the 5 candidate data sources and their corresponding identification information are as shown in Table 1:

[0073] Table 1

[0074] Candidate data source Data.So.1 Data.So.2 Data.So.3 Data.So.4 Data.So.5 Identification information ID.Me1 ID.Me2 ID.Me3 ID.Me4 ID.Me5

[0075] Based on the correspondence between the candidate data sources and the identification information in the data source set recorded in the above table, when the server obtains the identification information of the target data source from the data processing request initiated by the target object for the target data source, it can obtain the candidate data source that matches the identification information from the above data source set, and use the candidate data source as the target data source.

[0076] For example, assuming that the server obtains the identification information of the target data source as ID.Me3 from the data processing request initiated by the target object for the target data source, then based on the correspondence between the above candidate data sources and the identification information, it can be determined that the candidate data source corresponding to the identification information ID.Me3 is Data.So.3. Therefore, the candidate data source Data.So.3 can be used as the target data source.

[0077] In an optional implementation, after obtaining the data processing requirements of the target object from the data processing request initiated by the target object for the target data source, the server can parse the data processing requirements and obtain at least one data indicator of the target object for the target data source, thereby filtering out the data processing strategy containing at least one data indicator from the data processing strategy set, and using the data processing strategy as the target data processing strategy; wherein each data indicator includes: at least one statistical dimension and one metric, that is, a data indicator is usually composed of one or more dimensions plus a metric.

[0078] It should be noted that the metric is the numerical data in the data table, and the dimension is the categorical data in the data table.

[0079] It is not difficult to see that based on the above method, the server can quickly filter out the target data processing strategy that meets the data processing needs of the target object from the preset data processing strategy set, so as to meet the personalized multi-dimensional statistical summary needs of the target object.

[0080] It should also be noted that the above-mentioned preset data processing strategy set accurately unifies the data standard specifications (i.e., data processing strategies) of various business data and provides out-of-the-box data processing templates, which not only improves the efficiency of data processing, but also improves the data governance of related data to a certain extent.

[0081] S303: Based on the target data processing strategy, data processing is performed on the target data source to obtain corresponding data processing results.

[0082] In an alternative implementation, see Figure 5 As shown, when executing step S303, after obtaining the target data processing strategy, the server can perform data segmentation processing on the target data source based on at least one data indicator included in the target data processing strategy to obtain the target data source after the data segmentation processing; then, perform statistical aggregation on the target data source after the data segmentation processing to obtain statistical aggregation results corresponding to at least one data indicator; finally, based on the data analysis method included in the target data processing strategy, perform data analysis on at least one statistical aggregation result to obtain a data processing result.

[0083] It can be seen that the server performs data segmentation processing on the target data source based on at least one data indicator included in the target data processing strategy, and realizes slicing and dicing of the data according to different dimensions and metrics, which helps to subsequently perform statistical aggregation and analysis on the target data source after the data segmentation processing.

[0084] It should be noted that in the process of statistically aggregating the target data source after data segmentation, the above-mentioned server performs statistics, aggregation and analysis based on the dimensions in the data indicators. For example, for the time dimension, the data can be aggregated and compared according to the year, quarter or month; for another example, for the geographic location dimension, the data distribution analysis can be performed according to different regions; for another example, for the economic development level factor, a coefficient can be set to measure the statistics.

[0085] Furthermore, after step S303, the server can also respond to the target object's display operation on the data processing results, and filter out a target data display template that matches the display operation from a preset data display template set, thereby adding each data analysis result contained in the data processing result to the target data display template, and generating a data display interface for the data processing results; in this way, the target object can intuitively understand and analyze the statistical results of the data through the data display interface, which facilitates accurate and timely decision-making and planning.

[0086] It should be noted that through the user interaction interface (i.e., data display interface), the target object can also conveniently select dimensions, adjust statistical parameters, and view real-time statistical results; in addition, the user interaction interface (data display interface) can also provide visual operations, such as folding, pulling down, and dragging, etc., to facilitate the target object to clearly view the data and facilitate analysis and application.

[0087] Exemplarily, the above data processing results can be presented in the form of reports, charts, etc., so that the target object can intuitively understand and analyze the data.

[0088] Based on the data processing method described in the above steps, refer to Figure 6 As shown, the embodiment of the present application provides a multi-dimensional data statistics system, including: a data acquisition module 601, a data cleaning module 602, a data storage module 603, a statistical analysis module 604 and a system application module 605; wherein:

[0089] The data collection module 601 is used to collect internal data and external data. Internal data includes: various business data collected by various business databases and systems, such as sales data, customer data, supply chain data, etc., and real-time data collected by sensors, monitoring equipment or Internet of Things devices within the organization; external data includes: third-party data such as social media data, survey data, news and other data; it should be noted that internal data and external data are both relative to the data itself.

[0090] In addition, the data collected by the data collection module 601 may be other data in addition to the above-mentioned internal data and external data. In the embodiment of the present application, the source and type of the data are not limited.

[0091] The data cleaning module 602 is used to remove duplicate data and erroneous data to ensure the integrity and accuracy of the data; interpolate or eliminate missing data to ensure the subsequent analysis and integration of standardized data formats.

[0092] The data storage module 603 is used to store a large amount of (business) data through appropriate storage technologies (such as relational databases, non-relational structured query language (Not Only Structured Query Language, NoSQL) databases and distributed file systems, etc.).

[0093] Statistical analysis module 604 is used for descriptive analysis and predictive analysis; among them, descriptive analysis is to use a few numerical values ​​(such as mean, median, etc.) to describe the information expressed by a series of complex data (data characteristics, distribution and trend), for example, to describe the overall distribution, fluctuation, and data anomalies of the data, and use statistical methods to explore the data; predictive analysis uses data mining and machine learning algorithms to use regression and time series analysis methods to predict trends and demands based on patterns and correlations in the data.

[0094] System application module 605 is used for business decision-making and personalized support. It helps users evaluate the risks and potential benefits of various decisions based on the results of data prediction and simulation analysis, and implements precise management strategies.

[0095] Exemplarily, the functions of the above-mentioned system application module 605 include but are not limited to: rule configuration, indicator management, indicator warning, report generation and risk decision-making.

[0096] Obviously, based on the above-mentioned multi-dimensional data statistics system, the target object can perform data statistics and analysis according to multiple dimensions selected as needed, providing a more comprehensive perspective and realizing all-round and multi-angle statistics and display; and, using advanced algorithms and data processing technology, the system is able to process and calculate large-scale data in a shorter time; in addition, through intuitive forms such as visual reports and charts, the target object can intuitively understand and analyze the statistical results of the data, facilitating accurate and timely decision-making and planning.

[0097] Therefore, compared with traditional single-dimensional statistical methods, multi-dimensional statistics can analyze data more comprehensively and provide more accurate statistical results. In addition, a series of optimization strategies are adopted to improve the efficiency of data statistics and analysis and can process large-scale data. Furthermore, it supports flexible data modeling and query, and the target objects can perform customized data analysis according to their own needs. Moreover, the above system has a clear structure and can be easily expanded and upgraded to meet changing business needs.

[0098] In summary, in the data processing method provided in the embodiment of the present application, a data processing request initiated by a target object for a target data source is responded to, and identification information of the target data source and data processing requirements of the target object are obtained from the data processing request; then, a target data source matching the identification information is selected from a preset data source set, and a target data processing strategy associated with the data processing requirement is screened out from a preset data processing strategy set; finally, based on the target data processing strategy, data processing is performed on the target data source to obtain corresponding data processing results; in this way, data processing can be performed on the target data source according to the data processing request of the target object for the target data source, thereby meeting the user's personalized multi-dimensional statistical summary requirements and improving the flexibility of data processing.

[0099] Further, based on the same technical concept, the embodiment of the present application provides a data processing device, which is used to implement the above method flow of the embodiment of the present application. Figure 7 As shown, the data processing device includes: an acquisition module 701, a screening module 702 and a processing module 703, wherein:

[0100] The acquisition module 701 is used to respond to the data processing request initiated by the target object for the target data source, and obtain the identification information of the target data source and the data processing requirement of the target object from the data processing request; wherein the data processing requirement represents the data processing method of the target object for the target data source;

[0101] The screening module 702 is used to select a target data source matching the identification information from a preset data source set, and screen a target data processing strategy associated with the data processing requirement from a preset data processing strategy set; wherein each data processing strategy includes: a corresponding data statistical summary and analysis method;

[0102] The processing module 703 is used to process the target data source based on the target data processing strategy to obtain the corresponding data processing result.

[0103] In a possible embodiment, before responding to the data processing request initiated by the target object for the target data source, the processing module 703 is further configured to:

[0104] For each original data source, perform the following operations:

[0105] Obtaining each piece of raw data contained in an original data source, and cleaning each piece of raw data to obtain multiple pieces of raw data; wherein an original data source is a business data set;

[0106] Generate candidate data sources based on multiple raw data, and save the candidate data sources into a data source collection.

[0107] In a possible embodiment, when filtering out a target data processing strategy associated with a data processing requirement from a preset data processing strategy set, the filtering module 702 is specifically configured to:

[0108] Parse the data processing requirements to obtain at least one data indicator of the target object for the target data source; wherein each data indicator includes: at least one statistical dimension and one metric;

[0109] From the data processing strategy set, a data processing strategy including at least one data indicator is screened out, and the data processing strategy is used as a target data processing strategy.

[0110] In a possible embodiment, when performing data processing on the target data source based on the target data processing strategy to obtain corresponding data processing results, the processing module 703 is specifically used to:

[0111] Based on at least one data indicator included in the target data processing strategy, perform data segmentation processing on the target data source to obtain the target data source after the data segmentation processing;

[0112] Performing statistical aggregation on the target data source after the data segmentation processing to obtain statistical aggregation results corresponding to at least one data indicator;

[0113] Based on the data analysis method included in the target data processing strategy, data analysis is performed on at least one statistical summary result to obtain a data processing result.

[0114] In a possible embodiment, after the target data source is processed based on the target data processing strategy and the corresponding data processing result is obtained, Figure 7 As shown, the device further includes a display module 704, and the display module 704 is specifically used for:

[0115] In response to a display operation of a target object on a data processing result, a target data display template matching the display operation is selected from a preset data display template set;

[0116] Add each data analysis result included in the data processing result to the target data display template to generate a data display interface for the data processing result.

[0117] Based on the same technical concept, the embodiment of the present application also provides an electronic device, which can implement the data processing method provided in the above embodiment of the present application. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device.Figure 8 As shown, the electronic device may include:

[0118] At least one processor 801, and a memory 802 connected to the at least one processor 801. The specific connection medium between the processor 801 and the memory 802 is not limited in the embodiment of the present application. Figure 8 In the example, the processor 801 and the memory 802 are connected via a bus 800. Figure 8 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 800 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 801 can also be called a controller, and there is no limitation on the name.

[0119] In the embodiment of the present application, the memory 802 stores instructions that can be executed by at least one processor 801. The at least one processor 801 can execute a data processing method discussed above by executing the instructions stored in the memory 802. The processor 801 can implement Figure 7 The functions of each module in the device shown.

[0120] Among them, the processor 801 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 802 and calling the data stored in the memory 802, the various functions of the device and processing data, the device can be monitored as a whole.

[0121] In one possible design, the processor 801 may include one or more processing units, and the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 801. In some embodiments, the processor 801 and the memory 802 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0122] Processor 801 can be a general-purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a data processing method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or can be executed by a combination of hardware and software modules in the processor.

[0123] The memory 802 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 802 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 802 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 802 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0124] By programming the processor 801, the code corresponding to the data processing method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 3 The steps of a data processing method in the embodiment shown are as follows: How to design and program the processor 801 is a technique known to those skilled in the art and will not be described in detail here.

[0125] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes a data processing method discussed above.

[0126] In some possible implementations, the present application also provides various aspects of a data processing method that can also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a data processing method according to various exemplary implementations of the present application described above in this specification.

[0127] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, It is characterized in that include: Responding to a data processing request initiated by a target object for a target data source, and obtaining identification information of the target data source and a data processing requirement of the target object from the data processing request; wherein the data processing requirement represents a data processing method of the target object for the target data source; Selecting a target data source that matches the identification information from a preset data source set, and selecting a target data processing strategy associated with the data processing requirement from a preset data processing strategy set; wherein each data processing strategy includes: corresponding data statistics aggregation and analysis methods; Based on the target data processing strategy, data processing is performed on the target data source to obtain corresponding data processing results.

2. The method according to claim 1, It is characterized in that Before responding to the data processing request initiated by the target object for the target data source, the method further includes: For each original data source, perform the following operations: Acquire each original data contained in an original data source, and clean the each original data to obtain multiple original data; wherein the original data source is a business data set; A candidate data source is generated based on the multiple original data, and the candidate data source is saved in the data source set.

3. The method according to claim 1, It is characterized in that The step of selecting a target data processing strategy associated with the data processing requirement from a preset data processing strategy set includes: Parsing the data processing requirement to obtain at least one data indicator of the target object for the target data source; wherein each data indicator includes: at least one statistical dimension and one metric; From the data processing strategy set, a data processing strategy including the at least one data indicator is screened out, and the data processing strategy is used as the target data processing strategy.

4. The method according to claim 1, 2 or 3, It is characterized in that The step of performing data processing on the target data source based on the target data processing strategy to obtain corresponding data processing results includes: Based on at least one data indicator included in the target data processing strategy, perform data segmentation processing on the target data source to obtain the target data source after the data segmentation processing; Performing statistical aggregation on the target data source after the data segmentation processing to obtain statistical aggregation results corresponding to each of the at least one data indicator; Based on the data analysis method included in the target data processing strategy, data analysis is performed on at least one statistical summary result to obtain the data processing result.

5. The method according to claim 1, 2 or 3, It is characterized in that After performing data processing on the target data source based on the target data processing strategy to obtain corresponding data processing results, the method further includes: In response to a display operation of the target object on the data processing result, a target data display template matching the display operation is screened out from a preset data display template set; Each data analysis result included in the data processing result is added to the target data display template to generate a data display interface for the data processing result.

6. A data processing device, It is characterized in that include: An acquisition module, used to respond to a data processing request initiated by a target object for a target data source, and acquire identification information of the target data source and a data processing requirement of the target object from the data processing request; wherein the data processing requirement represents a data processing method of the target object for the target data source; A screening module is used to select a target data source matching the identification information from a preset data source set, and screen a target data processing strategy associated with the data processing requirement from a preset data processing strategy set; wherein each data processing strategy includes: a corresponding data statistical summary and analysis method; The processing module is used to process the target data source based on the target data processing strategy to obtain corresponding data processing results.

7. The device according to claim 6, It is characterized in that Before responding to the data processing request initiated by the target object for the target data source, the processing module is further used to: For each original data source, perform the following operations: Acquire each original data contained in an original data source, and clean the each original data to obtain multiple original data; wherein the original data source is a business data set; A candidate data source is generated based on the multiple original data, and the candidate data source is saved in the data source set.

8. The device according to claim 6, It is characterized in that When the target data processing strategy associated with the data processing requirement is screened out from the preset data processing strategy set, the screening module is specifically used to: Parsing the data processing requirement to obtain at least one data indicator of the target object for the target data source; wherein each data indicator includes: at least one statistical dimension and one metric; From the data processing strategy set, a data processing strategy including the at least one data indicator is screened out, and the data processing strategy is used as the target data processing strategy.

9. The device according to claim 6, 7 or 8, It is characterized in that When the target data source is processed based on the target data processing strategy to obtain a corresponding data processing result, the processing module is specifically used to: Based on at least one data indicator included in the target data processing strategy, perform data segmentation processing on the target data source to obtain the target data source after the data segmentation processing; Performing statistical aggregation on the target data source after the data segmentation processing to obtain statistical aggregation results corresponding to each of the at least one data indicator; Based on the data analysis method included in the target data processing strategy, data analysis is performed on at least one statistical summary result to obtain the data processing result.

10. The device according to claim 6, 7 or 8, It is characterized in that After the target data source is processed based on the target data processing strategy to obtain the corresponding data processing result, the device further includes a display module, which is specifically used to: In response to a display operation of the target object on the data processing result, a target data display template matching the display operation is screened out from a preset data display template set; Each data analysis result included in the data processing result is added to the target data display template to generate a data display interface for the data processing result.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images