Business report generation method and device, equipment and storage medium

Abstract

The invention provides a business report generation method and device, equipment and a storage medium, and the method comprises the steps: obtaining target business data from a preset business database through employing a pre-constructed business jar package according to a data analysis request when the data analysis request is received; classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to a hierarchical structure of the complex field data to obtain a split field data set; filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data; and matching from a data warehouse through the standard field data to obtain service matching data, and generating a service report according to the service matching data.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and to a method, device, equipment and storage medium for generating a business report. Background Art

[0002] With the development of society and the advancement of technology, in the process of the development of the Internet, enterprises need to continuously iterate their business to optimize user experience and transaction process.

[0003] In the prior art, in the process of iterating the business of an enterprise, business R&D personnel and analysts generally participate together. Business R&D personnel generate business data sources based on specific business needs to support the launch of business functions after business iteration. Analysts obtain business data sources, parse the meaning of business data sources, and generate business reports.

[0004] However, when analysts parse the meaning of business data sources, the parsing time is too long and errors are prone to occur. Especially when parsing complex fields in business data sources, the parsing time and error rate will be higher, resulting in low report generation efficiency. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for generating a business report, the main purpose of which is to solve the technical problem that the analysis efficiency of the traditional business data analysis method is relatively low, resulting in low report generation efficiency.

[0006] To achieve the above object, the present invention provides a method, device, equipment and medium for generating a business report, the method comprising:

[0007] When a data analysis request is received, the target business data is obtained from a preset business database using a pre-built business jar package according to the data analysis request;

[0008] Classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set;

[0009] Filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data;

[0010] The standard field data is matched from the data warehouse to obtain business matching data, and a business report is generated according to the business matching data.

[0011] Optionally, when receiving the data parsing request, before obtaining the target business data from a preset business database using a pre-built business jar package according to the data parsing request, the method further comprises:

[0012] Obtain the source code of the pre-written business logic code and table function and the corresponding configuration files;

[0013] Compiling the business logic code and the source code into executable preset code;

[0014] The preset executable code and configuration files are packaged using a preset construction tool to generate the pre-built business jar package.

[0015] Optionally, when receiving the data parsing request, obtaining target business data from a preset business database using a pre-built business jar package according to the data parsing request includes:

[0016] Constructing a target query statement according to the data parsing request;

[0017] The pre-built business jar package is called through the target query statement, and the target business data is obtained from the preset business database using the pre-built business jar package.

[0018] Optionally, the classifying the target business data into simple field data and complex field data according to a preset field length threshold includes:

[0019] Obtaining field metadata for each field in the target business data;

[0020] Extracting a field length feature of the field metadata;

[0021] According to the field length characteristics of each field metadata and the preset field length threshold, determining whether each field metadata belongs to complex metadata;

[0022] According to the proportion of the complex metadata in the target business data, the target business data is classified into simple field data or complex field data.

[0023] Optionally, splitting the complex field data according to the hierarchical structure of the complex field data to obtain split field data sets includes:

[0024] Parsing the data structure of the complex field data, and determining the hierarchical relationship according to the data structure;

[0025] The complex field data is split according to a predefined splitting strategy and the hierarchical relationship to obtain a split field data set.

[0026] Optionally, filtering the split field data set and the simple field data to obtain filtered field data includes:

[0027] Select from each split field data in the split field data set and the simple field data according to the data parsing request to obtain initial filtering field data;

[0028] The initial filtering field data is subjected to null value deletion processing to obtain the filtering field data.

[0029] Optionally, before matching the standard field data from the data warehouse to obtain business matching data and generating a business report according to the business matching data, the method further includes:

[0030] All data in the preset business database are parsed, and the parsed data are regularly synchronized to the data warehouse.

[0031] In order to solve the above problem, the present invention provides a device for generating a business report, the device comprising:

[0032] An acquisition module is used to acquire target business data from a preset business database using a pre-built business jar package according to the data analysis request when a data analysis request is received;

[0033] A splitting module, used for classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set;

[0034] A conversion module, used for filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data;

[0035] The generating module is used to match the standard field data from the data warehouse to obtain business matching data, and generate a business report according to the business matching data.

[0036] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0037] at least one processor; and,

[0038] a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a business report as described in any one of claims 1 to 7.

[0040] In order to solve the above problem, the present invention also provides a computer-readable storage medium, and when the computer program is executed by a processor, the method for generating a business report is implemented.

[0041] In this embodiment, data is obtained through a pre-built business jar package and subsequently processed, thereby avoiding manual parsing of data fields created by R&D personnel based on the business, improving the output efficiency of data reports in subsequent iterations, and through the participation of custom table functions, the business jar package can perform more complex data processing operations, thereby improving data processing efficiency; the complex fields of the target business data are identified through a field classification model, and the complex field data is automatically flattened through the business jar package to split out at least one split field data, and analysis is performed based on the split field data to avoid in-depth understanding of the field structure of the complex field data, thereby improving the efficiency and accuracy of subsequent data conversion; each split field data and the simple field data in the split field data set are filtered and processed through the business jar package to obtain filtered field data, thereby eliminating redundant data that is meaningless to subsequent report generation, reducing the cost of data processing and improving the efficiency of data processing, and performing data conversion on the filtered field data to obtain standard field data, thereby improving the efficiency of subsequent report generation.

[0042] In summary, the embodiments of the present invention can improve data analysis efficiency, thereby improving business report generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a method for generating a business report according to an embodiment of the present application;

[0044] Figure 2 It is a module schematic diagram of a device for generating a business report in an embodiment of the present application;

[0045] Figure 3 It is a structural diagram of an electronic device used to execute the method for generating a business report according to an embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] The embodiment of the present invention provides a method for generating a business report, and the execution subject of the method for generating a business report includes but is not limited to at least one of the electronic devices such as a client and a mobile terminal that can be configured to execute the method provided by the embodiment of the present application. The APP can be a blockchain platform. The mobile terminal includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0049] In an embodiment of the present invention, the method for generating a business report first obtains target business data from a preset business database using a pre-built business jar package according to a data parsing request when a data parsing request is received; classifies the target business data into simple field data or complex field data according to a preset field length threshold, and splits the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set; filters the split field data set and the simple field data to obtain filtered field data, performs data conversion on the filtered field data to obtain standard field data; matches the standard field data from a data warehouse to obtain business matching data, and generates a business report based on the business matching data.

[0050] To facilitate understanding of this embodiment, a method for generating a business report disclosed in an embodiment of the present application is first introduced in detail. Figure 1 A flow chart of a method for generating a business report provided in an embodiment of the present application is shown. Figure 1 As shown, the following steps are included:

[0051] S10. When a data analysis request is received, the target business data is acquired from a preset business database using a pre-built business jar package according to the data analysis request.

[0052] In this embodiment, when a data parsing request initiated by a user is received, a user-defined table generating function (UDTF, User-Defined Table-Generating Function) in a hive table in a data warehouse is used to access a business jar package, so that a data acquisition module in the business jar package acquires target business data from the preset business database. The preset business database contains data to be parsed customized by multiple R&D teams of an enterprise according to business requirements required for business iteration, that is, multiple business data sources. The preset business jar package is a code content written by a business R&D code, including the following functional modules: a data acquisition module, a data filtering module, a data splitting module, and a data conversion module. The data warehouse is a system for centralized storage and management of various types of enterprise data, which can be used as a key infrastructure to support the decision-making process and can integrate, store and analyze data from multiple sources. In Hive, a user-defined table generating function (UDTF, User-Defined Table-Generating Function) is used to generate one or more output tables. UDTF allows users to use customized functions in Hive tables to process data and return the results as tables.

[0053] In an implementable example, based on market demand, an e-commerce company launches an activity to give back to old users (users who have registered and placed orders) in order to maintain old users. The activity is to issue corresponding coupons to the corresponding old users. Since the final report is generated by a combination of multiple different data sources, here, we use the data to be parsed customized by one of the R&D teams as an example. One of the teams of the e-commerce company is responsible for the marketing strategy, specifically, accurately locating which users need to be issued coupons. The corresponding R&D team creates a field about the marketing strategy (a business data source in the preset business database) according to the marketing strategy, stores the field of the marketing strategy in the preset business database, receives a request for data parsing, and accesses the business jar package through the user-defined table function in the hive in the data warehouse, so that the data acquisition module of the business jar package obtains the field created by the R&D team for the marketing strategy from the preset business database (a business data source in the preset business database) to generate the target business data.

[0054] In one embodiment, when a data parsing request is received, before obtaining target business data from a preset business database using a pre-built business jar package according to the data parsing request, the method further includes:

[0055] Obtain the source code of the pre-written business logic code and table function and the corresponding configuration files;

[0056] Compiling the business logic code and the source code into executable preset code;

[0057] The preset executable code and configuration files are packaged using a preset construction tool to generate the pre-built business jar package.

[0058] In this embodiment, the functions of the business jar package are defined, the data acquisition module, data filtering module, data splitting module and data conversion module included in the business jar package are determined, the corresponding business logic code is written according to the functions of the business jar package, and the source code of the custom table function (UDTF) is written, the business logic code and the source code are compiled into executable Java bytecode or executable code in other languages, and the required configuration files are obtained, the configuration files include Hive dependency libraries, configuration files required for table functions, etc., and the preset build tools (such as Maven or Gradle build tools) are used to package the compiled code and configuration files to generate a pre-built business jar package.

[0059] Furthermore, after generating the pre-built business jar package, the business jar package needs to be tested. Specifically, the business jar package is deployed to the target environment, which includes a Hive cluster, etc., and a preset amount of training data is obtained as a test set. The business jar package is tested using the test set to ensure the correctness and stability of the business logic and table functions in the target environment.

[0060] In one embodiment, when a data parsing request is received, obtaining target business data from a preset business database using a pre-built business jar package according to the data parsing request includes:

[0061] Constructing a target query statement according to the data parsing request;

[0062] The pre-built business jar package is called through the target query statement, and the target business data is obtained from the preset business database using the pre-built business jar package.

[0063] In this embodiment, the target query statement refers to HiveQL (Hive query language), and may also refer to other user-defined functions (UDF, UDAF, etc.) that can mobilize the query language of the pre-built business jar package, determine the required target business field according to the data analysis request, construct a target query statement according to the target business field, and call the business jar package through the target query statement, so that the data acquisition module of the business jar package obtains the target business data from the preset business database according to the target business field.

[0064] In an implementable example, one of the teams of an e-commerce company is responsible for the marketing strategy. The corresponding R&D team creates fields about the marketing strategy based on the marketing strategy (a business data source in the preset business database), stores the fields of the marketing strategy in the preset business database, receives a request for data analysis, determines the target business fields (such as the user's historical behavior field) contained in the data analysis request, builds HiveQL (Hive query language) based on the user's historical behavior field, calls the data acquisition module of the business jar package through the Hive query language, and uses the data acquisition module to obtain the field set created by the R&D team for the marketing strategy from the preset business library in the data warehouse (a business data source in the preset business database) to generate target business data.

[0065] In one embodiment, after obtaining the target business data from a preset business database using a pre-built business jar package according to the data parsing request, the method further includes:

[0066] The target business data is subjected to abnormal data deletion processing, missing value filling processing and normalization processing.

[0067] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target business data, the target business data can also be stored in a node of a blockchain.

[0068] In this embodiment, the abnormal data refers to unreasonable values ​​in the target business data. For example, if the highest unit price of a single product on the platform is 1,000 yuan, and the user's historical behavior data shows that the platform has placed an order for a single product worth 5,000 yuan, then the data is an unreasonable value. In this embodiment, the method for judging the abnormal data can use one or more abnormal data judgment methods such as the 3δ principle judgment method, the clustering judgment method, and the statistical judgment method. It can also use one abnormal data judgment method for judgment, and use another abnormal data judgment method to supplement the judgment method, to judge the abnormal values ​​in the target business data, and delete the abnormal data in the target business data.

[0069] Specifically, a data interval is set, such as an interval of the amount of the ordered items, and the acquired data is compared and matched with the interval to determine that the data outside the interval is abnormal data. For example, the interval is set to [0, 1000], and the user's historical behavior data is obtained, which includes orders for items of -100, 50, 100, and 2000, respectively. These four data are matched with the interval, and the items of -100 and 2000 are not within the interval and are defined as abnormal data, and the abnormal data is deleted.

[0070] Furthermore, the embodiment of the present invention can use the missing value filling method of filling fixed value, filling median, filling mean, and filling mode to fill the target business data. For example, the missing value of the target business data (such as the user's historical behavior data) is filled with a fixed value of 10. The fixed value can be filled in the range mentioned above. One way is to use the fillna function of the pandas module of Python to achieve filling. Assuming that the target business data is A, the code is as follows:

[0071] A.fillna(0,inplace=True).

[0072] Furthermore, in order to improve the processing data of the target business data, the target business data after abnormal data deletion and missing value filling is normalized. The normalization process is to scale the data in proportion so that the target business data falls into a small specific interval, wherein the normalization method uses the normalization method of the Sigmoid function, etc., to obtain the pre-processed target business data.

[0073] In this embodiment, data is obtained through a pre-built business jar package and subsequently processed, thereby avoiding manual parsing of data fields created by R&D personnel based on the business, improving the output efficiency of data reports in subsequent iterations, and through the participation of custom table functions, the business jar package can perform more complex data processing operations, thereby improving data processing efficiency.

[0074] S20. Classify the target business data into simple field data and complex field data according to a preset field length threshold, and split the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set.

[0075] Since the target business data contains multiple complex data fields and multiple simple data fields, directly parsing the complex fields will result in slow efficiency and inaccurate parsing.

[0076] In this embodiment, complex field data in the target business data is identified through a trained field classification model. The preset data identification model can be a neural network. The target business data is classified into simple digital field data and complex field data, and the complex field data is split through the data splitting module of the business jar package. The complex field data is split into multiple split field data, and subsequent data conversion operations are performed based on the multiple split field data and simple field data after splitting.

[0077] In one embodiment, the classifying the target business data into simple field data and complex field data according to a preset field length threshold includes:

[0078] Obtaining field metadata for each field in the target business data;

[0079] Extracting a field length feature of the field metadata;

[0080] According to the field length characteristics of each field metadata and the preset field length threshold, determining whether each field metadata belongs to complex metadata;

[0081] According to the proportion of the complex metadata in the target business data, the target business data is classified into simple field data or complex field data.

[0082] In this embodiment, field metadata corresponding to each field in the target business data is extracted from the data dictionary, the field metadata including information such as the name, data type, length, and whether it is empty, etc. of the field, field length characteristics corresponding to each field metadata are extracted from the field metadata, field metadata greater than or equal to a preset field length threshold is regarded as complex metadata, field metadata less than a preset field length threshold is regarded as simple metadata, the proportion of field metadata to the target business data is calculated, when the proportion of complex metadata in the target business data is greater than the proportion threshold, the target business data is classified as complex field data, when the proportion of complex metadata in the target business data is less than or equal to the proportion threshold, the target business data is classified as simple field data;

[0083] In one embodiment, the classifying the target business data into simple field data or complex field data according to the proportion of the complex metadata in the target business data further includes:

[0084] According to the ratio of simple metadata to complex metadata in the target business data, the target business data is classified into simple field data or complex field data.

[0085] Furthermore, the target business data may be classified into simple field data and complex field data according to other contents included in the field metadata.

[0086] For example, the structural types of the target business data are identified: structured data and unstructured data, and the target business data is classified into simple field data and complex field data using the data type.

[0087] Since complex fields are the key points of data conversion, if the complex fields are converted directly, the efficiency and accuracy of the subsequent data conversion steps will be low. Therefore, in this embodiment, the complex field data in the target business data is split into at least one split field data through the data splitting module of the business jar package, and conversion is performed based on the split field data after splitting to improve the efficiency and accuracy of the subsequent conversion steps.

[0088] In one embodiment, splitting the complex field data according to the hierarchical structure of the complex field data to obtain split field data sets includes:

[0089] Parsing the data structure of the complex field data, and determining the hierarchical relationship according to the data structure;

[0090] The complex field data is split according to a predefined splitting strategy and the hierarchical relationship to obtain a split field data set.

[0091] In this embodiment, the split field data set includes at least one split field; first parse the data structure of the complex field (such as JSON, XML, etc.), determine the hierarchical relationship or separator according to the data structure, and use the hierarchical relationship or separator and the predefined split strategy to split the complex field data to obtain split data. For example, the complex field data is a json string, which exists in a multi-level nested form:

[0092] "attribute1": "123"

[0093] "attribute2": "456"

[0094] …

[0095] According to the hierarchical relationship of each level, the JSON string in a multi-level nested form is split into "attribute 1": "123", "attribute 2": "456"... to obtain a split data set.

[0096] For example, complex field data is in XML format:

[0097]

[0098] Extract user / id, user / name, user / address / street, and user / address / city into independent field data.

[0099] In this embodiment, complex fields of the target business data are identified through a field classification model, and the complex field data is automatically flattened through a business jar package to split at least one split field data. Analysis is performed based on each split field in the split field data set to avoid in-depth understanding of the field structure of the complex field data and improve the efficiency and accuracy of subsequent data conversion.

[0100] S30, filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data.

[0101] After splitting complex fields through the business jar package, some redundant data that is meaningless to subsequent report generation will be obtained. Therefore, it is necessary to filter out these redundant data to reduce the data processing volume, thereby reducing the processing cost and improving the efficiency of data processing.

[0102] In this embodiment, each split field data and simple field data in the split field data set is filtered through the data filtering module of the business jar package to obtain filtered field data, and each split field data and each simple field data based on the filtered field data are converted respectively, and the field naming of each field is unified to obtain multiple conversion data, and the multiple conversion data are combined to generate standard field data.

[0103] In one embodiment, filtering the split field data set and the simple field data to obtain filtered field data includes:

[0104] Select from each split field data in the split field data set and the simple field data according to the data parsing request to obtain initial filtering field data;

[0105] The initial filtering field data is subjected to null value deletion processing to obtain the filtering field data.

[0106] In this embodiment, the data analysis request also includes defining the data required for subsequent report generation. After the data analysis request defines the data requirements, all split field data and simple field data are matched according to the data requirements, and the initial filtering field data that meets the preset data requirements is selected, and invalid data or data that is meaningless to subsequent report generation is filtered out. In the computer field, some data will cause null values ​​after processing. In order to improve the accuracy and efficiency of subsequent report generation, it is necessary to delete null values ​​from the field data in the initial filtering data to obtain the filtering field data.

[0107] In an implementable example, an e-commerce company's marketing strategy is to launch an activity to give back to old users, that is, to issue corresponding coupons to the corresponding old users. In other words, the preset data requirements do not require the time data of the user's order. As long as the order is placed, the user is an old user, and the time data of the order is irrelevant data, and the time data needs to be filtered out.

[0108] In this embodiment, each split field data and the simple field data in the split field data set are filtered through the business jar package to obtain filtered field data, thereby eliminating meaningless and redundant data for subsequent report generation, reducing the cost of data processing and improving data processing efficiency, and performing data conversion on the filtered field data to obtain standard field data, thereby improving the efficiency of subsequent report generation.

[0109] S40, matching the standard field data from the data warehouse to obtain business matching data, and generating a business report according to the business matching data.

[0110] The above steps of the embodiment of the present invention parse one of the data for generating a report, and the report is generated by combining multiple data. Therefore, in this embodiment, before matching the standard field data from the data warehouse to generate a business report, the method further includes:

[0111] All data in the preset database are parsed, and the parsed data are regularly synchronized to the data warehouse.

[0112] Specifically, the preset database contains data to be analyzed designed by multiple R&D teams. All the data to be analyzed are analyzed, and the analyzed data to be analyzed are synchronized in the data warehouse regularly. In another embodiment, the analyzed data to be analyzed can also be synchronized in the data warehouse in real time.

[0113] In this embodiment, the generated standard field data is stored in the data warehouse, and according to the report generation requirements and standard data, association matching is performed in multiple data in the data warehouse to obtain business matching data, and the final business report is generated based on the business matching data.

[0114] In an implementable example, the field data of the marketing strategy of this application is parsed to determine the users to whom the coupons are issued. By matching the standard data of the users to whom the coupons are issued in the data warehouse, the corresponding amount of users eligible for coupon issuance is obtained, and the final business report is generated.

[0115] In one embodiment, matching the standard data from the data warehouse to generate a business report includes:

[0116] Matching the data from the data warehouse according to the preset data association relationship and the standard data to obtain associated field data;

[0117] A preset report generation requirement is obtained, and according to the preset report generation requirement, the standard field data and the associated field data are combined using a preset report generation tool to generate the business report.

[0118] In this embodiment, the standard data automatically parsed by the business jar package is matched in the data warehouse to generate the final business report, which improves the data parsing efficiency and thus improves the report generation efficiency. In addition, since the previous data and the report are highly coupled, the report and the original data (data to be parsed) are analyzed through the embodiment of the present application, so that when the original data is modified, the display of the report is not affected, thereby reducing the coupling between the data and the report. In this embodiment, data is obtained through a pre-built business jar package and subsequently processed, thereby avoiding manual parsing of data fields created by R&D personnel based on the business, improving the output efficiency of data reports in subsequent iterations, and through the participation of custom table functions, the business jar package can perform more complex data processing. operation, thereby improving data processing efficiency; identifying complex fields of target business data through a field classification model, and automatically flattening the complex field data through a business jar package, splitting out at least one split field data, and performing analysis based on each split field data in the split field data set, thereby avoiding in-depth understanding of the field structure of the complex field data, and improving the efficiency and accuracy of subsequent data conversion; filtering each split field data and the simple field data in the split field data set through the business jar package to obtain filtered field data, eliminating meaningless and cumbersome data for subsequent report generation, reducing the cost of data processing and improving the efficiency of data processing, and performing data conversion on the filtered field data to obtain standard field data, thereby improving the efficiency of subsequent report generation.

[0119] In summary, through the embodiments of the present invention, the data parsing efficiency can be improved, thereby improving the efficiency of generating business reports and reducing the coupling between original data and business reports.

[0120] Reference Figure 2 FIG. 1 is a schematic diagram of functional modules of a device 100 for generating a business report according to the present invention.

[0121] The acquisition module 110 is used to acquire target business data from a preset business database using a pre-built business jar package according to the data analysis request when a data analysis request is received;

[0122] In one embodiment, when receiving the data parsing request, before obtaining the target business data from the preset business database using the pre-built business jar package according to the data parsing request, the method further includes:

[0123] Obtain the source code of the pre-written business logic code and table function and the corresponding configuration files;

[0124] Compiling the business logic code and the source code into executable preset code;

[0125] The preset executable code and configuration files are packaged using a preset construction tool to generate the pre-built business jar package.

[0126] In one embodiment, when a data parsing request is received, obtaining target business data from a preset business database using a pre-built business jar package according to the data parsing request includes:

[0127] Constructing a target query statement according to the data parsing request;

[0128] The pre-built business jar package is called through the target query statement, and the target business data is obtained from the preset business database using the pre-built business jar package.

[0129] A splitting module 120, configured to classify the target business data into simple field data or complex field data according to a preset field length threshold, and split the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set;

[0130] In one embodiment, the classifying the target business data into simple field data or complex field data according to a preset field length threshold includes:

[0131] Obtaining field metadata for each field in the target business data;

[0132] Extracting a field length feature of the field metadata;

[0133] According to the field length characteristics of each field metadata and the preset field length threshold, determining whether each field metadata belongs to complex metadata;

[0134] According to the proportion of the complex metadata in the target business data, the target business data is classified into simple field data or complex field data.

[0135] In one embodiment, splitting the complex field data according to the hierarchical structure of the complex field data to obtain split field data sets includes:

[0136] Parsing the data structure of the complex field data, and determining the hierarchical relationship according to the data structure;

[0137] The complex field data is split according to a predefined splitting strategy and the hierarchical relationship to obtain a split field data set.

[0138] The conversion module 130 is used to filter the split field data set and the simple field data to obtain filtered field data, and perform data conversion on the filtered field data to obtain standard field data;

[0139] In one embodiment, filtering the split field data set and the simple field data to obtain filtered field data includes:

[0140] Select from each split field data in the split field data set and the simple field data according to the data parsing request to obtain initial filtering field data;

[0141] The initial filtering field data is subjected to null value deletion processing to obtain the filtering field data.

[0142] The generating module 140 is used to perform matching from the data warehouse through the standard field data to obtain business matching data, and generate a business report according to the business matching data.

[0143] In one embodiment, before matching the standard field data from the data warehouse to obtain business matching data and generating a business report based on the business matching data, the method further includes:

[0144] All data in the preset business database are parsed, and the parsed data are regularly synchronized to the data warehouse.

[0145] Reference Figure 3 FIG. 1 is a schematic diagram of a preferred embodiment of an electronic device 1 of the present invention.

[0146] The electronic device 1 includes but is not limited to: a memory 11, a processor 12, a display 13 and a network interface 14. The electronic device 1 is connected to a network through the network interface 14 to obtain raw data. The network may be a wireless or wired network such as an intranet, the Internet, a global system of mobile communication (GSM), a wideband code division multiple access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, a call network, etc.

[0147] Among them, the memory 11 includes at least one type of readable medium, and the readable medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a hard disk or memory of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped with the electronic device 1. Of course, the memory 11 can also include both the internal storage unit of the electronic device 1 and its external storage device. In this embodiment, the memory 11 is generally used to store the operating system and various application software installed on the electronic device 1, such as the program code of the business report generation program 10, etc. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0148] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the program code of the business report generation program 10.

[0149] The display 13 may be referred to as a display screen or a display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an organic light-emitting diode (OLED) touch device. The display 13 is used to display information processed in the electronic device 1 and to display a visual working interface, such as displaying the results of data statistics.

[0150] The network interface 14 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the network interface 14 is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0151] Figure 3Only the electronic device 1 having the components 11 - 14 and the business report generation program 10 is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0152] Optionally, the electronic device 1 may further include a user interface, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an organic light-emitting diode (Organic Light-Emitting Diode, OLED) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0153] The electronic device 1 may further include a radio frequency (RF) circuit, a sensor, an audio circuit, etc., which will not be described in detail here.

[0154] In the above embodiment, when the processor 12 executes the business report generation program 10 stored in the memory 11, the following steps can be implemented:

[0155] When a data analysis request is received, the target business data is obtained from a preset business database using a pre-built business jar package according to the data analysis request;

[0156] Classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set;

[0157] Filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data;

[0158] The standard field data is matched from the data warehouse to obtain business matching data, and a business report is generated according to the business matching data.

[0159] For a detailed description of the above steps, please refer to Figure 1 Description of a flowchart of an embodiment of a method for generating a business report.

[0160] In addition, an embodiment of the present invention further proposes a computer-readable medium, which may be non-volatile or volatile. The computer-readable medium may be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable medium includes a data storage area and a program storage area, the data storage area stores data created according to the use of the blockchain node, and the program storage area stores a business report generation program 10, and the business report generation program 10 is executed by the processor to implement the following operations:

[0161] When a data analysis request is received, the target business data is obtained from a preset business database using a pre-built business jar package according to the data analysis request;

[0162] Classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set;

[0163] Filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data;

[0164] The standard field data is matched from the data warehouse to obtain business matching data, and a business report is generated according to the business matching data.

[0165] The specific implementation of the computer-readable medium of the present invention is substantially the same as the specific implementation of the above-mentioned method for generating a business report, and will not be described in detail herein.

[0166] In another embodiment, the method for generating a business report provided by the present invention can further ensure the privacy and security of all the above data, and all the above data can also be stored in a blockchain node. For example, feature dimensions and feature embeddings can all be stored in blockchain nodes.

[0167] It should be noted that the blockchain referred to in the present invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0168] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, an electronic device, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0170] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection of the present invention.

Claims

1. A method for generating a business report, characterized in that: The method comprises: When a data analysis request is received, the target business data is obtained from a preset business database using a pre-built business jar package according to the data analysis request; Classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set; Filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data; The standard field data is matched from the data warehouse to obtain business matching data, and a business report is generated according to the business matching data.

2. The method for generating a business report according to claim 1, characterized in that: Before obtaining the target business data from a preset business database using a pre-built business jar package according to the data parsing request when the data parsing request is received, the method further comprises: Obtain the source code of the pre-written business logic code and table function and the corresponding configuration files; Compiling the business logic code and the source code into executable preset code; The preset executable code and configuration files are packaged using a preset construction tool to generate the pre-built business jar package.

3. The method for generating a business report according to claim 1, characterized in that: When receiving the data analysis request, obtaining target business data from a preset business database using a pre-built business jar package according to the data analysis request includes: Constructing a target query statement according to the data parsing request; The pre-built business jar package is called through the target query statement, and the target business data is obtained from the preset business database using the pre-built business jar package.

4. The method for generating a business report according to claim 1, characterized in that: The classifying the target business data into simple field data or complex field data according to a preset field length threshold includes: Obtaining field metadata for each field in the target business data; Extracting a field length feature of the field metadata; According to the field length characteristics of each field metadata and the preset field length threshold, determining whether each field metadata belongs to complex metadata; According to the proportion of the complex metadata in the target business data, the target business data is classified into simple field data or complex field data.

5. The method for generating a business report according to claim 1, characterized in that: The step of splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set includes: Parsing the data structure of the complex field data, and determining the hierarchical relationship according to the data structure; The complex field data is split according to a predefined splitting strategy and the hierarchical relationship to obtain a split field data set.

6. The method for generating a business report according to claim 1, characterized in that: The filtering process of the split field data set and the simple field data to obtain the filtered field data includes: Select from each split field data in the split field data set and the simple field data according to the data parsing request to obtain initial filtering field data; The initial filtering field data is subjected to null value deletion processing to obtain the filtering field data.

7. The method for generating a business report according to claim 1, characterized in that: Before matching the standard field data from the data warehouse to obtain business matching data and generating a business report based on the business matching data, the method further includes: All data in the preset business database are parsed, and the parsed data are regularly synchronized to the data warehouse.

8. A device for generating a business report, characterized in that: The device comprises: An acquisition module is used to acquire target business data from a preset business database using a pre-built business jar package according to the data analysis request when a data analysis request is received; A splitting module, used for classifying the target business data into simple field data or complex field data according to a preset field length threshold, and splitting the complex field data according to the hierarchical structure of the complex field data to obtain a split field data set; A conversion module, used for filtering the split field data set and the simple field data to obtain filtered field data, and performing data conversion on the filtered field data to obtain standard field data; The generating module is used to match the standard field data from the data warehouse to obtain business matching data, and generate a business report according to the business matching data.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a business report as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating a business report as described in any one of claims 1 to 7 is implemented.

Images