Data aggregation method, device, electronic device and medium

By assembling and storing business parameter codes with templates to MongoDB and third-party databases, data aggregation instructions are used to aggregate data, the problem of mismatch in the characteristics of the third-party database stage is solved, dynamic data splitting and statistics are realized, and development efficiency is improved and functional customization capabilities.

CN112732763BActive Publication Date: 2024-11-19BEIJING SUREKAM CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110077831.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-20
Publication Date
2024-11-19
Estimated Expiration
2041-01-20

AI Technical Summary

Technical Problem

During the development process of continuous integration and continuous deployment, the project phased tasks do not match the phase characteristics of the third-party database, making it difficult for developers to obtain business data stored in third-party data in a timely and efficient manner, limiting R&D personnel's functions for each phase development time.

Method used

By obtaining business parameter codes and assembling them with matching templates into parsed data, it is stored in MongoDB and third-party databases. When receiving the aggregation instruction, data aggregation is performed, and dynamic data splitting is achieved using technologies such as dual-pointer algorithms and other technologies. and statistics solve the dependence on the characteristics of third-party database stages and realize user-defined functions.

Benefits of technology

It realizes dynamic splitting and statistics of third-party database data, solves development limitations, realizes flexibility and change in product forms, and improves development efficiency and functional customization capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112732763B_ABST
    Figure CN112732763B_ABST
Patent Text Reader

Abstract

The present application discloses a data aggregation method, device, electronic device and medium. In the present application, after obtaining the business parameter code, the business parameter code and the first template can be assembled into the first parsed data with a data structure matching the first template, and the first parsed data is stored in the MongoDB database, and the business parameter code and the second template are assembled into the second template data to generate the second parsed data, and after the second parsed data is stored in the third-party database, when the aggregation instruction is obtained, the second parsed data in the third-party database and the first parsed data in the MongoDB database are aggregated. By applying the technical solution of the present application, the dependence on the stage characteristics of the third-party database can be solved, and the dynamic splitting of the own data and the third-party database data can be completed. Then statistics and aggregation are performed to realize the user-customizable functions in the display stage, solve the limitations of development, and realize the flexibility and diversity of product forms.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a data aggregation method, device, electronic device and medium. Background Art

[0002] When doing continuous integration and continuous deployment development work, you often encounter the problem that the project's phased tasks do not match the phased characteristics of the dependent third-party database. As a result, developers often cannot obtain business data stored in third-party databases in a timely and effective manner. This also greatly limits the development time of R&D personnel for the functions of each phase of continuous integration and continuous deployment. Summary of the invention

[0003] The embodiments of the present application provide a data aggregation method, device, electronic device, and medium. According to one aspect of the embodiments of the present application, a data aggregation method is provided, characterized in that it includes:

[0004] Get the business parameter code;

[0005] Assembling the business parameter code and the first template into first parsed data, and storing the first parsed data in a MongoDB database, wherein the first parsed data and the first template have matching data structures;

[0006] Assembling the business parameter code and the second template into second template data, generating second parsed data, and storing the second parsed data in a third-party database;

[0007] When the aggregation instruction is obtained, the second parsed data in the third-party database is aggregated with the first parsed data in the MongoDB database.

[0008] Optionally, in another embodiment of the method of the present application, assembling the business parameter code and the first template into first parsed data includes:

[0009] Acquire a target data structure and a second template corresponding to the third-party database;

[0010] Extracting a data structure identifier from the business parameter code, and converting the business parameter code into the target data structure;

[0011] The business parameter code converted into the target data structure is used to replace the dynamic parameter item in the first template group to obtain the first parsed data.

[0012] Optionally, in another embodiment of the method of the present application, before assembling the service parameter code and the first template into the first parsed data, the method further includes:

[0013] Generate a file upload template and an image upload template, wherein the image upload template is used to upload the image file to the private image warehouse of the MongoDB database;

[0014] Generate a deployment function template, where the deployment function template is used to decompress the first parsed data;

[0015] Based on the business scenario requirements, the file upload template, the image upload template and the deployment function template, the first template is generated, and the first template is used to parse the business parameter code, generate first parsed data and upload it to the MongoDB database.

[0016] Optionally, in another embodiment of the above method of the present application, after assembling the business parameter code and the first template into the first parsed data, the method further includes:

[0017] Recording a matching rule between the first parsed data and the second parsed data;

[0018] The matching rule and the first parsed data are stored together in a MongoDB database.

[0019] Optionally, in another embodiment of the method of the present application, when the aggregation instruction is obtained, the first parsed data in the MongoDB database and the second parsed data in the third-party database are aggregated, including:

[0020] When the aggregation instruction is obtained, the matching rule and the first parsed data in the MongoDB database are obtained;

[0021] Based on the matching rule, obtaining the second parsed data from the third-party database;

[0022] The first parsed data and the second parsed data are aggregated.

[0023] Optionally, in another embodiment of the method of the present application, assembling the business parameter code and the first template into first parsed data includes:

[0024] The first parsed data and the second parsed data are aggregated using a double pointer algorithm.

[0025] Optionally, in another embodiment of the above method of the present application, before assembling the service parameter code and the second template into the second template data, it also includes:

[0026] Acquire the data structure and parameter requirements of the third-party database, and generate template dynamic parameter items corresponding to the third-party database;

[0027] Obtaining a to-be-matched template identifier corresponding to the business parameter code, and matching the to-be-matched template identifier with a plurality of preset candidate template identifiers one by one to obtain a third template, wherein the third template is a template that matches the to-be-matched template identifier;

[0028] The template dynamic parameter item is loaded into the third template to generate the second template.

[0029] Optionally, in another embodiment of the method of the present application, after aggregating the second parsed data in the third-party database with the first parsed data in the MongoDB database, the method further includes:

[0030] Aggregating the second parsed data with the first parsed data to obtain an aggregated data group;

[0031] The aggregated data groups are assembled into lane data, and the lane data are sent to a receiver specified by the aggregation instruction.

[0032] According to another aspect of the embodiment of the present application, a data aggregation device is provided, characterized in that it includes:

[0033] An acquisition module, configured to acquire a business parameter code;

[0034] A generating module is configured to assemble the business parameter code and the first template into first parsed data, and store the first parsed data in a MongoDB database, wherein the first parsed data and the first template have a matching data structure;

[0035] A storage module, configured to assemble the business parameter code and the second template into second template data, generate second parsed data, and store the second parsed data in a third-party database;

[0036] The aggregation module is configured to aggregate the second parsed data in the third-party database with the first parsed data in the MongoDB database when an aggregation instruction is obtained.

[0037] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0038] A memory for storing executable instructions; and

[0039] A display, used to display with the memory to execute the executable instructions to complete the operation of any of the above-mentioned data aggregation methods.

[0040] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, which, when executed, perform operations of any of the above-mentioned data aggregation methods.

[0041] In the present application, after obtaining the business parameter code, the business parameter code and the first template are assembled into the first parsed data having a data structure matching the first template, and the first parsed data is stored in the MongoDB database, and the business parameter code and the second template are assembled into the second template data to generate the second parsed data, and after the second parsed data is stored in the third-party database, when the aggregation instruction is obtained, the second parsed data in the third-party database and the first parsed data in the MongoDB database are aggregated. By applying the technical solution of the present application, the dependence on the stage characteristics of the third-party database can be solved, and the dynamic splitting of the own data and the third-party database data can be completed. Then statistics and aggregation are performed to realize the user-customizable functions in the display stage, solve the limitations of development, and realize the flexibility and diversity of product forms.

[0042] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0044] The present application can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0045] Figure 1 A schematic diagram of a data aggregation method proposed in this application;

[0046] Figure 2 A schematic diagram of the structure of an electronic device for aggregating data of the present application;

[0047] Figure 3 A schematic diagram of the electronic device structure is shown for this application. DETAILED DESCRIPTION

[0048] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0049] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0050] The following description of at least one exemplary embodiment is merely illustrative in nature and is not intended to limit the present application, its application, or uses.

[0051] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0052] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0053] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0054] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0055] Combine the following Figure 1 To describe the method for aggregating data according to an exemplary embodiment of the present application. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0056] The present application also proposes a data aggregation method, device, target terminal and medium.

[0057] Figure 1 The following schematically shows a flow chart of a data aggregation method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0058] S101, obtaining a service parameter code.

[0059] S102, assembling the business parameter code and the first template into first parsed data, and storing the first parsed data in a MongoDB database, wherein the first parsed data and the first template have matching data structures.

[0060] Furthermore, after obtaining the business parameter code, the present application may first convert its data format to ensure that it can be stored in the corresponding MongoDB database. It should be noted that the present application does not specifically limit the business parameter code, that is, it can be any content and any number of business codes.

[0061] It should be noted that MongoDB is a database based on distributed file storage. It has collection-oriented storage and is easy to store object-type data. In MongoDB, data is grouped and stored in collections, which are similar to tables in RDBMS. An unlimited number of documents can be stored in a collection. It is also schema-free, supports full indexing, supports most queries in SQL, supports complex aggregation tasks, supports replication and data recovery, uses binary data storage, and automatically handles sharding.

[0062] In addition, the present application can also verify the integrity of the business parameter code before converting it into a data structure of a preset format. It is understandable that if it is detected that the part of the code is incomplete, in order to avoid the problem of consuming system resources caused by subsequent unnecessary storage, the present application can discard the incomplete code. Specifically, in the process of checking the integrity of the business parameter code, the business parameters of the target code input by the user can be checked one by one according to the pre-defined verification rules.

[0063] Among them, data structure conversion is the process of merging, cleaning and integrating data, changing it from one form of representation to another through conversion, and achieving semantic consistency among different source data.

[0064] Further, the present application does not specifically limit the method of data structure conversion. For example, the data can be first smoothed to help remove noise in the data. The main technical methods include Bin method, clustering method and regression method. Then, the data can be aggregated to ensure that the data is summarized or aggregated. For example, the daily data can be aggregated to obtain the monthly or annual total. This operation is often used to construct a data cube or perform multi-granular analysis on the data. Further, the data can be generalized. That is, a more abstract (higher level) concept is used to replace the data objects of the low level or data layer.

[0065] Furthermore, the data can be normalized so that the attribute data can be proportionally projected into a specific small range. Furthermore, the data can be attribute constructed, that is, new attributes can be constructed based on the existing attribute set to help the data processing process, etc.

[0066] Optional, common contents of data conversion include: data type conversion, data semantic conversion, data range conversion, data granularity conversion, table / data splitting, row and column conversion, data discretization, data discretization, extracting new fields, attribute construction, data compression, etc. Among them, for data type conversion, when data comes from different data sources, incompatibility of data types of different types of data sources may cause system errors. At this time, it is necessary to uniformly convert the data types of different data sources into a compatible data type. For data semantic conversion, since there may be dimension tables and fact tables based on the third normal form in traditional databases, there will be many fields in the fact table that need to be combined with dimension tables for semantic analysis. For example, if the business meaning of field M is browser type, its values ​​are 1 / 2 / 3 / 4 / 5. If these 5 numbers are not converted, it is difficult to understand them as business language, and they cannot be interpreted and applied later. Furthermore, with regard to data granularity conversion, since business systems generally store detailed data, and some systems even store timestamp-based data, and the data in the data warehouse is used for analysis and does not require very detailed data, in general, business system data will be aggregated according to the different granularity requirements in the data warehouse.

[0067] Further credibility, for data structure conversion, can also include table / data splitting, where some fields may store multiple data information, for example, the timestamp contains information such as year, month, day, hour, minute, second, etc. Some rules need to split some or all of the time attributes to meet the data aggregation requirements at multiple granularities. Similarly, multiple fields in a table may also have table field splitting. Or row-column conversion, that is, in some cases, the row and column data in the table will need to be converted (also called transposition), for example, before the calculation of collaborative filtering, the relationship between user and term is mutually row and column and can be converted to each other, which can be used to meet the similarity recommendation calculation based on items and users.

[0068] In addition, it can also include data discretization, that is, discretizing the attributes of continuous values ​​into several intervals to help reduce the number of values ​​of a continuous attribute. For example, for the income field, in order to facilitate statistics, it may be divided into several different intervals based on business experience: 0-3000, 3001-5000, 5001-10000, 10001-30000, greater than 30000, or represented by 1, 2, 3, 4, and 5 on this basis. And data standardization, due to the different business meanings of different fields, sometimes it is necessary to eliminate the disparity between the values ​​caused by the different orders of magnitude between the variables. For example, the sales volume is discretized to eliminate the inability to perform composite calculations on multiple columns due to the magnitude relationship between different sales volumes. The data standardization process can also be used to solve the impact of individual attributes with higher values ​​on clustering results, etc.

[0069] S103, assembling the business parameter code and the second template into second template data, generating second parsed data, and storing the second parsed data in a third-party database.

[0070] Furthermore, after obtaining the first parsed data, the present application may obtain the data structure and parameter requirements of the third-party database, extract the basic data by comparison, form the template dynamic parameter items, and then replace the dynamic parameter items in the first template that matches the first parsed data with the parameters in the first parsed data. Thus, the first parsed data and the first template can be integrated, and after the two are assembled, the second parsed data is stored in the third-party database.

[0071] The third-party database may be any database, and this application does not limit this.

[0072] S104, when the aggregation instruction is obtained, the second parsed data in the third-party database is aggregated with the first parsed data in the MongoDB database.

[0073] Furthermore, when receiving the aggregation instruction, the present application can take out the matching rules and data from MongDB and the third-party database. Optionally, in one method, the second parsed data stored in the third-party database and the first parsed data stored in the MongoDB database can be obtained through the pipeline identifier (the pipeline identifier is carried by the data stored in the MongoDB database) and the matching rule. And use the double pointer algorithm to aggregate the project data and the third-party database stage data, aggregate the time, status, and results of the stage data, and obtain the aggregated data, which includes the time, status, and results of the aggregated data.

[0074] In the present application, after obtaining the business parameter code, the business parameter code and the first template are assembled into the first parsed data having a data structure matching the first template, and the first parsed data is stored in the MongoDB database, and the business parameter code and the second template are assembled into the second template data to generate the second parsed data, and after the second parsed data is stored in the third-party database, when the aggregation instruction is obtained, the second parsed data in the third-party database and the first parsed data in the MongoDB database are aggregated. By applying the technical solution of the present application, the dependence on the stage characteristics of the third-party database can be solved, and the dynamic splitting of the own data and the third-party database data can be completed. Then statistics and aggregation are performed to realize the user-customizable functions in the display stage, solve the limitations of development, and realize the flexibility and diversity of product forms.

[0075] Optionally, in another embodiment of the method of the present application, assembling the service parameter code and the first template into first parsed data includes:

[0076] Obtaining a target data structure and a second template corresponding to a third-party database;

[0077] Extracting the data structure identifier in the business parameter code and converting the business parameter code into a target data structure;

[0078] The business parameter code converted into the target data structure is used to replace the dynamic parameter item in the first template group to obtain the first parsed data.

[0079] Optionally, in another embodiment of the method of the present application, before assembling the service parameter code and the first template into the first parsed data, the method further includes:

[0080] Generate a file upload template and an image upload template. The image upload template is used to upload the image file to the private image repository of the MongoDB database.

[0081] Generate a deployment function template, where the deployment function template is used to decompress the first parsed data;

[0082] Based on business scenario requirements, file upload templates, image upload templates, and deployment function templates, a first template is generated, and the first template is used to parse the business parameter code, generate first parsed data, and upload it to the MongoDB database.

[0083] Optionally, in another embodiment of the method of the present application, after assembling the service parameter code and the first template into the first parsed data, the method further includes:

[0084] Recording the matching rules between the first parsed data and the second parsed data;

[0085] The matching rule and the first parsed data are stored together in a MongoDB database.

[0086] Optionally, in another embodiment based on the above method of the present application, when an aggregation instruction is obtained, data aggregation is performed on the first parsed data in the MongoDB database and the second parsed data in the third-party database, including:

[0087] When the aggregation instruction is obtained, the matching rule and the first parsed data in the MongoDB database are obtained;

[0088] Based on the matching rule, obtaining second parsed data from a third-party database;

[0089] The first parsed data and the second parsed data are aggregated.

[0090] Optionally, in another embodiment of the method of the present application, assembling the service parameter code and the first template into first parsed data includes:

[0091] The first parsed data and the second parsed data are aggregated using a double pointer algorithm.

[0092] Optionally, in another embodiment of the method of the present application, before assembling the service parameter code and the second template into the second template data, the method further includes:

[0093] Obtain the data structure and parameter requirements of the third-party database, and generate template dynamic parameter items corresponding to the third-party database;

[0094] Obtaining a to-be-matched template identifier corresponding to the business parameter code, and matching the to-be-matched template identifier with a plurality of preset candidate template identifiers one by one to obtain a third template, where the third template is a template that matches the to-be-matched template identifier;

[0095] The template dynamic parameter items are loaded into the third template to generate the second template.

[0096] Optionally, in another embodiment of the method of the present application, after aggregating the second parsed data in the third-party database with the first parsed data in the MongoDB database, the method further includes:

[0097] Aggregating the second parsed data with the first parsed data to obtain an aggregated data group;

[0098] The aggregated data are grouped into lane data, and the lane data are sent to a recipient specified by the aggregation instruction.

[0099] Furthermore, in this application, the data format and data integrity of the business parameter code can be verified first; the business parameters of the target code input by the user can be checked one by one according to the predefined verification rules. In addition, the business parameters can also be parsed into a data structure that can be converted into a template that matches a third-party database.

[0100] In addition, business parameter parsing splits the complete tasks of continuous integration and continuous deployment into independent data structures according to stages. The parsed data is then converted to extract the parsed identifiers in the business parameters, compare them with the template identifiers, classify and match them to the corresponding templates, replace the dynamic parameter items in the templates, and generate data that meets the requirements of the third-party database (i.e., the second parsed data).

[0101] Furthermore, templates corresponding to different programming languages ​​are selected according to the parameter items input by the user, the parameter commands input by the user are executed, and the generated source code files of different types are packaged into compressed packages of a unified format. Among them, the present application can generate multiple templates (i.e., first templates) for generating first parsed data. For example, the first template can include a template for generating file uploads to support the file upload function. And generate an image upload template to support uploading image files to a private image repository. And generate a continuous deployment function template to decompress the packaged compressed package on the remote server back to the original format, and the parsing supports multiple programming languages ​​and multiple deployment methods.

[0102] Furthermore, the present application can also obtain the data structure and parameter requirements of the third-party database, generate the template dynamic parameter items corresponding to the third-party database, and match the template classification identifier parsed according to the business parameter code with the pre-prepared template to generate a data structure that can be used for third-party storage, and then obtain the template identifier to be matched corresponding to the business parameter code, and match the template identifier to be matched with multiple preset candidate template identifiers one by one, and after obtaining the matched third template, load the template dynamic parameter items into the third template to generate the second template. At the same time, the matching rules of the two are recorded for subsequent data display and analysis. The matching rules and the second parsed data are stored together in the MongoDB database.

[0103] Furthermore, when receiving the aggregation instruction, the present application can take out the matching rules and data from MongDB and the third-party database. Optionally, in one method, the first parsed data stored in the third-party database can be obtained through the pipeline identifier (the pipeline identifier is carried by the data stored in the MongoDB database) and the matching rules. Use the double pointer algorithm to aggregate the project data and the third-party database stage data, aggregate the time, status, and results of the stage data, and obtain the aggregated data, which includes the time, status, and results of the aggregated data.

[0104] In addition, after the second parsed data and the first parsed data are aggregated to obtain an aggregated data group, the present application may assemble the aggregated data group into lane data, and send the lane data to a receiver specified by the aggregation instruction.

[0105] In the present application, after obtaining the business parameter code, the business parameter code and the first template are assembled into the first parsed data having a data structure matching the first template, and the first parsed data is stored in the MongoDB database, and the business parameter code and the second template are assembled into the second template data to generate the second parsed data, and after the second parsed data is stored in the third-party database, when the aggregation instruction is obtained, the second parsed data in the third-party database and the first parsed data in the MongoDB database are aggregated. By applying the technical solution of the present application, the dependence on the stage characteristics of the third-party database can be solved, and the dynamic splitting of the own data and the third-party database data can be completed. Then statistics and aggregation are performed to realize the user-customizable functions in the display stage, solve the limitations of development, and realize the flexibility and diversity of product forms.

[0106] In another embodiment of the present application, Figure 2 As shown, the present application also provides a data aggregation device. It includes an acquisition module 201, a generation module 202, a storage module 203, and an aggregation module 204, wherein:

[0107] An acquisition module, configured to acquire a business parameter code;

[0108] A generating module is configured to assemble the business parameter code and the first template into first parsed data, and store the first parsed data in a MongoDB database, wherein the first parsed data and the first template have a matching data structure;

[0109] A storage module, configured to assemble the business parameter code and the second template into second template data, generate second parsed data, and store the second parsed data in a third-party database;

[0110] The aggregation module is configured to aggregate the second parsed data in the third-party database with the first parsed data in the MongoDB database when an aggregation instruction is obtained.

[0111] In the present application, after obtaining the business parameter code, the business parameter code and the first template are assembled into the first parsed data having a data structure matching the first template, and the first parsed data is stored in the MongoDB database, and the business parameter code and the second template are assembled into the second template data to generate the second parsed data, and after the second parsed data is stored in the third-party database, when the aggregation instruction is obtained, the second parsed data in the third-party database and the first parsed data in the MongoDB database are aggregated. By applying the technical solution of the present application, the dependence on the stage characteristics of the third-party database can be solved, and the dynamic splitting of the own data and the third-party database data can be completed. Then statistics and aggregation are performed to realize the user-customizable functions in the display stage, solve the limitations of development, and realize the flexibility and diversity of product forms.

[0112] In another embodiment of the present application, the acquisition module 201 further includes:

[0113] The acquisition module 201 is configured to acquire the target data structure and the second template corresponding to the third-party database;

[0114] The acquisition module 201 is configured to extract the data structure identifier in the business parameter code and convert the business parameter code into the target data structure;

[0115] The acquisition module 201 is configured to replace the dynamic parameter items in the first template group with the business parameter code converted into the target data structure to obtain the first parsed data.

[0116] In another embodiment of the present application, the acquisition module 201 further includes:

[0117] The acquisition module 201 is configured to generate a file upload template and an image upload template, wherein the image upload template is used to upload the image file to the private image repository of the MongoDB database;

[0118] An acquisition module 201 is configured to generate a deployment function template, wherein the deployment function template is used to decompress the first parsed data;

[0119] The acquisition module 201 is configured to generate the first template based on business scenario requirements, the file upload template, the image upload template and the deployment function template, and the first template is used to parse the business parameter code, generate first parsed data and upload it to the MongoDB database.

[0120] In another embodiment of the present application, the acquisition module 201 further includes:

[0121] An acquisition module 201 is configured to record a matching rule between the first parsed data and the second parsed data;

[0122] The acquisition module 201 is configured to store the matching rule and the first parsed data together in a MongoDB database.

[0123] In another embodiment of the present application, the acquisition module 201 further includes:

[0124] The acquisition module 201 is configured to acquire the matching rule and the first parsed data in the MongoDB database when an aggregation instruction is acquired;

[0125] An acquisition module 201 is configured to acquire the second parsed data from the third-party database based on the matching rule;

[0126] The acquisition module 201 is configured to perform data aggregation on the first parsed data and the second parsed data.

[0127] In another embodiment of the present application, the acquisition module 201 further includes:

[0128] The acquisition module 201 is configured to perform data aggregation on the first parsed data and the second parsed data by using a double pointer algorithm.

[0129] In another embodiment of the present application, the acquisition module 201 further includes:

[0130] The acquisition module 201 is configured to acquire the data structure and parameter requirements of the third-party database and generate a template dynamic parameter item corresponding to the third-party database;

[0131] The acquisition module 201 is configured to acquire the to-be-matched template identifier corresponding to the business parameter code, and match the to-be-matched template identifier with a plurality of preset candidate template identifiers one by one to obtain a third template, wherein the third template is a template that matches the to-be-matched template identifier;

[0132] The acquisition module 201 is configured to load the template dynamic parameter item into the third template to generate the second template.

[0133] In another embodiment of the present application, the acquisition module 201 further includes:

[0134] An acquisition module 201 is configured to aggregate the second parsed data with the first parsed data to obtain an aggregated data group;

[0135] The acquisition module 201 is configured to assemble the aggregated data group into lane data, and send the lane data to a receiver specified by the aggregation instruction.

[0136] Figure 3 3 is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0137] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, the instructions can be executed by an electronic device processor to complete the above data aggregation method, the method comprising: obtaining a business parameter code; assembling the business parameter code with a first template into a first parsed data, and storing the first parsed data in a MongoDB database, wherein the first parsed data has a matching data structure with the first template; assembling the business parameter code with a second template into a second template data, generating a second parsed data, and storing the second parsed data in a third-party database; when the aggregation instruction is obtained, the second parsed data in the third-party database is aggregated with the first parsed data in the MongoDB database. Optionally, the instructions can also be executed by a processor of an electronic device to complete other steps involved in the above exemplary embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, an optical data storage device, and the like.

[0138] In an exemplary embodiment, an application / computer program product is also provided, including one or more instructions, which can be executed by a processor of an electronic device to complete the above-mentioned data aggregation method, the method comprising: obtaining a business parameter code; assembling the business parameter code with a first template into first parsed data, and storing the first parsed data in a MongoDB database, wherein the first parsed data and the first template have a matching data structure; assembling the business parameter code with a second template into second template data, generating second parsed data, and storing the second parsed data in a third-party database; when an aggregation instruction is obtained, the second parsed data in the third-party database is aggregated with the first parsed data in the MongoDB database. Optionally, the above-mentioned instructions can also be executed by a processor of an electronic device to complete other steps involved in the above-mentioned exemplary embodiment.

[0139] Figure 3 FIG. 3 is an example diagram of a computer device 30. A person skilled in the art will understand that Figure 3 It is only an example of the computer device 30 and does not constitute a limitation of the computer device 30. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 30 may also include input and output devices, network access devices, buses, etc.

[0140] The processor 302 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 302 may also be any conventional processor, etc. The processor 302 is the control center of the computer device 30, and uses various interfaces and lines to connect various parts of the entire computer device 30.

[0141] The memory 301 can be used to store computer-readable instructions 303. The processor 302 implements various functions of the computer device 30 by running or executing the computer-readable instructions or modules stored in the memory 301 and calling the data stored in the memory 301. The memory 301 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device 30, etc. In addition, the memory 301 can include a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM) or other non-volatile / volatile storage devices.

[0142] If the module integrated in the computer device 30 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware to complete it through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented.

[0143] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0144] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data aggregation method, characterized in that: include: Get the business parameter code; Generate a file upload template and an image upload template, wherein the image upload template is used to upload the image file to the private image warehouse of the MongoDB database; Generate a deployment function template, wherein the deployment function template is used to decompress the first parsed data; Generate a first template based on business scenario requirements, the file upload template, the image upload template, and the deployment function template, wherein the first template is used to parse the business parameter code, generate first parsed data, and upload the first parsed data to the MongoDB database; Assembling the business parameter code and the first template into first parsed data, and storing the first parsed data in a MongoDB database, wherein the first parsed data and the first template have matching data structures; Assembling the business parameter code and the second template into second template data, generating second parsed data, and storing the second parsed data in a third-party database; When the aggregation instruction is obtained, the second parsed data in the third-party database is aggregated with the first parsed data in the MongoDB database.

2. The method according to claim 1, characterized in that After assembling the business parameter code and the first template into the first parsed data, the method further includes: Recording a matching rule between the first parsed data and the second parsed data; The matching rule and the first parsed data are stored together in a MongoDB database.

3. The method according to claim 2, characterized in that When the aggregation instruction is obtained, the first parsed data in the MongoDB database and the second parsed data in the third-party database are aggregated, including: When the aggregation instruction is obtained, the matching rule and the first parsed data in the MongoDB database are obtained; Based on the matching rule, obtaining the second parsed data from the third-party database; The first parsed data and the second parsed data are aggregated.

4. The method according to claim 3, characterized in that The step of assembling the business parameter code and the first template into first parsed data includes: The first parsed data and the second parsed data are aggregated using a double pointer algorithm.

5. The method according to claim 1, characterized in that Before assembling the service parameter code and the second template into the second template data, the method further includes: Acquire the data structure and parameter requirements of the third-party database, and generate template dynamic parameter items corresponding to the third-party database; Obtaining a to-be-matched template identifier corresponding to the business parameter code, and matching the to-be-matched template identifier with a plurality of preset candidate template identifiers one by one to obtain a third template, wherein the third template is a template that matches the to-be-matched template identifier; The template dynamic parameter item is loaded into the third template to generate the second template.

6. The method according to claim 1, characterized in that After aggregating the second parsed data in the third-party database with the first parsed data in the MongoDB database, the method further includes: Aggregating the second parsed data with the first parsed data to obtain an aggregated data group; The aggregated data groups are assembled into lane data, and the lane data are sent to a receiver specified by the aggregation instruction.

7. A data aggregation device, characterized in that: include: An acquisition module, configured to acquire a business parameter code; Generate a file upload template and an image upload template, wherein the image upload template is used to upload the image file to the private image warehouse of the MongoDB database; generate a deployment function template, wherein the deployment function template is used to decompress the first parsed data; Generate a first template based on business scenario requirements, the file upload template, the image upload template, and the deployment function template, wherein the first template is used to parse the business parameter code, generate first parsed data, and upload the first parsed data to the MongoDB database; A generating module is configured to assemble the business parameter code and the first template into first parsed data, and store the first parsed data in a MongoDB database, wherein the first parsed data and the first template have a matching data structure; A storage module, configured to assemble the business parameter code and the second template into second template data, generate second parsed data, and store the second parsed data in a third-party database; The aggregation module is configured to aggregate the second parsed data in the third-party database with the first parsed data in the MongoDB database when an aggregation instruction is obtained.

8. A computer-readable storage medium for storing computer-readable instructions, characterized in that: When the instructions are executed, the operations of the data aggregation method described in any one of claims 1-6 are performed.