A multi-source heterogeneous data automatic collection intelligent report generation method

Abstract

The application discloses a kind of multi-source heterogeneous data automatic collection intelligent report generation method, it is related to data processing technical field, including the following steps: obtaining target report generation task, extracting target report index and index calculation relationship;Reverse analysis obtains index dependent node and source data node, establishes reverse dependent relationship;Filter data gap node, form data support interval and index support interval;Convergent contribution is calculated in combination with influence path, target data gap node is selected and target source data is obtained;Update index support interval and determine index convergence state, continue data collection when not convergent, reach index convergence state when using already collected data to generate target report.The application can reduce unnecessary data access and data processing, improve report generation efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an intelligent report generation method for automatically aggregating multi-source heterogeneous data. Background Technology

[0002] With the continuous development of information systems and data management platforms, enterprises are generating an increasing number of data sources during production, operation, and management. Significant differences exist in data structures, storage methods, and formats between different business systems. Report generation typically requires acquiring relevant data from multiple data sources and completing data aggregation, indicator calculation, and result display according to the calculation relationships corresponding to the report indicators. In a multi-source heterogeneous data environment, a target report indicator may depend on data fields from multiple data sources simultaneously, and there may be hierarchical calculation relationships between multiple indicators, increasing the complexity of report data aggregation and processing.

[0003] In the prior art, Chinese invention patent CN111444293B discloses an intelligent method for generating reports of multi-source heterogeneous security monitoring data. This method standardizes, classifies, and stores the multi-source heterogeneous data, and then generates reports using report templates. Chinese invention patent CN114610791B discloses a method, apparatus, computer equipment, and storage medium for analyzing data lineage relationships. This method parses the relationships between reports and data queries to establish data lineage relationships between reports, indicators, data tables, and data fields. While existing solutions can achieve unified processing of multi-source data and association of data sources, when generating target reports, they typically obtain relevant data according to pre-configured data ranges or fixed data dependencies.

[0004] When some source data has been acquired while others have not, the impact of the different unacquired data on the calculation results of the target report indicators varies. Existing technologies struggle to adjust the data collection order based on the calculation status of the target report indicators. Even when the target report indicators have stabilized, it may still be necessary to acquire and process the remaining related data, increasing the amount of data access, data transfer, and computation across data sources, thus prolonging the target report generation time. Therefore, reducing unnecessary data collection and processing during the target report generation process becomes a problem that needs to be addressed in the generation of multi-source heterogeneous data reports.

[0005] Therefore, how to provide an intelligent report generation method for automatically aggregating multi-source heterogeneous data is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an intelligent report generation method for automatic aggregation of multi-source heterogeneous data. This invention establishes a reverse dependency relationship corresponding to the target report indicators, forming a data support interval for data gap nodes and an indicator support interval for the target report indicators. Based on the convergence contribution of the data gap nodes, the target data gap nodes are dynamically selected and the target source data is obtained. This solves the problem of unnecessary data access, data transmission, and data processing caused by data aggregation according to fixed data ranges or fixed data dependencies in the prior art. It realizes selective aggregation of multi-source heterogeneous data and target report generation based on the convergence state of the target report indicators.

[0007] A method for generating intelligent reports by automatically aggregating multi-source heterogeneous data according to an embodiment of the present invention includes the following steps: S1. Obtain the target report generation task, and extract the target report indicators and corresponding indicator calculation relationships from the target report generation task; S2. Based on the indicator calculation relationship, reversely analyze the target report indicator to obtain the indicator dependency node and source data node, and establish a reverse dependency relationship based on the data transmission relationship between the target report indicator, indicator dependency node and source data node. S3. Filter data gap nodes based on the data acquisition status of the source data nodes, obtain the data support intervals corresponding to the data gap nodes, and pass the data support intervals to the target report indicators along the reverse dependency relationship to form the indicator support intervals corresponding to the target report indicators. S4. Combine the data support interval of the data gap node and the influence path between the data gap node and the target report indicator to calculate the convergence contribution of the data gap node. S5. Select the target data gap node according to the convergence contribution, form a reverse aggregation path along the reverse dependency relationship corresponding to the target data gap node, and obtain the target source data according to the reverse aggregation path. S6. Update the data acquisition status and indicator support range of the target data gap node using the target source data, and determine the indicator convergence status of the target report indicator based on the updated indicator support range. S7. If the target report indicator has not reached the indicator convergence state, the convergence contribution will be recalculated for the remaining data gap nodes and data collection will continue; if the target report indicator has reached the indicator convergence state, the target report will be generated using the collected data.

[0008] Optionally, S1 specifically includes: Obtain the report generation request and read the report fields, indicator fields, and indicator calculation expressions from the report generation request; The target report generation task is formed based on the report fields and indicator fields, and the target report indicators that need to be generated are extracted from the indicator fields. The expression for calculating the target report indicator is parsed to obtain the indicator items, operation relationships, and operation order involved in the calculation of the target report indicator. Establish the calculation association between the target report indicators and indicator items according to the order of calculation, and form the indicator calculation relationship corresponding to the target report indicators.

[0009] Optionally, the step of reverse parsing from the target report indicators based on the indicator calculation relationship to obtain the indicator dependency nodes and source data nodes specifically involves: Starting from the calculation relationship of the target report indicator, extract the indicator items that have a calculation relationship with the target report indicator in reverse order of the calculation order, and set the extracted indicator items as indicator dependency nodes. Obtain the indicator calculation expression of the indicator item corresponding to the indicator dependency node, continue to reverse parse the indicator item with the indicator calculation expression, extract the next level indicator item participating in the calculation according to the corresponding operation order, and set the next level indicator item as the indicator dependency node of the corresponding level. Repeatedly reverse-parse the indicator items corresponding to the indicator dependency nodes until the final-level indicator item that does not have a corresponding indicator calculation expression is obtained. Read the data source configuration corresponding to the last-level indicator item, obtain the data source identifier, source data table and source data field from the data source configuration, and set the data corresponding to the source data field as the source data node.

[0010] Optionally, the establishment of a reverse dependency relationship based on the data transfer relationship between the target report metric, the metric dependent nodes, and the source data nodes specifically includes: Obtain the computational relationships between the target report metrics and the nodes that the metrics depend on, as well as the computational relationships between nodes that depend on metrics at adjacent levels; Following the reverse parsing order from the target report indicators to the final level indicators, establish a hierarchical correspondence between the target report indicators and the indicator dependent nodes that have a calculation relationship, and establish a hierarchical correspondence between the indicator dependent nodes that have a calculation relationship between adjacent levels. Establish a data source correspondence between the indicator dependency node corresponding to the final-level indicator item and the corresponding source data node; Based on the operational relationships and order of operations in the indicator calculation relationship, configure the data transfer direction and calculation order between the target report indicator and the indicator dependent node, and between adjacent level indicator dependent nodes, and configure the data transfer direction between the indicator dependent node corresponding to the last level indicator item and the source data node. Based on hierarchical correspondence, data source correspondence, data transmission direction, and calculation order, a reverse dependency relationship is formed, extending from the target report indicator through different hierarchical indicator dependency nodes to the source data node.

[0011] Optionally, the step of filtering data gap nodes based on the data acquisition status of source data nodes and obtaining the data support intervals corresponding to the data gap nodes specifically involves: Get the data acquisition status of the source data node and determine whether the source data node has acquired the corresponding source data. Filter source data nodes for which no source data has been obtained as data gap nodes, and obtain the historical data range, related data range, and business constraint range corresponding to the data gap nodes; Based on the range of historical data, the range of related data, and the range of business constraints, determine the range of data values ​​corresponding to the data gap nodes; Analyze the data value relationships between data gap nodes, adjust the data value range corresponding to the data gap nodes with correlation, and obtain the data support interval corresponding to the data gap nodes.

[0012] Optionally, the step of passing the data support interval to the target report indicator along the reverse dependency relationship to form the indicator support interval corresponding to the target report indicator specifically involves: Obtain the data support interval corresponding to the data gap node and the data value corresponding to the source data node of the acquired source data. Pass the data support interval to the corresponding final indicator dependency node, and form a data support interval with the same lower limit and upper limit based on the data value. Pass the formed data support interval to the corresponding final indicator dependency node. According to the hierarchical correspondence and calculation order in the reverse dependency relationship, the data support range is passed layer by layer from the last-level indicator dependency node to the target report indicator; For an indicator-dependent node that receives multiple data support intervals, the lower and upper limits of the multiple data support intervals are calculated according to the corresponding operation relationship to obtain the lower and upper limits of the corresponding indicator-dependent node, and thus form the data support interval of the corresponding indicator-dependent node. The data support interval of the corresponding indicator dependency node is passed to the indicator dependency node of the next higher level. The interval calculation and data support interval passing are repeated until the data support interval is passed to the indicator dependency node that has a calculation relationship with the target report indicator. According to the calculation relationship and calculation order corresponding to the target report indicators, the data support interval corresponding to the indicator dependency node that has a calculation relationship with the target report indicators is calculated to obtain the lower limit and upper limit of the interval of the target report indicators, and form the indicator support interval corresponding to the target report indicators.

[0013] Optionally, S4 specifically includes: Extract the data gap node from the reverse dependency relationship and the index dependency nodes that the data gap node passes through in sequence to the target report index, and obtain the operation relationship and calculation order between adjacent nodes to form the influence path corresponding to the data gap node; For any data gap node, the lower limit of the data support interval is set to a first fixed interval with the same lower and upper limits, and the upper limit of the data support interval is set to a second fixed interval with the same lower and upper limits, while keeping the data support intervals corresponding to the remaining data gap nodes unchanged. According to the operational relationship and calculation order in the influence path, the data support intervals corresponding to the first fixed interval and the remaining data gap nodes are passed to the target report indicators layer by layer to obtain the first candidate indicator support interval; The data support intervals corresponding to the second fixed interval and the remaining data gap nodes are passed to the target report indicators layer by layer to obtain the support intervals of the second candidate indicator. Calculate the interval widths of the indicator support interval, the first candidate indicator support interval, and the second candidate indicator support interval corresponding to the target report indicator, and select the larger interval width between the first candidate indicator support interval and the second candidate indicator support interval. Calculate the difference between the width of the support interval of the target report indicator and the larger interval width, and set the difference as the convergence contribution of the corresponding data gap node; Repeatedly calculate the convergence contribution of different data gap nodes to obtain the convergence contribution of all data gap nodes.

[0014] Optionally, S5 specifically includes: Compare the convergence contributions of the data gap nodes and select the data gap node with the largest convergence contribution as the target data gap node. Extract the metric dependency nodes between the target report metric and the target data gap node along the reverse dependency relationship, and obtain the data source identifier, source data table and source data field corresponding to the target data gap node; A reverse aggregation path is formed based on the correspondence between the target report indicators, indicator dependency nodes, target data gap nodes, data source identifiers, source data tables, and source data fields; Access the corresponding data source according to the reverse aggregation path, and obtain the target source data from the source data field of the source data table.

[0015] Optionally, S6 specifically includes: Update the data acquisition status of the target data gap node to the acquired status, and form a data support interval with the same lower and upper limits based on the target source data; The updated data support interval is passed to the target report indicator layer by layer along the reverse dependency relationship. The data support interval of each level indicator dependency node is recalculated to obtain the updated indicator support interval. Obtain the report display precision corresponding to the target report indicator, and process the lower limit and upper limit of the updated indicator support interval according to the report display precision; If the processing result corresponding to the lower limit of the interval is the same as the processing result corresponding to the upper limit of the interval, the target report indicator is determined to have reached the indicator convergence state; if the processing result corresponding to the lower limit of the interval is different from the processing result corresponding to the upper limit of the interval, the target report indicator is determined to have not reached the indicator convergence state.

[0016] Optionally, S7 specifically includes: If the target report indicator has not reached the indicator convergence state, filter the remaining data gap nodes from the data gap nodes whose data acquisition status is not acquired, and recalculate the convergence contribution of the remaining data gap nodes. Select new target data gap nodes based on the recalculated convergence contribution, form a new reverse aggregation path and obtain the corresponding target source data, and repeatedly update the indicator support interval and determine the indicator convergence status. If the target report indicator reaches the indicator convergence state, stop the data collection of the remaining data gap nodes, and calculate the collected data according to the indicator calculation relationship to obtain the report indicator value corresponding to the target report indicator. The report index values ​​are mapped to the report fields in the target report generation task to generate the target report.

[0017] The beneficial effects of this invention are: (1) By establishing the reverse dependency relationship corresponding to the target report indicator, the present invention transmits the data support interval corresponding to the data gap node to the target report indicator layer by layer along the reverse dependency relationship, forming the indicator support interval corresponding to the target report indicator. This can characterize the impact of not obtaining source data on the value range of the target report indicator and provide a quantitative basis for the necessity of collecting different data gap nodes. (2) The present invention combines the data support interval of the data gap node and the influence path between the data gap node and the target report indicator to calculate the convergence contribution, and selects the target data gap node according to the convergence contribution to form a reverse aggregation path. This can prioritize the acquisition of target source data with a greater convergence effect on the indicator support interval, and reduce invalid access, data transmission and data processing of low-impact source data. (3) The present invention utilizes the target source data to continuously update the indicator support interval, and determines the indicator convergence state of the target report indicator based on the updated indicator support interval. After the target report indicator reaches the indicator convergence state, the data collection of the remaining data gap nodes is stopped. The target report can be generated without completing the collection of all related source data, reducing the data access volume and computation volume in the multi-source heterogeneous data collection process, and improving the generation efficiency of the target report. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an intelligent report generation method for automatic aggregation of multi-source heterogeneous data proposed in this invention; Figure 2 This is a schematic diagram of the reverse dependency relationship proposed in this invention; Figure 3 This is a schematic diagram illustrating the layer-by-layer transmission of the indicator support interval proposed in this invention; Figure 4 This is a schematic diagram illustrating the calculation of the convergence contribution proposed in this invention; Figure 5 This is a schematic diagram of the target data gap node and reverse aggregation path proposed in this invention; Figure 6 This is a schematic diagram illustrating the convergence state determination of the index proposed in this invention; Figure 7 This is a schematic diagram of the multi-round data aggregation iteration proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-7 A method for generating intelligent reports by automatically aggregating multi-source heterogeneous data includes the following steps: S1. Obtain the target report generation task, and extract the target report indicators and corresponding indicator calculation relationships from the target report generation task; S2. Based on the indicator calculation relationship, reversely analyze the target report indicator to obtain the indicator dependency node and source data node, and establish a reverse dependency relationship based on the data transmission relationship between the target report indicator, indicator dependency node and source data node. S3. Filter data gap nodes based on the data acquisition status of the source data nodes, obtain the data support intervals corresponding to the data gap nodes, and pass the data support intervals to the target report indicators along the reverse dependency relationship to form the indicator support intervals corresponding to the target report indicators. S4. Combine the data support interval of the data gap node and the influence path between the data gap node and the target report indicator to calculate the convergence contribution of the data gap node. S5. Select the target data gap node according to the convergence contribution, form a reverse aggregation path along the reverse dependency relationship corresponding to the target data gap node, and obtain the target source data according to the reverse aggregation path. S6. Update the data acquisition status and indicator support range of the target data gap node using the target source data, and determine the indicator convergence status of the target report indicator based on the updated indicator support range. S7. If the target report indicator has not reached the indicator convergence state, the convergence contribution will be recalculated for the remaining data gap nodes and data collection will continue; if the target report indicator has reached the indicator convergence state, the target report will be generated using the collected data.

[0021] In this embodiment, S1 specifically refers to: Obtain the report generation request, and retrieve the report fields, indicator fields, and indicator calculation expressions from the report generation request. The report fields correspond to the fields that need to be generated in the target report, the indicator fields correspond to the fields in the report fields that need to be calculated to obtain the results, and the indicator calculation expressions record the indicator items that the indicator fields participate in the calculation, the operation relationship between the indicator items, and the order of operation. Establish the correspondence between report fields and indicator fields according to the order of report fields in the report generation request, and form the target report generation task by combining the report fields and indicator fields after establishing the correspondence. The indicator fields in the target report generation task are obtained one by one. The indicator fields that need to be calculated by the indicator expression to obtain the report result are extracted as target report indicators, and the indicator calculation expressions corresponding to the target report indicators are obtained. Extract the indicator fields that participate in the calculation of the target report indicator from the indicator calculation expression in sequence, and record the extracted indicator fields as indicator items; Identify the addition, subtraction, multiplication, and division operations between adjacent indicator items in the indicator calculation expression to obtain the operational relationships between the indicator items; Arrange the indicator items according to the order of the parentheses and the corresponding operations in the indicator calculation expression to obtain the operation order of the indicator items. Establish calculation associations between the target report indicators and the indicator items involved in the calculation of the target report indicators according to the calculation order, record the calculation relationships between adjacent indicator items, and form the indicator calculation relationship corresponding to the target report indicators.

[0022] In this implementation, the target report metrics are reverse-analyzed based on the metric calculation relationship to obtain the metric dependency nodes and source data nodes, specifically: Obtain the indicator items that directly participate in the calculation of the target report indicator from the indicator calculation relationship corresponding to the target report indicator, arrange the indicator items in reverse order of the calculation order, and set the indicator items that directly participate in the calculation of the target report indicator as the first-level indicator dependency node. Obtain the indicator calculation expression of the indicator item corresponding to the first-level indicator dependency node, extract the next-level indicator item participating in the calculation from the indicator item with the indicator calculation expression, arrange the next-level indicator items according to the operation order in the corresponding indicator calculation expression, and set the next-level indicator item as the second-level indicator dependency node. Following the order of expanding from the target report indicator to the next level indicator item, for indicator items with indicator calculation expressions, continue to extract the next level indicator items that participate in the calculation. Each time an indicator item expansion is completed, an indicator dependency node level is added until the indicator item no longer has a corresponding indicator calculation expression. Set the indicator items that do not have corresponding indicator calculation expressions as the last-level indicator items, and set the indicator dependency nodes corresponding to the last-level indicator items as the last-level indicator dependency nodes. Get the data source configuration corresponding to the last-level indicator item. The data source configuration records the data source identifier, source data table and source data field corresponding to the last-level indicator item. The data source identifier represents the data source where the source data of the last-level indicator item is located, the source data table represents the data table where the source data is located, and the source data field represents the data field that participates in the calculation of the last-level indicator item. Based on the correspondence between the final-level indicator item and the data source configuration, extract the data source identifier, source data table and source data field respectively to locate the source data corresponding to the final-level indicator item; By setting the data corresponding to the source data field as the source data node, the reverse parsing result is obtained, which extends from the dependent nodes of different levels of indicators to the source data node layer by layer.

[0023] In this implementation, a reverse dependency relationship is established based on the data transfer relationship between the target report metric, the metric dependent nodes, and the source data nodes, specifically as follows: Obtain the indicator calculation relationship corresponding to the target report indicator, extract the calculation association between the target report indicator and the first-level indicator dependency node from the indicator calculation relationship, and extract the calculation association between adjacent level indicator dependency nodes from the indicator calculation expressions corresponding to indicator dependency nodes of different levels. The indicator dependency nodes at different levels are arranged in the reverse parsing order from the target report indicator to the last-level indicator item. The indicator dependency nodes that directly participate in the calculation of the target report indicator are mapped to the target report indicator, and the indicator dependency nodes of the next level that participate in the calculation of the indicator dependency nodes of the previous level are mapped to the indicator dependency nodes of the previous level, thus forming a hierarchical correspondence between the target report indicator and the indicator dependency nodes at different levels. Obtain the data source configuration corresponding to the last-level indicator item, match the data source identifier, source data table and source data field in the data source configuration with the indicator dependency node corresponding to the last-level indicator item, and match the source data node corresponding to the source data field with the indicator dependency node corresponding to the last-level indicator item to form a data source correspondence relationship; Extract the operation relationship and operation order from the indicator calculation relationship and the indicator calculation expression corresponding to the indicator dependency node at different levels, and configure the data transmission direction according to the direction from the source data node to the last level indicator dependency node, the last level indicator dependency node to the next level indicator dependency node, and the next level indicator dependency node to the target report indicator. Configure the calculation order between adjacent level indicator dependency nodes according to the order in which each indicator item participates in the calculation in the indicator calculation expression, and configure the calculation order between indicator dependency nodes that have a calculation relationship with the target report indicator according to the order of calculation in the indicator calculation expression corresponding to the target report indicator. The target report indicators and the dependent nodes of different levels of indicators are associated according to the hierarchical correspondence. The final level indicator dependent nodes and the source data nodes are associated according to the data source correspondence. The data transmission direction and calculation order between each associated node are recorded to form a reverse dependency relationship that extends from the target report indicators through the dependent nodes of different levels of indicators to the source data nodes.

[0024] In this embodiment, data gap nodes are filtered based on the data acquisition status of the source data nodes, and the data support intervals corresponding to the data gap nodes are obtained, specifically as follows: Obtain the data source identifier, source data table, and source data field corresponding to the source data node; query the source data corresponding to the source data field; mark the source data node that successfully obtains the corresponding source data as obtained; mark the source data node that does not obtain the corresponding source data as not obtained; and filter the source data node in the not obtained state as data gap node. Obtain the data records of the source data fields corresponding to the data gap nodes in the historical report generation task, extract the minimum and maximum data values ​​in the historical data records, and form the historical data range from the minimum and maximum data values; Obtain source data nodes that have computational association with data gap nodes from the reverse dependency relationship, extract the data values ​​corresponding to the source data nodes with the obtained state, and convert the data range allowed for the data gap nodes according to the operation relationship in the indicator calculation relationship to form the associated data range; Obtain the data type, allowed value boundaries, and data relationships defined in the indicator calculation relationship of the source data fields corresponding to the data gap nodes, and use the data range jointly defined by the allowed value boundaries and data relationships as the business constraint range; Extract the lower and upper limits of the historical data range, the associated data range, and the business constraint range respectively. Select the maximum value in the lower limit as the lower limit of the data range and select the minimum value in the upper limit as the upper limit of the data range to obtain the data range corresponding to the data gap node. For data gap nodes with computational correlation, the data value range of a data gap node is converted to the associated data gap node according to the operation relationship in the indicator calculation relationship. The converted interval is then used to further limit the corresponding data value range until the data value range of the data gap node with computational correlation no longer changes. The data value range after the limit is completed is used as the data support interval of the corresponding data gap node.

[0025] In this implementation, the data support interval is passed to the target report indicator along the reverse dependency relationship, forming the indicator support interval corresponding to the target report indicator, specifically as follows: Obtain the data support interval corresponding to the data gap node and the data value corresponding to the source data node of the acquired source data. Pass the data support interval of the data gap node to the corresponding final indicator dependency node. Use the data value of the acquired source data as the lower limit and upper limit of the interval to form the data support interval. Pass the formed data support interval to the corresponding final indicator dependency node. According to the hierarchical correspondence and calculation order in the reverse dependency relationship, starting from the last-level indicator dependency node, obtain multiple data support intervals and corresponding operation relationships that participate in the calculation of the same upper-level indicator dependency node. When the operation relationship is addition, the lower limits of the data support intervals involved in the calculation are added together to obtain the lower limit of the interval of the corresponding indicator dependent node, and the upper limits of the data support intervals involved in the calculation are added together to obtain the upper limit of the interval of the corresponding indicator dependent node. When the operation relationship is subtraction, the lower limit of the interval supported by the subtracted data is subtracted from the upper limit of the interval supported by the subtracted data to obtain the lower limit of the interval of the corresponding indicator dependent node, and the upper limit of the interval supported by the subtracted data is subtracted from the lower limit of the interval supported by the subtracted data to obtain the upper limit of the interval of the corresponding indicator dependent node. When the operation relationship is multiplication, calculate the product of the lower limit of the two data support intervals, the lower limit of the interval and the upper limit of the interval, the upper limit of the interval and the lower limit of the interval, and the upper limit of the interval and the upper limit of the interval respectively. Select the minimum value from the obtained products as the lower limit of the interval of the corresponding indicator dependent node, and select the maximum value as the upper limit of the interval of the corresponding indicator dependent node. When the operation relationship is division, obtain the support interval of the divisor data. If the support interval of the divisor data does not contain zero value, calculate the ratio of the lower limit of the support interval of the dividend data to the lower limit of the support interval of the divisor data, the lower limit of the support interval to the upper limit of the support interval, the upper limit of the support interval to the lower limit of the support interval, and the upper limit of the support interval to the upper limit of the support interval. Select the minimum value from the obtained ratios as the lower limit of the corresponding indicator dependency node, and select the maximum value as the upper limit of the corresponding indicator dependency node. When the divisor data support interval contains zero values, zero values ​​that cannot be used as divisors are excluded according to the business constraint range, and the data support interval after excluding zero values ​​is used for interval calculation; The interval calculations corresponding to the same indicator dependency node are completed sequentially according to the operation order. The lower limit and upper limit of the interval are used to form the data support interval of the corresponding indicator dependency node, and the formed data support interval is passed to the next higher level indicator dependency node. Repeatedly retrieve the data support interval, operation relationship and calculation order corresponding to the indicator dependency node of the previous level, and perform interval calculation according to the corresponding addition, subtraction, multiplication or division relationship until the data support interval is passed to the indicator dependency node that has a calculation relationship with the target report indicator; According to the calculation relationship and calculation order corresponding to the target report indicator, the data support interval corresponding to the indicator dependency node that is calculationally related to the target report indicator is calculated to obtain the lower limit and upper limit of the interval of the target report indicator, and the lower limit and upper limit of the interval of the target report indicator are combined to form the indicator support interval corresponding to the target report indicator.

[0026] In this embodiment, S4 specifically refers to: The data transmission direction from the data gap node to the target report indicator is obtained from the reverse dependency relationship. The indicator dependency nodes are extracted sequentially according to the data transmission direction. The operation relationship and calculation order between the data gap node and the indicator dependency node, between adjacent level indicator dependency nodes, and between the indicator dependency node and the target report indicator are extracted. The data gap node, indicator dependency node, target report indicator, operation relationship and calculation order are arranged in sequence to form the influence path corresponding to the data gap node. For a data gap node, obtain the lower limit and upper limit of the data support interval corresponding to the data gap node. Use the lower limit of the interval as the new lower limit and upper limit of the interval to form the first fixed interval. Use the upper limit of the interval as the new lower limit and upper limit of the interval to form the second fixed interval. The remaining data gap nodes continue to use the corresponding data support interval. The source data nodes that have obtained the source data continue to use the data support interval with the same lower limit and upper limit of the interval. The first fixed interval is passed to the corresponding last-level indicator dependency node, and interval calculation is performed with other data support intervals that participate in the calculation of the same indicator dependency node in the influence path according to the operation relationship and calculation order in the influence path. The data support intervals corresponding to the indicator dependency nodes at different levels are obtained layer by layer until the first candidate indicator support interval corresponding to the target report indicator is obtained. The second fixed interval is passed to the corresponding final-level indicator dependency node, and interval calculation is performed with other data support intervals that participate in the calculation of the same indicator dependency node in the same influence path, according to the same influence path, operation relationship and calculation order. The data support intervals corresponding to different levels of indicator dependency nodes are obtained layer by layer until the second candidate indicator support interval corresponding to the target report indicator is obtained. Obtain the lower and upper limits of the support intervals for the indicator, the first candidate indicator, and the second candidate indicator, respectively. Subtract the corresponding lower limit from the upper limit of each interval to obtain the width of the support interval, the width of the first candidate indicator, and the width of the second candidate indicator. Compare the interval widths of the first candidate indicator support interval and the second candidate indicator support interval, select the interval width with the larger value, and subtract the interval width with the larger value from the interval width of the indicator support interval to obtain the convergence contribution corresponding to the data gap node; process the remaining data gap nodes in the same way to obtain the convergence contribution corresponding to each data gap node.

[0027] In this embodiment, S5 specifically refers to: Obtain the convergence contribution of each data gap node, compare the convergence contributions, and select the data gap node with the largest convergence contribution as the target data gap node. Locate the target data gap node from the reverse dependency relationship, extract the index dependency nodes passed through layer by layer along the data transmission direction corresponding to the target data gap node to the target report index, and arrange the index dependency nodes in the hierarchical order from the target report index to the target data gap node. Obtain the data source configuration corresponding to the target data gap node, extract the data source identifier, source data table and source data field from the data source configuration, and establish the corresponding relationship between the target report indicators, indicator dependency nodes arranged in hierarchical order, target data gap node, data source identifier, source data table and source data field in sequence to form a reverse aggregation path; Locate the data source corresponding to the target data gap node according to the data source identifier in the reverse aggregation path, locate the data table corresponding to the target data gap node according to the source data table, and extract the data value corresponding to the target data gap node from the data table according to the source data fields; use the extracted data value as the target source data, and establish the correspondence between the target source data and the target data gap node.

[0028] In this embodiment, S6 specifically refers to: Obtain the target data gap node corresponding to the target source data, update the data acquisition status of the target data gap node from the unacquired state to the acquired state, and use the data value of the target source data as both the lower limit and the upper limit of the interval to form the data support interval after the target data gap node is updated. The updated data support interval of the target data gap node is passed to the corresponding last-level indicator dependency node, and the indicator dependency nodes with computational association with the target data gap node are obtained layer by layer along the data transmission direction in the reverse dependency relationship. According to the operation relationship and calculation order corresponding to the nodes of each level of indicator dependence, the data support interval after the target data gap node is updated is calculated with the other data support intervals participating in the calculation of the same indicator dependence node. The lower limit and upper limit of the interval of the corresponding indicator dependence node are updated layer by layer, and the data support interval after the node of each level of indicator dependence is formed. The updated data support intervals of the indicator dependent nodes that are computationally related to the target report indicator are calculated according to the operation relationship and calculation order corresponding to the target report indicator to obtain the lower limit and upper limit of the updated interval of the target report indicator, and thus form the updated indicator support interval. Obtain the report display precision corresponding to the target report indicators from the target report generation task. The report display precision includes the number of decimal places retained by the target report indicators in the target report and the corresponding numerical rounding method. The updated indicator support interval is processed according to the decimal places and rounding method in the report display precision, and the lower limit and upper limit of the interval are obtained respectively to obtain the report display value corresponding to the lower limit and the report display value corresponding to the upper limit. If the report value corresponding to the lower limit of the interval is the same as the report value corresponding to the upper limit of the interval, the target report indicator is determined to have reached the indicator convergence state; if the report value corresponding to the lower limit of the interval is different from the report value corresponding to the upper limit of the interval, the target report indicator is determined to have not reached the indicator convergence state.

[0029] In this embodiment, S7 specifically refers to: If the target report indicator has not reached the indicator convergence state, and the data gap node whose data acquisition status is still not acquired is still selected as the remaining data gap node; Keeping the data support intervals corresponding to the acquired source data unchanged, the first candidate indicator support interval and the second candidate indicator support interval are formed according to the data support intervals corresponding to the remaining data gap nodes and the influence path between the remaining data gap nodes and the target report indicators, respectively. The convergence contribution of the remaining data gap nodes is updated according to the difference in interval width between the indicator support interval and the first and second candidate indicator support intervals. Compare the updated convergence contributions, select the remaining data gap node with the largest convergence contribution as the new target data gap node, form a new reverse aggregation path along the reverse dependency relationship, and obtain the new target source data according to the new reverse aggregation path. Update the data acquisition status and data support interval of the corresponding target data gap node according to the new target source data, update the data support interval of the indicator dependent node and the indicator support interval of the target report indicator layer by layer along the reverse dependency relationship, and process the lower limit and upper limit of the updated indicator support interval according to the report display precision. If the report value corresponding to the lower limit of the interval is different from the report value corresponding to the upper limit of the interval, continue to filter the remaining data gap nodes from the data gap nodes whose data acquisition status is not acquired, and repeatedly update the convergence contribution, select the target data gap node, acquire the target source data, and update the indicator support interval; If the report display value corresponding to the lower limit of the interval is the same as the report display value corresponding to the upper limit of the interval, it is determined that the target report indicator has reached the indicator convergence state, the data collection corresponding to the remaining data gap nodes is stopped, and the same report display value corresponding to the lower limit and upper limit of the interval is used as the report indicator value corresponding to the target report indicator. Obtain the report fields corresponding to the target report indicators from the target report generation task, configure the report indicator values ​​to the corresponding report fields, and form the target report according to the arrangement relationship of the report fields in the target report generation task.

[0030] Example 1: To verify the feasibility of this invention in practice, it was applied to the automatic generation of monthly operational analysis reports for a manufacturing company. The company's business data is stored in the order system, procurement system, inventory system, logistics system, financial system, e-commerce platform, and customer management system, involving 18 source data nodes and approximately 5.86 million monthly business records. The target report indicator is the gross profit margin, with a report display accuracy of 0.1%. The indicator is calculated as the ratio of operating revenue minus operating costs to operating revenue. Operating revenue is related to product sales revenue, service revenue, refund amount, and discount amount; operating costs are related to procurement costs, processing costs, and logistics costs. Due to differences in the timing of monthly settlements across different business systems, some source data may have been acquired while others have not yet been acquired when generating the report.

[0031] After obtaining the target report generation task, reverse analysis is performed on the indicator calculation relationship corresponding to the gross profit margin to form a reverse dependency relationship consisting of gross profit margin, operating revenue, operating cost, indicator dependency nodes at different levels, and source data nodes. Data gap nodes are screened according to the data acquisition status of the source data nodes, and a data support interval is formed by combining the historical data range of the past 90 days, the range of acquired related data, and the business constraint range. The interval is calculated layer by layer along the reverse dependency relationship, and the initial indicator support interval is [18.60%, 24.30%].

[0032] For each data gap node, the lower and upper limits of the data support interval are respectively formed into a first fixed interval and a second fixed interval. These are then passed to the overall gross profit margin to form a first candidate indicator support interval and a second candidate indicator support interval. The convergence contribution is obtained by the difference between the interval widths of the indicator support interval and the candidate indicator support interval. Target data gap nodes are selected round by round according to the convergence contribution. The data source identifier, source data table, and source data field are located via the reverse aggregation path, and the target source data is obtained. The indicator support interval is updated after each acquisition of the target source data. When both the lower and upper limits of the interval are processed to 21.0% after being processed with a 0.1% report display precision, the data aggregation of the remaining data gap nodes is stopped, and the target report is generated using the aggregated data.

[0033] To compare the data aggregation results, under the same target report generation task, the same database operating environment, and the same time range, the method of completing the aggregation of all related data according to fixed data dependencies was used as a comparison. The test results are shown in Table 1.

[0034] Table 1. Performance Comparison of Target Report Generation under Different Data Collection Methods

[0035] As shown in Table 1, when facing the same 18 related source data nodes, the fixed data dependency aggregation method requires accessing all source data nodes. This invention stops aggregating the remaining data after the indicator convergence state is reached within the indicator support interval, actually only accessing 11 source data nodes. The business record processing volume decreased from 5.86 million to 3.37 million, the cross-data source transmission volume decreased from 4.82GB to 2.74GB, the number of data queries decreased by 17, and the target report generation time was shortened from 52.4s to 31.6s. Meanwhile, the overall gross profit margin remained at 21.0%, indicating that reducing source data access and processing still maintained consistency in the final display results of the target report indicators. Table 2 shows the changes in the indicator support interval and target data gap nodes of the overall gross profit margin during the data aggregation process.

[0036] Table 2. Convergence Data of the Support Range for the Comprehensive Gross Profit Margin Indicator in Each Round

[0037] As shown in Table 2, the initial support interval width for the indicator is 5.70%. The procurement cost has the largest convergence contribution and is therefore initially selected as the target data gap node. After obtaining the procurement cost, the indicator support interval narrows to [20.10%, 22.40%]. The convergence contribution of the remaining data gap nodes is recalculated, and then the refund amount, logistics cost, and discount amount are collected sequentially. After the fourth round of collection, the indicator support interval narrows to [21.02%, 21.04%]. The lower and upper limits of the interval are both 21.0% after processing with a 0.1% report display precision. At this point, there is still one data gap node, but the remaining data support interval can no longer change the report display value of the target indicator. Therefore, collection is stopped, verifying the feasibility of dynamically selecting target source data based on the convergence contribution and ending data collection based on the indicator convergence status.

[0038] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatically aggregating multi-source heterogeneous data to generate intelligent reports, characterized in that, Includes the following steps: S1. Obtain the target report generation task, and extract the target report indicators and corresponding indicator calculation relationships from the target report generation task; S2. Based on the indicator calculation relationship, reversely analyze the target report indicator to obtain the indicator dependency node and source data node, and establish a reverse dependency relationship based on the data transmission relationship between the target report indicator, indicator dependency node and source data node. S3. Filter data gap nodes based on the data acquisition status of the source data nodes, obtain the data support intervals corresponding to the data gap nodes, and pass the data support intervals to the target report indicators along the reverse dependency relationship to form the indicator support intervals corresponding to the target report indicators. S4. Combine the data support interval of the data gap node and the influence path between the data gap node and the target report indicator to calculate the convergence contribution of the data gap node. S5. Select the target data gap node according to the convergence contribution, form a reverse aggregation path along the reverse dependency relationship corresponding to the target data gap node, and obtain the target source data according to the reverse aggregation path. S6. Update the data acquisition status and indicator support range of the target data gap node using the target source data, and determine the indicator convergence status of the target report indicator based on the updated indicator support range. S7. If the target report indicator has not reached the indicator convergence state, the convergence contribution will be recalculated for the remaining data gap nodes and data collection will continue; if the target report indicator has reached the indicator convergence state, the target report will be generated using the collected data.

2. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, Specifically, S1 is: Obtain the report generation request and read the report fields, indicator fields, and indicator calculation expressions from the report generation request; The target report generation task is formed based on the report fields and indicator fields, and the target report indicators that need to be generated are extracted from the indicator fields. The expression for calculating the target report indicator is parsed to obtain the indicator items, operation relationships, and operation order involved in the calculation of the target report indicator. Establish the calculation association between the target report indicators and indicator items according to the order of calculation, and form the indicator calculation relationship corresponding to the target report indicators.

3. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, The process of reverse parsing from the target report indicators based on the indicator calculation relationship to obtain the indicator dependency nodes and source data nodes is as follows: Starting from the calculation relationship of the target report indicator, extract the indicator items that have a calculation relationship with the target report indicator in reverse order of the calculation order, and set the extracted indicator items as indicator dependency nodes. Obtain the indicator calculation expression of the indicator item corresponding to the indicator dependency node, continue to reverse parse the indicator item with the indicator calculation expression, extract the next level indicator item participating in the calculation according to the corresponding operation order, and set the next level indicator item as the indicator dependency node of the corresponding level. Repeatedly reverse-parse the indicator items corresponding to the indicator dependency nodes until the final-level indicator item that does not have a corresponding indicator calculation expression is obtained. Read the data source configuration corresponding to the last-level indicator item, obtain the data source identifier, source data table and source data field from the data source configuration, and set the data corresponding to the source data field as the source data node.

4. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, The establishment of reverse dependencies based on the data transfer relationships between the target report metrics, metric dependent nodes, and source data nodes is specifically as follows: Obtain the computational relationships between the target report metrics and the nodes that the metrics depend on, as well as the computational relationships between nodes that depend on metrics at adjacent levels; Following the reverse parsing order from the target report indicators to the final level indicators, establish a hierarchical correspondence between the target report indicators and the indicator dependent nodes that have a calculation relationship, and establish a hierarchical correspondence between the indicator dependent nodes that have a calculation relationship between adjacent levels. Establish a data source correspondence between the indicator dependency node corresponding to the final-level indicator item and the corresponding source data node; Based on the operational relationships and order of operations in the indicator calculation relationship, configure the data transfer direction and calculation order between the target report indicator and the indicator dependent node, and between adjacent level indicator dependent nodes, and configure the data transfer direction between the indicator dependent node corresponding to the last level indicator item and the source data node. Based on hierarchical correspondence, data source correspondence, data transmission direction, and calculation order, a reverse dependency relationship is formed, extending from the target report indicator through different hierarchical indicator dependency nodes to the source data node.

5. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, The process of filtering data gap nodes based on the data acquisition status of source data nodes and obtaining the data support intervals corresponding to the data gap nodes specifically involves: Get the data acquisition status of the source data node and determine whether the source data node has acquired the corresponding source data. Filter source data nodes for which no source data has been obtained as data gap nodes, and obtain the historical data range, related data range, and business constraint range corresponding to the data gap nodes; Based on the range of historical data, the range of related data, and the range of business constraints, determine the range of data values ​​corresponding to the data gap nodes; Analyze the data value relationships between data gap nodes, adjust the data value range corresponding to the data gap nodes with correlation, and obtain the data support interval corresponding to the data gap nodes.

6. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, The process of passing the data support interval to the target report indicator along the reverse dependency relationship to form the indicator support interval corresponding to the target report indicator is as follows: Obtain the data support interval corresponding to the data gap node and the data value corresponding to the source data node of the acquired source data. Pass the data support interval to the corresponding final indicator dependency node, and form a data support interval with the same lower limit and upper limit based on the data value. Pass the formed data support interval to the corresponding final indicator dependency node. According to the hierarchical correspondence and calculation order in the reverse dependency relationship, the data support range is passed layer by layer from the last-level indicator dependency node to the target report indicator; For an indicator-dependent node that receives multiple data support intervals, the lower and upper limits of the multiple data support intervals are calculated according to the corresponding operation relationship to obtain the lower and upper limits of the corresponding indicator-dependent node, and thus form the data support interval of the corresponding indicator-dependent node. The data support interval of the corresponding indicator dependency node is passed to the indicator dependency node of the next higher level. The interval calculation and data support interval passing are repeated until the data support interval is passed to the indicator dependency node that has a calculation relationship with the target report indicator. According to the calculation relationship and calculation order corresponding to the target report indicators, the data support interval corresponding to the indicator dependency node that has a calculation relationship with the target report indicators is calculated to obtain the lower limit and upper limit of the interval of the target report indicators, and form the indicator support interval corresponding to the target report indicators.

7. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, Specifically, S4 is: Extract the data gap node from the reverse dependency relationship and the index dependency nodes that the data gap node passes through in sequence to the target report index, and obtain the operation relationship and calculation order between adjacent nodes to form the influence path corresponding to the data gap node; For any data gap node, the lower limit of the data support interval is set to a first fixed interval with the same lower and upper limits, and the upper limit of the data support interval is set to a second fixed interval with the same lower and upper limits, while keeping the data support intervals corresponding to the remaining data gap nodes unchanged. According to the operational relationship and calculation order in the influence path, the data support intervals corresponding to the first fixed interval and the remaining data gap nodes are passed to the target report indicators layer by layer to obtain the first candidate indicator support interval; The data support intervals corresponding to the second fixed interval and the remaining data gap nodes are passed to the target report indicators layer by layer to obtain the support intervals of the second candidate indicator. Calculate the interval widths of the indicator support interval, the first candidate indicator support interval, and the second candidate indicator support interval corresponding to the target report indicator, and select the larger interval width between the first candidate indicator support interval and the second candidate indicator support interval. Calculate the difference between the width of the support interval of the target report indicator and the larger interval width, and set the difference as the convergence contribution of the corresponding data gap node; Repeatedly calculate the convergence contribution of different data gap nodes to obtain the convergence contribution of all data gap nodes.

8. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, Specifically, S5 is: Compare the convergence contributions of the data gap nodes and select the data gap node with the largest convergence contribution as the target data gap node. Extract the metric dependency nodes between the target report metric and the target data gap node along the reverse dependency relationship, and obtain the data source identifier, source data table and source data field corresponding to the target data gap node; A reverse aggregation path is formed based on the correspondence between the target report indicators, indicator dependency nodes, target data gap nodes, data source identifiers, source data tables, and source data fields; Access the corresponding data source according to the reverse aggregation path, and obtain the target source data from the source data field of the source data table.

9. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, Specifically, S6 is: Update the data acquisition status of the target data gap node to the acquired status, and form a data support interval with the same lower and upper limits based on the target source data; The updated data support interval is passed to the target report indicator layer by layer along the reverse dependency relationship. The data support interval of each level indicator dependency node is recalculated to obtain the updated indicator support interval. Obtain the report display precision corresponding to the target report indicator, and process the lower limit and upper limit of the updated indicator support interval according to the report display precision; If the processing result corresponding to the lower limit of the interval is the same as the processing result corresponding to the upper limit of the interval, the target report indicator is determined to have reached the indicator convergence state; if the processing result corresponding to the lower limit of the interval is different from the processing result corresponding to the upper limit of the interval, the target report indicator is determined to have not reached the indicator convergence state.

10. The intelligent report generation method for automatic aggregation of multi-source heterogeneous data according to claim 1, characterized in that, Specifically, S7 is: If the target report indicator has not reached the indicator convergence state, filter the remaining data gap nodes from the data gap nodes whose data acquisition status is not acquired, and recalculate the convergence contribution of the remaining data gap nodes. Select new target data gap nodes based on the recalculated convergence contribution, form a new reverse aggregation path and obtain the corresponding target source data, and repeatedly update the indicator support interval and determine the indicator convergence status. If the target report indicator reaches the indicator convergence state, stop the data collection of the remaining data gap nodes, and calculate the collected data according to the indicator calculation relationship to obtain the report indicator value corresponding to the target report indicator. The report index values ​​are mapped to the report fields in the target report generation task to generate the target report.

Images