An adaptive decision method for file export and related device

Abstract

This invention discloses an adaptive decision-making method and related apparatus for file export, relating to the computer field. The method includes: acquiring export parameter information associated with an export request; processing the export parameter information based on a pre-evaluation engine to determine estimated export information; matching the optimal export mode from a variety of preset export modes based on the estimated export information; executing the file export task according to the optimal export mode; and adjusting the model parameters of the pre-evaluation engine based on the actual execution result of the file export task. This invention processes export parameters through a pre-evaluation engine, allowing users to know the estimated data volume and time consumption in advance, eliminating waiting uncertainty and improving the interactive experience; by matching the optimal export mode adapted to the current system resources and network environment, it solves the problems of high failure rate for large files and redundant waiting for small files; and by dynamically adjusting model parameters based on actual execution results, it achieves self-learning closed-loop optimization, improving export success rate and resource utilization efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to an adaptive decision-making method and related apparatus for file export. Background Technology

[0002] In current Enterprise Resource Planning (ERP) systems, e-commerce back-end systems, and various data analysis platforms, data export functionality is a core tool for users to obtain business reports and conduct offline analysis. With the explosive growth of business data volume and the increasing frequency of concurrent access, the complexity and resource consumption of file export tasks are becoming increasingly significant.

[0003] Existing mainstream technical solutions typically employ two implementation paths: one is a direct submission mechanism based on HyperText Markup Language (HTML) forms, where the front-end encapsulates the filtering conditions and sends them to the server, which then synchronously generates a file and returns a download stream; the other is an asynchronous processing mechanism based on a background task queue, where after the client initiates a request, the server writes the task to a database or message queue, and a separate daemon process executes it sequentially or periodically, notifying the user upon completion. These solutions constitute the basic architecture for data output in current information management software and are widely used in various business scenarios to support data persistence and migration.

[0004] However, existing technologies often lack flexible strategy adjustment mechanisms when facing changing network environments, varying server load states, and data characteristics of different scales. Users cannot know the actual scale and estimated time of the task before initiating an export operation, leading to uncertainty during the waiting process. Furthermore, when encountering situations such as large file transfer timeouts or small file response delays due to inappropriate resource matching, the system struggles to automatically adapt to the optimal processing flow, affecting the stability of the data export process and the smoothness of the user experience. Summary of the Invention

[0005] In view of this, the present invention discloses an adaptive decision-making method and related apparatus for file export, so as to provide a flexible strategy adjustment mechanism for file export when facing changing network environments, different server load states and data characteristics of different magnitudes.

[0006] An adaptive decision-making method for file export, comprising:

[0007] Retrieve the export parameter information associated with the export request;

[0008] The pre-evaluation engine processes the exported parameter information to determine the estimated exported information;

[0009] Based on the estimated export information, the optimal export mode is matched from a variety of preset export modes;

[0010] Perform the file export task according to the optimal export mode described above;

[0011] Based on the actual execution results of the file export task, the model parameters of the pre-evaluation engine are adjusted.

[0012] Optionally, the step of processing the exported parameter information based on the pre-evaluation engine to determine the estimated exported information includes:

[0013] Based on the data filtering conditions configured by the user in the export interface, the database metadata is queried to estimate the number of data rows that meet the data filtering conditions, which is used as the estimated data volume.

[0014] The estimated file size is calculated based on the estimated data volume, the average number of bytes per line, and the compression ratio corresponding to the target file format.

[0015] Based on the estimated data volume, server processing speed, real-time effective transmission bandwidth, fixed overhead, estimated file size, and correction coefficient, the estimated export duration is calculated using a multi-factor duration prediction model.

[0016] The estimated export information includes at least one of the estimated data volume, the estimated file size, and the estimated export duration.

[0017] Optionally, the adaptive decision-making method further includes:

[0018] An environmental adaptation factor is introduced through the pre-evaluation engine, and the environmental adaptation factor is determined based on real-time environmental parameters;

[0019] The estimated export time and / or the estimated file size are corrected using the environment adaptation factor.

[0020] Optionally, the environment adaptation factor includes a server processing capacity correction coefficient and a bandwidth attenuation coefficient, which are determined through the following steps:

[0021] The server processing capacity correction coefficient is determined based on the server load, and the server processing capacity correction coefficient is used to make a weighted correction to the data processing time item in the estimated export duration.

[0022] The bandwidth attenuation coefficient is determined based on the network status, and the bandwidth attenuation coefficient is used to perform weighted correction on the data transmission time item in the estimated derived duration.

[0023] Optionally, adjusting the model parameters of the pre-evaluation engine based on the actual execution results of the file export task includes:

[0024] Calculate the deviation between the actual execution result and the estimated derived information;

[0025] Adjust the correction factor or statistical parameters in the pre-evaluation engine based on the deviation value;

[0026] The correction factor is used to adjust the prediction results for environmental adaptation, and the statistical parameters are used to characterize the execution characteristics of the historical file export task.

[0027] Optionally, the step of matching the optimal export mode from a preset set of multiple export modes based on the estimated export information includes:

[0028] Based on the estimated file size and estimated export time in the estimated export information and the real-time environment parameters in the export parameter information, the optimal export mode is obtained by matching from a variety of preset export modes.

[0029] The various export modes include real-time synchronous export mode, lightweight asynchronous export mode, fragmented parallel export mode, and streaming generation and transmission mode, with different export modes corresponding to different execution strategies.

[0030] Optionally, the exported parameter information includes at least one of basic parameters, real-time environmental parameters, and historical parameters;

[0031] The basic parameters are used to characterize the attributes of the data to be exported;

[0032] The real-time environmental parameters are used to characterize the current system operating status;

[0033] The historical parameters are used to characterize the execution characteristics of the historical file export task.

[0034] An adaptive decision-making system based on file export, comprising:

[0035] The parameter acquisition unit is used to obtain the export parameter information associated with the export request;

[0036] The pre-evaluation engine unit is used to process the exported parameter information based on the pre-evaluation engine to determine the estimated exported information;

[0037] A multi-mode adaptive mode decision unit is used to match the optimal export mode from a variety of preset export modes based on the estimated export information.

[0038] The task execution unit is used to execute the file export task according to the optimal export mode;

[0039] The feedback optimization unit is used to adjust the model parameters of the pre-evaluation engine based on the actual execution results of the file export task.

[0040] A computer storage medium storing at least one instruction that, when executed by a processor, implements the adaptive decision-making method for file export described above.

[0041] An electronic device, comprising: a memory and a processor;

[0042] The memory is used to store at least one instruction;

[0043] The processor is used to execute the at least one instruction to implement the adaptive decision-making method for file export described above.

[0044] As can be seen from the above technical solution, this invention discloses an adaptive decision-making method and related apparatus for file export. It obtains export parameter information associated with the export request, processes the export parameter information based on a pre-evaluation engine to determine estimated export information, matches the optimal export mode from a set of preset export modes based on the estimated export information, executes the file export task according to the optimal export mode, and adjusts the model parameters of the pre-evaluation engine based on the actual execution result of the file export task. This invention, by processing export parameter information through a pre-evaluation engine, allows users to know the estimated data volume and expected time before the export task is executed, eliminating uncertainty during the waiting process and improving interactive transparency and user experience. By matching the optimal export mode from a set of preset export modes that is compatible with the current system resources and network environment, it ensures the best adaptation of the matched export mode to the real-time environment, effectively solving the technical problems of high failure rate for large file exports and redundant waiting for small file responses. By dynamically adjusting the model parameters of the pre-evaluation engine based on the actual execution result, the system can continuously optimize the evaluation accuracy based on real feedback, realizing a self-learning and closed-loop optimization mechanism, thereby improving the success rate of export tasks and the utilization efficiency of system resources. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the published drawings without creative effort.

[0046] Figure 1 This is a flowchart of an adaptive decision-making method for file export disclosed in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of an adaptive decision-making system for file export disclosed in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] This invention discloses an adaptive decision-making method and related apparatus for file export. By processing export parameter information through a pre-evaluation engine, users can know the estimated data volume and expected time before the export task is executed, eliminating uncertainty during the waiting process and improving interactive transparency and user experience. By matching the optimal export mode from a variety of preset export modes to suit the current system resources and network environment, the optimal export mode is ensured to be best adapted to the real-time environment, effectively solving the technical problems of high failure rate for large file exports and redundant waiting for small file responses. By dynamically adjusting the model parameters of the pre-evaluation engine based on actual execution results, the system can continuously optimize the evaluation accuracy based on real feedback, realizing a self-learning and closed-loop optimization mechanism, thereby improving the success rate of export tasks and the utilization efficiency of system resources.

[0051] Explanation of relevant terms:

[0052] APIs (Application Programming Interfaces) act as "bridges" or "messengers" between different software systems, enabling them to communicate, exchange data and functions without needing to know each other's internal implementation details.

[0053] The CPU (Central Processing Unit) is the core component of a computer system, responsible for interpreting computer instructions and processing data in software. It is essentially the "brain" of the computer, performing all calculations, logical judgments, and control operations.

[0054] RTT (Round-Trip Time) refers to the total time it takes for a data packet to travel from the sender to the destination and back.

[0055] See Figure 1 The present invention discloses a flowchart of an adaptive decision-making method for file export, the method comprising:

[0056] Step S101: Obtain the export parameter information associated with the export request.

[0057] Export parameter information refers to the collection of basic configuration data, current system operating environment data, and historical execution characteristic data required to trigger a file export operation. This step aims to provide a comprehensive and accurate input basis for subsequent pre-assessment and decision-making. The sources of export parameter information mainly include data filtering conditions configured by the user in the front-end interface, real-time network status collected by the client through the browser API, and resource load monitored by the server.

[0058] In one embodiment, the export parameter information includes at least one of basic parameters, real-time environment parameters, and historical parameters; wherein, the basic parameters are used to characterize the attributes of the data to be exported; the real-time environment parameters are used to characterize the current system operating status; and the historical parameters are used to characterize the execution characteristics of historical file export tasks.

[0059] Specifically, basic parameters refer to the data filtering conditions and target format settings configured by the user when initiating an export operation on the export page (such as the order management page). These parameters originate from the input of the front-end interactive interface or the API request payload. Basic parameters specifically include data filtering conditions (such as date range "2024-06-01" to "2025-01-31", "Business Type = Completed Order", "Export Fields = Order Number, Payment Amount, Completion Time, Recipient Name, Recipient Address"), target file format (such as Excel, CSV, PDF), and a list of selected fields, such as fields: ["order_id", "amount", "time"] (which determines the number of bytes per row, compression rate, and format fixation overhead, directly affecting file size and estimated duration calculations).

[0060] The role of basic parameters is to anchor the specific range and structural dimensions of the data to be exported, providing a deterministic boundary for subsequent data volume estimation. For example, when a user selects paid orders from June 1, 2024 to January 31, 2025 and specifies the order number, amount, and time fields to be exported, these conditions constitute the basic parameters. Based on these parameters, the system locks specific partitions and column sets in the database, avoiding full table scans.

[0061] Real-time environmental parameters refer to the dynamic operating metrics of the system at the moment an export request is received. These are obtained in real-time through calls to operating system interfaces, network probing tools, or monitoring agents. Specifically, real-time environmental parameters include server-side resource load status (such as CPU utilization, memory usage, and current task queue length) and client-server network communication quality (such as downlink bandwidth, network latency (RTT), and packet loss rate). The purpose of real-time environmental parameters is to quantify the current execution feasibility boundary and dynamically adjust the processing rate and transmission bandwidth variables in the prediction model. For example, if the server CPU utilization is detected to be 85% and the network packet loss rate reaches 5%, the real-time environmental parameters indicate that the current system is in a high-load and weak network state, requiring the triggering of corresponding load reduction or retry mechanisms.

[0062] Historical parameters refer to long-term behavioral characteristic data obtained from the statistics of past file export task logs, which are derived from historical task databases or the training set of machine learning models. Specifically, historical parameters include the historical average export duration distribution of similar business types, the mean and variance of historical file sizes, and the network bandwidth attenuation patterns during specific time periods (such as weekday peak hours). The role of historical parameters is to provide statistical prior support for the pre-evaluation model, especially in scenarios lacking real-time samples or cold start scenarios, providing an initial convergence direction for correction factors. For example, if the system records that the average time for exporting similar order files increases by 30% compared to weekdays during Friday afternoons from 2:00 PM to 4:00 PM, this statistical pattern is stored in the model as a historical parameter to predict potential performance bottlenecks.

[0063] This application achieves comprehensive and refined decision-making by organically combining basic parameters, real-time environmental parameters, and historical parameters. Basic parameters accurately define the physical attributes of the data to be processed, ensuring the accuracy of the predicted object; real-time environmental parameters instantly capture the instantaneous fluctuations in system operation, enabling the prediction model to dynamically adapt to current resource constraints; historical parameters introduce statistical regularities over time, compensating for potential random errors in single sampling. The combined use of these three parameters not only improves the prediction accuracy of the pre-evaluation engine for file size and export duration but also enables the system to automatically adjust evaluation weights in complex scenarios such as high concurrency, weak networks, or large data volumes. This reduces the failure rate of file export tasks, optimizes system resource allocation efficiency, and provides users with a more transparent and controllable export experience.

[0064] Step S102: Process the exported parameter information based on the pre-evaluation engine to determine the estimated exported information.

[0065] The pre-evaluation engine can refer to a processing module with built-in multi-dimensional mathematical models and statistical algorithms, used to transform discrete derived parameters into quantifiable predictive indicators. Predicted export information includes at least one of the following: estimated data volume, estimated file size, and estimated export duration. This step aims to break through the traditional blind execution model and provide users with a perceptible basis for decision-making.

[0066] Step S103: Based on the estimated export information, select the optimal export mode from a variety of preset export modes.

[0067] After obtaining the estimated export information, the system uses it as a basis for decision-making and matches it with preset decision rules. The preset export modes cover different execution strategies, such as the real-time synchronization mode suitable for small-scale data and the sharded parallel mode suitable for large-scale data.

[0068] The matching process is essentially a resource adaptation process. The system selects a mode that can both ensure the successful execution of the task and maximize resource utilization efficiency as the optimal export mode based on the estimated data volume, the duration of the process, and the current environmental resource status.

[0069] For example, if the estimated file size is small and the system load is low, the instant synchronization mode is used to reduce user waiting time; if the estimated file size is large, the sharded parallel mode is used to avoid single-threaded blocking. This adaptive matching avoids mismatches such as "small files causing slow processes" or "large files consuming resources".

[0070] Step S104: Execute the file export task according to the optimal export mode.

[0071] Once the optimal export mode is determined, the system invokes the corresponding executor to perform the specific export actions. During execution, the system processes data, generates files, and transmits them according to the strategy of the selected mode. For example, if the optimal export mode is the split-parallel mode, the system will break down the large task into multiple subtasks for parallel processing, and finally merge them to generate a complete file. The execution phase is the actual implementation stage of the export function, and the diversity of its execution strategies directly improves the system's processing power and robustness.

[0072] Step S105: Based on the actual execution results of the file export task, adjust the model parameters of the pre-evaluation engine.

[0073] This step is crucial for building a closed loop of "evaluation-decision-execution-feedback". After the task is exported, the system does not simply end the process but records the actual execution results, such as the actual file size, actual time taken, and actual transfer rate. The system compares and analyzes the actual execution results with the estimated export information previously output by the pre-evaluation engine, calculating the deviation between the two. If the deviation exceeds a preset range, it indicates that there are errors in the model parameters of the pre-evaluation engine, which need to be corrected. Specifically, the adjustment can be a correction factor in the model or a statistical parameter. For example, if the actual time taken is significantly higher than the estimated time taken, the system can appropriately increase the correction coefficient for processing time in the model to reflect the actual processing capacity of the current environment. Through this feedback adjustment based on actual results, the pre-evaluation engine can continuously optimize itself, improve the accuracy of subsequent predictions, and thus achieve the system's self-learning capability.

[0074] In summary, this invention discloses an adaptive decision-making method for file export. It obtains export parameter information associated with the export request, processes this information using a pre-evaluation engine to determine estimated export information, matches the optimal export mode from a set of preset export modes based on this estimated information, executes the file export task according to the optimal export mode, and adjusts the model parameters of the pre-evaluation engine based on the actual execution results of the file export task. This invention, by processing export parameter information through a pre-evaluation engine, allows users to know the estimated data volume and expected time before the export task is executed, eliminating uncertainty during the waiting process and improving interactive transparency and user experience. By matching the optimal export mode from a set of preset export modes that is compatible with the current system resources and network environment, it ensures the best adaptation of the matched export mode to the real-time environment, effectively solving the technical problems of high failure rate for large file exports and redundant waiting for small file responses. By dynamically adjusting the model parameters of the pre-evaluation engine based on actual execution results, the system can continuously optimize the evaluation accuracy based on real feedback, realizing a self-learning and closed-loop optimization mechanism, thereby improving the success rate of export tasks and the utilization efficiency of system resources.

[0075] In one embodiment, step S102 may specifically include:

[0076] Based on the data filtering conditions configured by the user in the export interface, the database metadata is queried to estimate the number of data rows that meet the data filtering conditions, which is then used as the estimated data volume.

[0077] The estimated file size is calculated based on the estimated data volume, the average number of bytes per line, and the compression ratio corresponding to the target file format.

[0078] Based on the estimated data volume, server processing speed, real-time effective transmission bandwidth, fixed overhead, estimated file size, and correction coefficient, the estimated export duration is calculated using a multi-factor duration prediction model.

[0079] The estimated export information includes at least one of the estimated data volume, the estimated file size, and the estimated export duration.

[0080] The estimated data volume can refer to the number of data rows contained in the file to be exported. It is obtained by querying database metadata or index statistics and is used to characterize the basic scale of the file export task.

[0081] In practical applications, database metadata (such as index statistics) can be queried based on data filtering conditions to quickly estimate the number of data rows that meet the conditions (e.g., the number of rows in a partitioned table associated by date range and business type).

[0082] Estimated file size refers to the space occupied by the generated exported file on the storage medium. It is calculated based on the estimated data volume, the average number of bytes per line, and the compression ratio of the target file format. It is used to directly measure storage resource usage and network transmission load.

[0083] The estimated file size (S) is calculated as follows: number of data rows (Q) × average number of bytes per row × (1 - compression factor corresponding to the target file format).

[0084] The number of data rows (Q) is also the estimated data volume.

[0085] The average number of bytes per line is calculated as follows: The number of uncompressed bytes per line is calculated based on the field type and encoding format. Example: The `order_id` field is a string type; under UTF-8 encoding, each character occupies 1-3 bytes, with an average of 2 bytes, totaling 16 characters and 32 bytes. The `amount` field is a numeric type and occupies 8 bytes, for a total of 40 bytes.

[0086] The encoding format can be the default encoding format for the target file preset by the system, such as UTF-8 / GBK.

[0087] Format compression ratio R: Different file formats correspond to different compression ratios. Among them, the default compression ratio of Excel format is about 30%-50%; CSV format has no compression; PDF format has an additional 20%-40% added to the result calculated by the above formula due to the high proportion of layout and metadata.

[0088] Estimated export time refers to the estimated time required to complete the entire process from data reading and format conversion to file transfer. It is calculated based on a multi-factor model built on server processing capacity, effective transmission bandwidth, and fixed overhead, and is used as a core indicator of user experience and a key threshold for switching export modes.

[0089] The pre-evaluation engine internally incorporates a duration prediction model, which is not a simple linear calculation but rather integrates a dynamic correction mechanism. As an example, the estimated derived duration T can be calculated using the following formula:

[0090] T = (Q / V) × Kv + [S / (B × Kb)] + C;

[0091] In the formula, Q represents the number of data rows, which is also the estimated data volume;

[0092] V represents the server processing speed, which can be the single-threaded data processing rate for different business types based on historical task statistics. For example, "order data" V = 1000 lines / second, and "log data" V = 5000 lines / second.

[0093] S represents the estimated file size in MB. The formula for calculating the estimated file size is S = Q × number of bytes per line × (1 − R), where R is the format compression ratio.

[0094] B represents the effective transmission bandwidth (MB / s: megabytes per second), which can be specifically taken as 80% of the real-time network downlink bandwidth (with 20% reserved for fluctuation resistance).

[0095] C represents fixed system overhead, such as the fixed time consumed by parameter validation, file initialization, and format conversion (e.g., generating the header of an Excel file takes approximately 200ms). Fixed system overhead does not need to be collected in real time and is considered a pre-configured basic parameter of the system.

[0096] Kv is the server processing capacity correction factor, which can be used to reflect factors affecting processing speed such as CPU load and IO bottlenecks. The initial value is 1.0.

[0097] Kb is the bandwidth attenuation coefficient, used to reflect the impact of network fluctuations, packet loss, latency, etc. on the effective transmission bandwidth. The initial value is 1.0.

[0098] In this formula, Kv and Kb are environmental adaptation factors. Kv and Kb are not fixed constants, but variables that fluctuate dynamically according to real-time environmental parameters. By introducing environmental adaptation factors, the pre-evaluation model can map the "instantaneous state" of the current system to the prediction results, thereby correcting the bias caused by the static model ignoring environmental fluctuations.

[0099] In one embodiment, the adaptive decision-making method further includes:

[0100] An environmental adaptation factor is introduced through the pre-evaluation engine, and the environmental adaptation factor is determined based on real-time environmental parameters;

[0101] The estimated export time and / or the estimated file size are corrected using the environment adaptation factor.

[0102] To further improve the accuracy of the estimated export information, especially the reliability of the estimated export duration, the pre-evaluation engine in this embodiment also introduces an environment adaptation factor. This environment adaptation factor is determined based on real-time environment parameters and is used to correct the estimated export duration and / or estimated file size.

[0103] By introducing an environmental adaptation factor, the pre-evaluation model can map the "instantaneous state" of the current system to the prediction results, thereby correcting the bias caused by the static model ignoring environmental fluctuations.

[0104] In one embodiment, the environment adaptation factor includes a server processing capacity correction coefficient and a bandwidth attenuation coefficient, which are determined through the following steps:

[0105] The server processing capacity correction coefficient is determined based on the server load, and the server processing capacity correction coefficient is used to make a weighted correction to the data processing time item in the estimated export duration.

[0106] The bandwidth attenuation coefficient is determined based on the network status, and the bandwidth attenuation coefficient is used to perform weighted correction on the data transmission time item in the estimated derived duration.

[0107] Specifically, when the server load is high (e.g., CPU utilization exceeds a preset threshold), the server's actual processing capacity decreases. In this case, the server processing capacity correction coefficient Kv will be greater than 1, indicating that data processing time needs to be increased. For example, Kv can be linearly or non-linearly mapped based on CPU utilization. If CPU utilization is 60%, Kv can be set to 1.0 (the baseline value); if CPU utilization rises to 80%, Kv can be adjusted to 1.2, meaning that data processing time will increase by 20% based on the baseline calculation. Conversely, if the server is idle, Kv can be less than 1 to fully utilize idle resources. This dynamic adjustment mechanism ensures that the pre-assessment results accurately reflect the server's current computing power, avoiding task backlog or crashes caused by server overload.

[0108] Network conditions include metrics such as network latency (RTT) and packet loss rate. When network conditions are poor (e.g., high latency or high packet loss rate), the effective transmission bandwidth will be significantly lower than the theoretical bandwidth. In this case, the bandwidth attenuation coefficient Kb will be less than 1, indicating that the actual available bandwidth needs to be discounted during calculation. For example, if high network latency is detected, Kb can be attenuated based on the latency value. If Kb is 1.0 when the latency is 50ms, and drops to 0.7 when the latency rises to 300ms, it means that the transmission bandwidth is calculated at 70%, thus extending the estimated transmission time. By correcting the bandwidth attenuation coefficient, the pre-evaluation model can accurately predict the derived time under fluctuating network conditions, avoiding transmission timeouts caused by network congestion.

[0109] This embodiment constructs a pre-assessment model with "environmental awareness" capabilities through the aforementioned multi-dimensional parameter collection and dynamic correction of environmental adaptation factors. This model not only considers the static characteristics of the data itself but also incorporates the dynamic impact of server load and network conditions in real time. This makes the predicted information more closely reflect the actual execution environment, providing accurate data support for subsequent adaptive decision-making and effectively solving the problems of large prediction deviations and severe resource mismatches caused by ignoring environmental factors in existing technologies.

[0110] In one embodiment, step S105 may specifically include:

[0111] Calculate the deviation between the actual execution result and the estimated derived information;

[0112] Adjust the correction factor or statistical parameters in the pre-evaluation engine based on the deviation value;

[0113] The correction factor is used to adjust the prediction results for environmental adaptation, and the statistical parameters are used to characterize the execution characteristics of the historical file export task.

[0114] Specifically, after the export task is completed, the system does not immediately release resources, but triggers a feedback optimization process. First, the system records the actual execution result of this export task, which includes at least the actual data volume, actual file size, actual export duration, and actual environmental parameters (such as actual transmission bandwidth, actual server load, etc.). Then, the system compares the actual execution result with the estimated export information previously output by the pre-evaluation engine and calculates the deviation between the two.

[0115] The purpose of calculating the deviation value is to quantify the accuracy of the pre-evaluation model. For example, if the estimated export duration is T_est and the actual export duration is T_act, then the duration deviation value ΔT = T_act - T_est. Similarly, the file size deviation value ΔS = S_act - S_est can also be calculated. The system presets a deviation threshold range. If the deviation value exceeds this preset deviation threshold range, it is determined that the parameters of the pre-evaluation model have room for optimization and need to be corrected.

[0116] After calculating the deviation value, the system adjusts the correction factor or statistical parameters in the pre-evaluation engine based on the deviation value.

[0117] Among them, the correction factor is used to adjust the prediction results for environmental adaptation, such as the server processing capacity correction coefficient Kv and bandwidth attenuation coefficient Kb mentioned in the previous embodiment; the statistical parameters are used to characterize the execution characteristics of historical exported tasks, such as compression ratio coefficient R, baseline processing rate V, etc.

[0118] As a specific implementation method, the adjustment logic for the correction factor is as follows:

[0119] If the actual data processing time during the export process is significantly longer than the estimated time, it indicates that the current server processing capacity correction factor Kv is set too low and fails to accurately reflect the server's load. In this case, the system will positively correct Kv based on the deviation. For example, if the actual data processing time is 20% longer than the estimated time, the system can increase the value of Kv by a certain percentage (e.g., by 0.1) so that the pre-evaluation engine can output a longer estimated duration when encountering similar load conditions in the future, thereby avoiding the risk of timeouts due to insufficient estimation. Conversely, if the actual time is significantly shorter than the estimated time, Kv will be corrected in the opposite direction. Similarly, if the data transmission time deviation is large, a similar correction operation will be performed on the bandwidth attenuation factor Kb.

[0120] As another specific implementation method, the adjustment logic for the statistical parameters is as follows:

[0121] Statistical parameters are typically derived from the statistical averages of a large number of historical tasks, such as the file compression ratio R. During the export process, the compression characteristics of data from different business types may vary. If there is a significant deviation between the actual file size S_act and the estimated file size S_est, the system will recalculate the actual compression ratio R_act for this task and include it in the historical statistical database. The system can update the compression ratio R using a sliding window average or a weighted average method. For example, the system can assign a higher weight (e.g., 30%) to the latest actual compression ratio and combine it with historical compression ratios to calculate a new compression ratio R_new. In this way, the pre-evaluation engine can continuously learn the characteristics of different business data, making subsequent file size predictions more accurate.

[0122] This embodiment constructs a closed-loop system of "evaluation-execution-feedback-correction" through the aforementioned feedback optimization mechanism. Each completed export task serves as a training exercise for the pre-evaluation model. As the system's runtime increases, the model parameters continuously approach the optimal values ​​of the real environment, thereby achieving self-learning and continuous optimization of the system and effectively solving the long-term prediction bias problem caused by the solidification of model parameters in existing technologies.

[0123] In one embodiment, step S103 may specifically include:

[0124] Based on the estimated file size and estimated export time in the estimated export information and the real-time environment parameters in the export parameter information, the optimal export mode is obtained by matching from a variety of preset export modes.

[0125] The various export modes include real-time synchronous export mode, lightweight asynchronous export mode, fragmented parallel export mode, and streaming generation and transmission mode. Different export modes correspond to different execution strategies to adapt to data processing needs of different scales and network transmission environments of different quality.

[0126] Specifically, the real-time synchronous export mode is suitable for scenarios with small data sizes (i.e., small files) and sufficient system resources (good network, idle server). When the estimated file size is less than or equal to a first preset threshold (e.g., 10MB), and the estimated export time is less than or equal to a first duration threshold (e.g., 5 seconds), while real-time environmental parameters show that the network bandwidth is greater than or equal to a first bandwidth threshold (e.g., 10Mbps) and the server load is less than a first load threshold (e.g., 60%), the system determines the current task as a lightweight task and matches it to the synchronous export mode. In this mode, the system uses a synchronous processing method. The front end directly submits the request through a form, and the back end instantly generates the file and returns a download link. Users do not need to wait too long to obtain the file, resulting in a smooth interactive experience. It should be understood that the above thresholds are only examples of preferred implementation methods, and can be adjusted according to system performance in actual applications.

[0127] The lightweight asynchronous export mode is suitable for scenarios with medium-sized data (i.e., medium-sized files) or moderate system load (stable network, light server load). When the estimated file size is between a first and a second preset threshold (e.g., greater than 10MB and less than or equal to 100MB), or the estimated export duration is between a first and a second duration threshold (e.g., greater than 5 seconds and less than or equal to 60 seconds), and the network bandwidth is greater than or equal to the second bandwidth threshold (e.g., 5Mbps) and the server load is less than the second load threshold (e.g., 70%), the system matches the asynchronous export mode. In this mode, the system writes the export task to a background task table and processes it asynchronously using a low-priority thread to avoid blocking the main thread. After the task is completed, the system notifies the user to download via push notification or front-end polling. This mode effectively balances user waiting time and system resource consumption, and is suitable for most regular export needs.

[0128] The sharded parallel export mode is suitable for scenarios with massive data volumes (i.e., large files) and systems with parallel processing capabilities (high bandwidth, multi-core available servers). When the estimated file size exceeds a second preset threshold (e.g., 100MB), network bandwidth is greater than or equal to a third bandwidth threshold (e.g., 20Mbps), the number of server CPU cores is greater than or equal to a preset number (e.g., 4 cores), and the data has divisible attributes (e.g., sharded by date or ID range), the decision module matches the sharded parallel export mode. In this mode, the execution engine breaks down the large task into multiple subtasks, processes each subtask in parallel using multi-threading, and finally merges the generated sub-files into a complete file. This strategy fully utilizes the server's multi-core computing resources and high-bandwidth network environment, significantly reducing the export time of large files and avoiding timeout issues caused by single-threaded processing.

[0129] The streaming, simultaneous generation and transmission mode is suitable for scenarios with extremely large data volumes (i.e., very large files) or unstable network environments (i.e., large network fluctuations). When the estimated file size exceeds a third preset threshold (e.g., 1GB), or real-time environmental parameters show a network packet loss rate exceeding a preset threshold (e.g., 10%), or the estimated export duration exceeds a third duration threshold (e.g., 300 seconds), the system switches to streaming export mode. In this mode, the system uses streaming processing technology to compress or convert data while reading it, and then streams it to the front end over the network. The front end receives and buffers the data simultaneously. This mode eliminates the need to load all data on the server side, significantly reducing memory usage, and also adapts to fluctuating network environments, avoiding full retransmissions due to network interruptions.

[0130] This embodiment achieves adaptive decision-making for the derived mode through the aforementioned multi-dimensional matching logic. The system no longer mechanically executes a single strategy, but dynamically selects the most suitable execution plan based on the "urgency" of the tasks and the "quality" of the environment. This mechanism effectively solves the resource waste caused by "slow processes for small files" and the system crashes caused by "resource overload for large files" in existing technologies, maximizing system throughput and stability while ensuring a good user experience.

[0131] Corresponding to the above method embodiments, the present invention also discloses an adaptive decision-making system for file export.

[0132] See Figure 2 The present invention discloses a schematic diagram of the structure of a file export adaptive decision system. The adaptive decision system is located in the back-end server layer and includes: a parameter acquisition unit 201, a pre-evaluation engine unit 202, a multi-mode adaptive mode decision unit 203, a task execution unit 204, and a feedback optimization unit 205.

[0133] The parameter acquisition unit 201 is used to acquire the export parameter information associated with the export request.

[0134] The exported parameter information includes at least one of the following: basic parameters, real-time environment parameters, and historical parameters.

[0135] Specifically, the parameter acquisition unit 201 is used to obtain basic parameters such as data filtering conditions, target file format and field list configured by the user in the front-end interface from the export trigger module located in the front-end interaction layer.

[0136] The parameter acquisition unit 201 is also used to acquire real-time environmental parameters from the environment awareness module located in the front-end interaction layer. The real-time environmental parameters include network parameters and server parameters. The network parameters include the current network bandwidth, network latency, packet loss rate, etc. The server parameters include server CPU utilization, memory usage, and other operating status information.

[0137] The parameter acquisition unit 201 is also used to acquire historical parameters from the business database located in the data storage layer.

[0138] It should be noted that the collection frequency and timing of the parameter acquisition unit 201 can be configured according to actual business needs. For example, collection can be triggered when the user clicks the "Export" button, or environmental parameters can be collected periodically according to a preset cycle to reduce real-time calculation delay.

[0139] The parameter acquisition unit 201 sends the acquired exported parameter information to the pre-evaluation engine unit 202 for processing.

[0140] The pre-evaluation engine unit 202 is used to process the exported parameter information based on the pre-evaluation engine to determine the estimated exported information.

[0141] The pre-evaluation engine unit 202 is the core computing unit of the system, which encapsulates the pre-evaluation model. The pre-evaluation engine unit 202 receives the derived parameter information sent by the parameter acquisition unit 201 and performs parsing and feature extraction.

[0142] Specifically, the pre-evaluation engine unit 202 queries metadata from the business database based on data filtering conditions, evaluates the number of data rows that meet the conditions, and uses the number of data rows as the estimated data volume; it calculates the estimated file size based on the estimated data volume, the average number of bytes per row, and the compression ratio coefficient corresponding to the target file format; it calculates the estimated export duration through a multi-factor duration prediction model based on the estimated data volume, server processing speed, real-time effective transmission bandwidth, fixed overhead, estimated file size, and correction coefficient; it introduces an environment adaptation factor through the pre-evaluation engine; it uses the environment adaptation factor to correct the estimated export duration and / or estimated file size, and outputs estimated export information including at least one of the estimated data volume, estimated file size, and estimated export duration to the mode decision unit 203.

[0143] The pre-evaluation display module, located in the front-end interaction layer, can also obtain the required data from the business database.

[0144] The mode decision unit 203 is used to match the optimal export mode from a variety of preset export modes based on the estimated export information.

[0145] The mode decision unit 203 is the system's strategy hub, internally containing multiple export modes and their corresponding matching rule bases. The mode decision unit 203 receives the estimated export information output by the pre-evaluation engine unit 202 and, combined with real-time environmental parameters, performs multi-dimensional logical judgments. Specifically, the mode decision unit 203 compares indicators such as estimated file size and estimated export duration with preset threshold conditions. For example, if the estimated file size is less than a first preset threshold and the estimated export duration is less than a first duration threshold, then the instant synchronous export mode is matched; if the estimated file size is greater than a second preset threshold and the network bandwidth meets the conditions, then the fragmented parallel export mode is matched. The mode decision unit 203, through table lookup or rule engine technology, selects the optimal export mode most suitable for the current scenario from the preset instant synchronous export mode, lightweight asynchronous export mode, fragmented parallel export mode, and streaming edge-generation-and-transmission mode, and generates corresponding execution instructions to send to the task execution unit 204. The existence of the pattern decision unit 203 enables the system to make adaptive decisions based on the "priority" of tasks and the "quality" of the environment, thus avoiding resource misallocation caused by a single strategy.

[0146] The task execution unit 204 is used to execute the file export task according to the optimal export mode.

[0147] The task execution unit 204 is the system's execution mechanism, responsible for specific data processing and file generation. Internally, the task execution unit 204 is equipped with various executors, including a synchronous export executor, an asynchronous task executor, a sharded parallel executor, and a streaming generation-while-transmitting executor, corresponding to different export modes issued by the mode decision unit 203. Upon receiving an execution command from the mode decision unit 203, the task execution unit 204 calls the corresponding executor to execute the export task. For example, if the optimal export mode is the sharded parallel export mode, the task execution unit 204 splits the large task into multiple subtasks, allocates them to multi-threaded parallel processing, and merges them to generate a complete file after processing. During execution, the task execution unit 204 is also responsible for monitoring the task progress and pushing the progress data to the front end for display. The task execution unit 204 records and transmits the actual execution results of the export task (such as actual file size, actual time consumption, etc.) to the feedback optimization unit 205.

[0148] Among them, the synchronous export executor corresponds to the execution flow of the "real-time synchronous export" mode. Its core function is to receive the synchronous export instruction, directly call the business query interface to pull the full amount of data, generate the target format file through libraries such as Apache POI, and return the download link to the front end directly after generation, blocking the current request until completion.

[0149] The asynchronous task executor corresponds to the execution flow of the "lightweight asynchronous export" mode. Its core function is to receive the asynchronous export instruction, write the export parameters into the Task table, put them into the low-priority task queue, and schedule them in order by the background thread pool. After completion, it notifies the user to download via message push / frontend polling.

[0150] The sharded parallel executor corresponds to the execution flow of the "sharded parallel export" mode. Its core functions are: to support splitting large tasks into multiple independent subtasks according to rules such as time and ID hashing, to schedule multi-threaded parallel processing, and to have built-in logic for subtask failure retry, progress summary, and result merging, and finally output a complete file.

[0151] The streaming-based generation and transmission executor corresponds to the execution flow of the "streaming-based generation and transmission" mode. Its core function is to adopt a streaming response mechanism (such as Spring's StreamingResponseBody), which eliminates the need to load the entire file into the server memory. It reads data from the database while performing format conversion and compression, and pushes the data stream to the front end, adapting to scenarios of exporting extremely large files.

[0152] Additionally, the task execution unit 204 can store task records / progress to an exported task database in the bit data storage layer. This exported task database can also route the task to the export center module located in the front-end interaction layer for user viewing. Furthermore, the export center module can send download requests to each executor to obtain the required download data from the executor.

[0153] The feedback optimization unit 205 is used to adjust the model parameters of the pre-evaluation engine based on the actual execution results of the file export task.

[0154] The feedback optimization unit 205 is the self-learning core of the system, used to continuously optimize the pre-evaluation model. The feedback optimization unit 205 receives the actual execution results sent by the task execution unit 204 and compares and analyzes them with the predicted output information previously output by the pre-evaluation engine unit 202.

[0155] Specifically, the feedback optimization unit 205 calculates the deviation between the actual value and the estimated value; based on the magnitude and direction of the deviation, it reverse-corrects the model parameters in the pre-evaluation engine unit 202, and can write the corrected parameters into the model parameter library, or update the corresponding parameters in the model parameter library.

[0156] For example, if the actual transmission time is significantly higher than the estimated transmission time, the parameter correction subunit will increase the bandwidth attenuation coefficient Kb in the pre-evaluation engine unit 202; if the actual file size deviates significantly from the estimated file size, the compression ratio coefficient R in the historical statistical parameters will be updated. The feedback optimization unit 205 writes the corrected parameters back to the pre-evaluation engine unit 202, thereby updating the pre-evaluation model and making the next pre-evaluation more accurate. Through this closed-loop mechanism of execution, feedback, and correction, the system has the ability to adapt to changes in the environment, and the long-term prediction accuracy is improved.

[0157] Corresponding to the above embodiments, the present invention also discloses a computer storage medium that stores at least one instruction, which, when executed by a processor, implements the steps shown in the embodiments of the adaptive decision-making method for file export.

[0158] Corresponding to the above embodiments, such as Figure 3 As shown, the present invention also provides a schematic diagram of the structure of an electronic device, which may include: a processor 1 and a memory 2;

[0159] The processor 1 and memory 2 communicate with each other via communication bus 3.

[0160] Processor 1, for executing at least one instruction;

[0161] Memory 2 is used to store at least one instruction;

[0162] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0163] Memory 2 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0164] The processor executes at least one instruction to implement the steps shown in the embodiment of the adaptive decision-making method for file export.

[0165] To more clearly illustrate the beneficial effects of the technical solution of this invention, this embodiment combines a specific application scenario to provide a detailed explanation of the entire process of the adaptive decision-making method for file export. This embodiment takes exporting "2023 full-year log data" from a data analysis platform as an example to demonstrate the complete closed-loop process from parameter collection, pre-evaluation calculation, decision matching, execution monitoring to feedback optimization.

[0166] Example 1:

[0167] I. Exporting parameters and collecting context (front-end → back-end);

[0168] Users set filter criteria (such as "2025-01-01 to 2026-01-01", "Business Type = Paid Orders", "Export Fields = Order Number, Amount, Time") on the business page (such as the "Order Management Page"), and click the "Export File" button (binding the onExportClick event).

[0169] Front-end actions:

[0170] 1. Input export filter criteria ({dateRange: ["2024-01-01", "2024-01-31"], bizType:"pay_order", fields: ["order_id", "amount", "time"]}) and target format (fileFormat: "xlsx");

[0171] 2. Call the environment awareness module: Obtain bandwidth (in Mbps) through navigator.connection.downlink and calculate network latency (RTT) through performance.timing.responseStart;

[0172] 3. Send a POST request to the / api / export / pre-assess interface, carrying the above parameters.

[0173] Backend actions:

[0174] 1. Parse request parameters and verify their validity (e.g., the date range is not empty, and the business type exists in the whitelist);

[0175] 2. Call the business database metadata interface:

[0176] 2.1 Example (MySQL): Executing SELECT COUNT(*) FROM orders WHERE order_timeBETWEEN '2025-01-01' AND '2026-01-01' AND biz_type='pay_order', a quick estimate of the number of data rows is Q≈120000;

[0177] 2.2 If the data volume is too large (e.g., >1 million rows), use sampling estimation: randomly sample 0.1% of the data to calculate the average number of bytes per row (e.g., avgRowSize=200 bytes), then Q≈total data volume × 0.1% (obtain the total number of rows in the table by SHOW TABLE STATUS LIKE 'orders').

[0178] 3. Pull server load: Call the Prometheus API to get the current CPU usage (cpu_usage=55%), memory usage (mem_usage=62%), and export the task queue length (queue_len=3).

[0179] 4. Return the collected results to the front end and trigger the pre-evaluation engine.

[0180] II. Export the pre-evaluation engine calculations;

[0181] After receiving the collected data, the pre-evaluation engine performs calculations in three stages:

[0182] Phase 1: Pre-calculation of data volume and file size;

[0183] Data volume confirmation: Based on the database query results, Q = 12,000 rows (exact value);

[0184] Average byte count per row: estimated based on the exported field type (e.g., order_id is a string, 16 bytes; amount is a floating-point number, 8 bytes; time is a timestamp, 8 bytes), totaling 16+8+8=32 bytes (uncompressed).

[0185] Compression ratio correction: Excel's compression ratio for text data is approximately 40% (empirical value), therefore, the number of bytes per row after compression = 32 × (1 - 40%) = 19.2 bytes;

[0186] Estimated file size: S = Q × number of bytes per line after compression = 12000 × 19.2 = 2,304,000 bytes ≈ 0.22MB (Note: The actual Excel file size will increase by about 10% due to formatting and metadata, so the final estimated size is approximately 0.24MB).

[0187] Phase Two: Modeling the Estimated Export Duration;

[0188] The formula used is: T=(Q / V)×Kv+[S / (B×Kb)]+C where:

[0189] Server processing capacity V: For querying historical task data, the single-threaded processing rate for the "Payment Order" type is V = 2000 rows / second (historical average).

[0190] Effective transmission bandwidth B: Front-end reporting bandwidth downlink = 10Mbps, take 80% effective bandwidth = 8Mbps = 1MB / s (1Byte = 8bit);

[0191] Fixed overhead C: Time spent generating the Excel file header + format conversion; empirical value C = 0.5 seconds.

[0192] Kv: Server processing capacity correction coefficient, initial value 1.0, the coefficient increases with higher CPU utilization. Assuming server CPU utilization = 60% and kv = 1.0, the coefficient increases by 0.01 for every 1% increase.

[0193] Kb: Bandwidth attenuation factor, initial value 1.0, the factor decreases as network latency increases. Calculated based on Kb=1.0−β above;

[0194] Substitute into the calculation:

[0195] Data processing time: Q / V×Kv=12000 / 2000×1.1=6.6 seconds;

[0196] Transmission time: S / (B×kb) = 0.24MB / 0.6 = 0.4 seconds;

[0197] Total estimated duration: T = 6.6 + 0.4 + 0.5 ≈ 7.5 seconds.

[0198] Phase 3: Environmental Adaptation Factor Correction;

[0199] Network conditions: Front-end reporting latency RTT = 50ms (normal), packet loss rate = 0% (ideal), no attenuation required, B remains at 1MB / s.

[0200] Server load: CPU utilization 55% < 60%, no need to reduce load, V maintain 2000 lines / second.

[0201] Final estimated results: File size ≈ 0.24MB, data volume = 12000 rows, estimated duration ≈ 6.24 seconds.

[0202] III. Multi-modal adaptive decision-making;

[0203] Pre-assessment result matching mode: if the estimated file size is 2.4MB ≤ 10MB and the estimated duration is 9 seconds or greater than or equal to 5 seconds, then adjust to a more reasonable threshold (e.g., "≤ 30 seconds" for instant synchronization). Assuming the threshold is "≤ 30 seconds", then it is satisfied.

[0204] Network bandwidth ≥ 10Mbps, server CPU usage 55% < 60%;

[0205] Decision-making mode: Real-time synchronous export.

[0206] IV. Dynamic Execution;

[0207] Real-time synchronous export process:

[0208] The backend calls the synchronous export executor, queries the business database based on the filtering conditions, and generates an Excel file (using the Apache POI library).

[0209] After the file is generated, it is stored in a temporary directory (e.g., / tmp / export / 20240520123456_abc123.xlsx).

[0210] Generate a download link (e.g., https: / / domain.com / api / export / download?taskId=abc123) and return it directly to the front end.

[0211] V. Front-end feedback and interaction;

[0212] The pre-assessment pop-up window displays: "Estimated file size 2.4MB, data volume 12,000 lines, estimated duration 9 seconds, 'Instant Synchronous Export' recommended (good network + idle server)";

[0213] When the user clicks "Confirm Export", the pop-up window is hidden on the front end, and "Exporting..." is displayed.

[0214] After the backend returns the download link, the frontend automatically triggers the download (or displays the link through the "Export Center").

[0215] Example 2:

[0216] Complex scenario (parallel export of large file fragments): Export "2023 full year log data" (estimated file size 500MB, data volume 10 million rows).

[0217] 1. Data collection and pre-assessment:

[0218] Data volume Q = 10 million rows, average number of bytes per row = 150 bytes (including log content), compression rate = 50%, estimated file size S = 10 million × 150 × 0.5 = 750,000,000 bytes ≈ 715MB.

[0219] Server processing capacity V = 5000 lines / second (log data is easily processed in parallel), effective bandwidth B = 20Mbps = 2.5MB / s (users use 5G network).

[0220] Estimated duration T = (10 million / 5000) + (715 MB / 2.5 MB / s) + 1 second = 2000 + 286 + 1 = 2287 seconds ≈ 38 minutes.

[0221] Environment: Server CPU = 40% (4 cores), data can be sharded by date (365 days → 36 sub-tasks, 10 days / sub-task).

[0222] 2. Decision-making model:

[0223] The system meets the requirements of "estimated file size > 100MB, bandwidth ≥ 20Mbps, 4-core server, and data splitting", and the decision mode is: parallel export in shards.

[0224] 3. Execution process:

[0225] The backend breaks down the task into 36 subtasks (such as "2023-01-01 to 2023-01-10", "2023-01-11 to 2023-01-20", etc.), each subtask containing independent filtering conditions.

[0226] Start 36 threads to process the subtask in parallel (9 threads per core to avoid overload), and each thread generates a sub-file (such as subtask_001.csv).

[0227] After all subtasks are completed, call the merger (such as Python pandas.concat or Java CSV Merger) to merge them into a complete file 2023_logs.xlsx.

[0228] Store the file and generate a download link, then push it to the front-end "Export Center".

[0229] 4. Exception handling:

[0230] If a subtask fails due to network fluctuations (e.g., the 15th subtask times out), the feedback optimization module will trigger a retry:

[0231] First retry: after a 10-second interval, allocate a higher-priority thread separately;

[0232] Second retry: If it still fails, check if there are any abnormalities in the data during that time period (such as a sudden increase in log volume on a certain day), adjust the subtask granularity (split into 5 days / subtasks) and then retry;

[0233] If the retry is successful, the merging process continues; if it fails, the task status is marked as "partially completed," and the user is prompted with "15% of data export failed. Do you want to retry the failed fragment?"

[0234] Example 3:

[0235] During the model's self-learning optimization process, the system records the following after each export:

[0236] Actual file size S_actual = 700MB (deviation from estimated 715MB -2.1%).

[0237] Actual duration T_actual = 35 minutes (deviation from estimated 38 minutes -7.9%).

[0238] Time spent on each step: data processing 28 minutes (80%), transmission 6 minutes (17%), merging 1 minute (3%).

[0239] The feedback optimization module calculates the deviation value and updates the pre-evaluation model parameters:

[0240] Compression ratio correction: Original assumption 50%, actual compression ratio = 10 million × 150 / (700MB × 1024 × 1024) ≈ 10 million × 150 / 734003200 ≈ 20.4 → Compression ratio = 1 - (700MB × 1024 × 1024) / (10 million × 150) ≈ 1 - 734003200 / 1500000000 ≈ 51% (fine-tuning);

[0241] Bandwidth attenuation factor: If the actual transmission time is shorter than the estimate, it may be because the actual bandwidth of the 5G network is higher than the reported value (e.g., reported 10Mbps, actual 15Mbps). The bandwidth for the next similar scenario will be adjusted by downlink × 1.2.

[0242] In summary, this invention breaks through the "blind execution" mode of traditional export tasks where users cannot predict the execution result. By performing multi-dimensional pre-calculations on the amount of data to be exported, estimated file size, export duration, and current environmental parameters, it provides users with perceptible decision-making basis. Based on the pre-assessment results, this invention intelligently matches the optimal export mode among real-time synchronous mode, lightweight asynchronous mode, fragmented parallel mode, or streaming generation and transmission mode, effectively solving the technical problems of timeouts for large file exports and redundant waiting for small file responses. Simultaneously, this invention dynamically adjusts the export strategy based on real-time network status and server load, avoiding resource mismatch and improving the export success rate by 30% to 50%. Furthermore, through pre-assessment pop-ups, real-time progress monitoring, and self-learning optimization mechanisms, this invention can reduce user anxiety during the waiting process, improving the controllability of the export experience and user satisfaction.

[0243] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0244] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0245] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An adaptive decision-making method for file export, characterized in that, include: Retrieve the export parameter information associated with the export request; The pre-evaluation engine processes the exported parameter information to determine the estimated exported information; Based on the estimated export information, the optimal export mode is matched from a variety of preset export modes; Perform the file export task according to the optimal export mode described above; Based on the actual execution results of the file export task, the model parameters of the pre-evaluation engine are adjusted.

2. The adaptive decision-making method according to claim 1, characterized in that, The process of processing the exported parameter information based on the pre-evaluation engine to determine the estimated exported information includes: Based on the data filtering conditions configured by the user in the export interface, the database metadata is queried to estimate the number of data rows that meet the data filtering conditions, which is used as the estimated data volume. The estimated file size is calculated based on the estimated data volume, the average number of bytes per line, and the compression ratio corresponding to the target file format. Based on the estimated data volume, server processing speed, real-time effective transmission bandwidth, fixed overhead, estimated file size, and correction coefficient, the estimated export duration is calculated using a multi-factor duration prediction model. The estimated export information includes at least one of the estimated data volume, the estimated file size, and the estimated export duration.

3. The adaptive decision-making method according to claim 2, characterized in that, The adaptive decision-making method further includes: An environmental adaptation factor is introduced through the pre-evaluation engine, and the environmental adaptation factor is determined based on real-time environmental parameters; The estimated export time and / or the estimated file size are corrected using the environment adaptation factor.

4. The adaptive decision-making method according to claim 3, characterized in that, The environmental adaptation factors include server processing capacity correction coefficients and bandwidth attenuation coefficients, which are determined through the following steps: The server processing capacity correction coefficient is determined based on the server load, and the server processing capacity correction coefficient is used to make a weighted correction to the data processing time item in the estimated export duration. The bandwidth attenuation coefficient is determined based on the network status, and the bandwidth attenuation coefficient is used to perform weighted correction on the data transmission time item in the estimated derived duration.

5. The adaptive decision-making method according to claim 1, characterized in that, The adjustment of the model parameters of the pre-evaluation engine based on the actual execution results of the file export task includes: Calculate the deviation between the actual execution result and the estimated derived information; Adjust the correction factor or statistical parameters in the pre-evaluation engine based on the deviation value; The correction factor is used to adjust the prediction results for environmental adaptation, and the statistical parameters are used to characterize the execution characteristics of the historical file export task.

6. The adaptive decision-making method according to claim 1, characterized in that, The step of matching the optimal export mode from a set of preset export modes based on the estimated export information includes: Based on the estimated file size and estimated export time in the estimated export information and the real-time environment parameters in the export parameter information, the optimal export mode is obtained by matching from a variety of preset export modes. The various export modes include real-time synchronous export mode, lightweight asynchronous export mode, fragmented parallel export mode, and streaming generation and transmission mode, with different export modes corresponding to different execution strategies.

7. The adaptive decision-making method according to any one of claims 1 to 6, characterized in that, The exported parameter information includes at least one of basic parameters, real-time environment parameters, and historical parameters; The basic parameters are used to characterize the attributes of the data to be exported; The real-time environmental parameters are used to characterize the current system operating status; The historical parameters are used to characterize the execution characteristics of the historical file export task.

8. An adaptive decision-making system for file export, characterized in that, include: The parameter acquisition unit is used to obtain the export parameter information associated with the export request; The pre-evaluation engine unit is used to process the exported parameter information based on the pre-evaluation engine to determine the estimated exported information; A multi-mode adaptive mode decision unit is used to match the optimal export mode from a variety of preset export modes based on the estimated export information. The task execution unit is used to execute the file export task according to the optimal export mode; The feedback optimization unit is used to adjust the model parameters of the pre-evaluation engine based on the actual execution results of the file export task.

9. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction, which, when executed by a processor, implements the adaptive decision-making method for file export as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The electronic device includes: a memory and a processor; The memory is used to store at least one instruction; The processor is used to execute the at least one instruction to implement the adaptive decision-making method for file export as described in any one of claims 1 to 7.

Images