Cloud document collaborative office management method and system based on big data behavior analysis

Abstract

This invention discloses a cloud document collaborative office management method and system based on big data behavior analysis, belonging to the field of big data analysis technology. The method and system include: S1, collecting cloud document collaborative operation data, preprocessing the data, storing it, and constructing a document management database; S2, performing window span adjustment, partition parallelism shaping, and operator migration optimization operations based on performance diagnosis results; S3, performing concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on stress assessment results; and S4, integrating operation logs and scheduling process data for scheduling optimization analysis, and achieving closed-loop optimization of resource orchestration and scheduling backtracking based on the optimization analysis results. This solves the problems of large fluctuations in link performance, difficulty in quantifying and controlling resource load pressure, and difficulty in closed-loop optimization of scheduling strategies during cloud document collaborative operation.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, specifically to a cloud-based collaborative office management method and system for documents based on big data behavioral analysis. Background Technology

[0002] Cloud-based collaborative document processing is gradually replacing traditional local documents and standalone office software, becoming the core platform for daily collaborative creation, version circulation, and approval tracking within enterprises. When users create documents, edit them collaboratively, annotate online, embed attachments, and access them across devices in the cloud, continuous access requests, event logs, and metadata updates are generated. A large amount of behavioral data streams through the access layer, gateway layer, collaborative editing engine, and storage cluster, placing continuous pressure on overall processing capabilities. To support high-concurrency access and high-throughput document operations, mainstream cloud-based collaborative document platforms generally adopt distributed storage, message queues, streaming computing frameworks, container orchestration frameworks, and monitoring and alerting platforms to build their infrastructure.

[0003] For example, invention patent CN110019279B discloses a method, apparatus, device, and storage medium for collaborative updating of online documents. The method involves a first client identifying a first target cell to be updated in its current first display interface; obtaining the first location information of the first target cell in a database; and sending an update notification message carrying the first location information to a database server. The database server then synchronizes the update notification message to other second clients collaborating with the first client. The second clients determine a third target cell corresponding to the first location information in their current second display interface based on the update notification message and display the third target cell. The second display interface has a different display view than the first display interface. By using different display interfaces on different clients to collaboratively edit online documents in a database, conflicts arising from multiple clients simultaneously editing data at the same location can be avoided.

[0004] For example, invention patent CN112989124A discloses a multi-network linkage data collaborative configuration method, device, computing equipment, and storage medium, including: parsing northbound files as XML files to obtain northbound file data; obtaining a neighboring cell list by associating the northbound file data, existing network modification work orders, and multi-network management data; generating a neighboring cell parameter work order based on the neighboring cell list; and sending the neighboring cell parameter work order to the multi-network management server for automatic synchronization and updating of multi-network neighboring cell level parameters. Through the above method, the embodiments of the present invention can shorten the time required for batch work order modification, while reducing the probability of errors when manually matching neighboring cells, achieving extremely high accuracy and completeness, and greatly improving parameter production efficiency.

[0005] However, existing cloud document collaboration solutions primarily focus on using logs for operational profiling and usage habit analysis in terms of big data behavior analysis, for recommendation capabilities and capacity planning. The coupling between behavioral data and stream processing engines and resource scheduling logic is relatively weak. Faced with complex scenarios such as simultaneous collaborative editing by multiple teams, batch migration of large amounts of historical documents, and concentrated bursts of cross-regional access, existing stream processing and resource scheduling frameworks often need to compromise between latency control, throughput assurance, and heterogeneous computing power utilization. They lack link performance diagnosis based on behavioral characteristics, load assessment, and scheduling strategy linkage mechanisms, making it difficult to provide more refined and agile performance assurance capabilities for cloud document collaborative office scenarios.

[0006] Therefore, in order to address the above issues, there is an urgent need for cloud-based document collaborative office management methods and systems based on big data behavioral analysis. Summary of the Invention

[0007] Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a cloud document collaborative office management method and system based on big data behavior analysis, which solves the problems of large fluctuations in link performance, difficulty in quantifying and controlling resource carrying pressure, and difficulty in optimizing scheduling strategies in a closed loop during cloud document collaborative operation.

[0009] Technical solution

[0010] To achieve the above objectives, the present invention provides the following technical solution: a cloud document collaborative office management method and system based on big data behavior analysis, comprising: S1, collecting cloud document collaborative operation data, preprocessing the cloud document collaborative operation data, storing it, and constructing a document management database; S2, performing link performance diagnosis through stream processing operation data, and performing window span adjustment, partition parallelism shaping, and operator migration optimization operations based on the performance diagnosis results; S3, performing load pressure assessment through task and resource utilization data, and performing concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the pressure assessment results; S4, integrating operation logs and scheduling process data to construct a scheduling optimization model and perform scheduling optimization analysis, and achieving resource orchestration and scheduling backtracking closed-loop optimization based on the optimization analysis results.

[0011] Further, the specific steps for collecting cloud document collaborative operation data are as follows: Collect cloud document collaborative operation data: Collect the number of processed data records and the average time consumed per data record through the stream processing framework monitoring interface; read the event time for each record entering the operator and write the processing time into the operator processing entry point; calculate the single-record latency by subtracting the event time from the processing time to obtain the data stream latency value; collect CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization through the operating system and cluster monitoring components; collect task scheduling data through the resource scheduler interface to obtain task processing time, task resource requirements, task bandwidth requirements, and the total number of concurrent tasks; collect resource utilization monitoring data through the cluster monitoring platform interface to statistically analyze the overall load rate and memory usage. The system monitors system utilization, I / O wait queue length, and packet loss rate. It collects task metadata through the task orchestration and configuration management interface, obtaining service level tags, target end-to-end latency, enqueue time and current time, resource requirement tags, and bandwidth requirement tags. It generates real-time and non-real-time task identifiers by comparing the target end-to-end latency with latency thresholds, and calculates queue waiting time. Based on the real-time task identifier, queue waiting time, and waiting threshold, tasks identified as real-time with a queue waiting time greater than the waiting threshold are marked as first priority; tasks identified as real-time with a queue waiting time not greater than the waiting threshold are marked as second priority; and tasks identified as non-real-time are marked as third priority and written to the task scheduling record.

[0012] Further, the specific steps for collecting cloud document collaborative operation data, preprocessing it, storing it, and building a document management database are as follows: Service level tags, real-time task identifiers, non-real-time task identifiers, resource requirement tags, and bandwidth requirement tags are uniformly standardized and encoded, converted into an internal unified representation format to achieve dimensionless processing. The minimum-maximum normalization method is used to scale numerical attributes such as the number of processed data records, average time consumption per data item, task processing time, overall load rate, overall memory utilization, I / O wait queue length, and packet loss rate to a unified range. The standardized and normalized cloud document collaborative operation data is then appended with job identifiers, execution topology numbers, and collection timestamps before being stored and used to build the document management database.

[0013] Furthermore, the specific steps for link performance diagnosis using stream processing runtime data are as follows: Obtain the number of data records successfully processed per unit time, the average time for a single data record, the data stream latency, CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization; Multiply the average time for a single data record by adding the data stream latency to obtain the latency correction time item; Multiply the CPU resource utilization and GPU resource utilization to obtain the computational resource utilization item; Add the I / O efficiency and network bandwidth utilization to obtain the data channel utilization item; Divide the computational resource utilization item by the data channel utilization item to obtain the resource matching correction item; Multiply the latency correction time item by the resource matching correction item to obtain the comprehensive time baseline; Finally, divide the number of data records successfully processed per unit time by the comprehensive time baseline to obtain the data processing efficiency value.

[0014] Furthermore, the specific steps for performing window span adjustment, partition parallelism shaping, and operator migration optimization based on the performance diagnosis results are as follows: By comparing the data processing efficiency value and efficiency threshold in real time, when the data processing efficiency value is less than the efficiency threshold, shorten the event monitoring window span and increase the partition parallelism, migrate the upper-level operators with CPU utilization greater than k% to the graphics accelerator card for execution and enable the local solid-state cache link, and enable back pressure throttling and batch pull compression queue waiting latency; if the I / O wait queue length is greater than the length threshold and the packet loss rate is greater than the packet loss threshold, then adjust the corresponding chain... The system enables data compression and binary compact encoding, switches to long-connection streaming, and performs operator chain merging and deduplication. If the calculated data processing efficiency value is still less than the efficiency threshold, the job is generated into a bottleneck queue in descending order of operator time. The parallelism limit of the front-end operator of the bottleneck queue is increased, and elastic expansion is triggered to add computing nodes. At the same time, the triggering reason and adjustment action are archived to the document management database. When the data processing efficiency value is greater than or equal to the efficiency threshold, the efficiency consolidation strategy is triggered: the window and backpressure configurations remain unchanged, and the effective configuration is archived to the document management database.

[0015] Furthermore, the specific steps for assessing carrying capacity pressure using task and resource utilization data are as follows:

[0016] The system retrieves the total number of concurrent tasks, as well as the task priority, processing time, resource requirements, and bandwidth requirements for each task. It also retrieves the overall load rate, CPU utilization, memory utilization, and data processing efficiency. For each task, the task priority is divided by the processing time to obtain its unit-time priority contribution. This unit-time priority contribution is then multiplied by the task resource requirements and bandwidth requirements to obtain the single-task load contribution. The load contributions of all concurrent tasks are summed to obtain the total load requirement for the current set of concurrent tasks. The overall load rate, CPU utilization, and memory utilization are multiplied to obtain the basic capacity. This is then multiplied by the data processing efficiency to obtain the performance capacity adjustment. Finally, the total load requirement is divided by the performance capacity adjustment to obtain the performance stress value.

[0017] Furthermore, the specific steps for implementing concurrent scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration based on the stress assessment results are as follows: By comparing the performance stress value and performance threshold in real time, when the performance stress value is less than the performance threshold, the current heterogeneous resource allocation and task concurrency configuration are maintained, graphics accelerator card instances and edge computing nodes with idle time exceeding the idle time threshold are reclaimed, and the released CPU and memory are preferentially reserved for the upcoming real-time job queue. The performance stress value, task scheduling records, and operator migration results of this round are written to the document management database. When the performance stress value is greater than or equal to the performance threshold, the upper limit of the number of concurrent tasks is reduced, non-real-time tasks are enqueued and rate-limited, and new tasks are imported into the buffer queue. For tasks whose resource requirements and bandwidth requirements both exceed the requirement threshold, resource isolation is enabled and single-task resource quotas are limited. For tasks whose queue waiting time exceeds the waiting threshold, tasks are split into queues and delayed until the performance stress value falls back below the performance threshold.

[0018] Furthermore, the specific steps for constructing a scheduling optimization model and conducting scheduling optimization analysis by integrating operation logs and scheduling process data are as follows: Using stream processing monitoring logs, task scheduling records, resource utilization monitoring sequences, and data processing efficiency and performance pressure values ​​as basic inputs, efficiency alarm records and pressure exceedance flag records are generated based on data processing efficiency values ​​and efficiency thresholds, and performance pressure values ​​and performance thresholds. The service level tags, real-time and non-real-time task identifiers, resource requirements, and bandwidth requirements tags in the task metadata are combined to form a scheduling decision input set for scheduling optimization analysis. A mixed-integer linear programming scheduling optimization algorithm is used to optimize the adjustment of the concurrent task limit, non-real-time task queuing rate limiting and buffer queue import, resource isolation and single-task resource quota limits, and splitting queuing and delayed execution strategies when queue waiting time exceeds the waiting threshold. Finally, a scheduling optimization model is constructed and resource scheduling optimization values ​​are output.

[0019] Furthermore, the specific steps for achieving closed-loop optimization of resource orchestration and scheduling backtracking based on the optimization analysis results are as follows: By comparing the resource scheduling optimization value and the optimization threshold in real time, when the resource scheduling optimization value is greater than or equal to the optimization threshold, a strategy solidification record is generated, and the current constraint set, solution results, and resource scheduling optimization value change trajectory are written into the document management data for subsequent constraint set calibration and fallback baseline update; when the resource scheduling optimization value is less than the optimization threshold, a scheduling backtracking record is generated, and the current task scheduling record, resource utilization monitoring sequence, efficiency alarm record, and pressure over-limit marker record are locked as training samples. The trigger source label is extracted, and the constraint set and penalty term configuration of the constraint planning solution are updated. The next round of resource orchestration scheme is output and a fast recalculation closed loop is entered until the resource scheduling optimization value rises back to the optimization threshold.

[0020] Furthermore, the second aspect of this invention provides a cloud-based collaborative office management system for documents based on big data behavior analysis. This system applies a cloud-based collaborative office management method for documents based on big data behavior analysis, comprising: a data acquisition and preprocessing module for acquiring cloud document collaborative operation data, preprocessing the data, storing it, and constructing a document management database; a stream processing module for performing link performance diagnosis using stream processing operation data, and performing window span adjustment, partition parallelism shaping, and operator migration optimization operations based on the performance diagnosis results; a resource scheduling module for assessing load pressure using task and resource utilization data, and performing concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the pressure assessment results; and a performance evaluation and monitoring module for integrating operation logs and scheduling process data to construct a scheduling optimization model and perform scheduling optimization analysis, and achieving resource orchestration and scheduling backtracking closed-loop optimization based on the optimization analysis results.

[0021] Beneficial effects

[0022] The present invention has the following beneficial effects:

[0023] (1) This invention obtains the data processing efficiency value by combining the processing data record volume, single processing time, data stream latency, CPU and GPU resource utilization, I / O efficiency and network bandwidth utilization. Based on this, it drives the window span adjustment, partition parallelism shaping, operator migration to graphics accelerator card, local solid-state cache activation, data compression and compact encoding, and long connection streaming transmission, thereby achieving the effect of improving the overall data stream processing capability and alleviating I / O and network bottlenecks while maintaining short latency. It effectively solves the problem of relying on experience to tune parameters and difficulty in balancing latency and high throughput in the prior art.

[0024] (2) This invention uses the upper limit of concurrent task number, non-real-time task queuing rate limiting and buffer queue import, resource isolation and single task resource quota limit, and split queuing and delayed execution strategy when queue waiting time exceeds the threshold as decision variables. It constructs a scheduling decision input set by combining service level label, real-time and non-real-time task identifier, and resource and bandwidth requirement label, and uses a mixed integer linear programming scheduling optimization algorithm to output resource scheduling optimization value. In this way, it achieves the effect of overall collaborative planning of key scheduling levers under multiple constraints, effectively solving the problem that the scheduling parameters are adjusted independently in the prior art and it is difficult to obtain the overall optimal scheduling scheme.

[0025] (3) This invention generates real-time task identifiers and non-real-time task identifiers by comparing the end-to-end delay of the task target with the delay threshold, and calculates the queue waiting time by combining the enqueue time with the current time. Real-time tasks with waiting time exceeding the waiting threshold are marked as first priority, real-time tasks with waiting time not exceeding the waiting threshold are marked as second priority, and non-real-time tasks are marked as third priority and written into the task scheduling record. This achieves the effect of explicitly distinguishing between urgent real-time loads and bufferable non-real-time loads in scheduling decisions, and effectively solves the problem of rough task priority division and inability to reflect the impact of end-to-end delay targets in the prior art.

[0026] (4) This invention constructs a scheduling optimization model by taking stream processing monitoring logs, task scheduling records, resource utilization monitoring sequences, data processing efficiency values ​​and performance pressure values ​​as inputs. It compares the resource scheduling optimization value with the optimization threshold in real time. When the optimization value meets the standard, it solidifies the constraint set and the solution result. When the optimization value does not meet the standard, it locks the current round of scheduling data and alarm records as training samples, updates the constraint set and penalty item configuration, and outputs the next round of resource orchestration scheme to form a fast recalculation closed loop. This achieves the effect that the resource orchestration strategy can continuously and adaptively optimize with the business load and operating status. It effectively solves the problem in the prior art that the scheduling strategy relies on static configuration based on human experience and is difficult to dynamically optimize in a timely manner with the changes in cloud document collaborative load.

[0027] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0028] Figure 1 This is a flowchart of the cloud-based collaborative office management method for documents based on big data behavior analysis, as described in this invention.

[0029] Figure 2 This is a structural diagram of the cloud-based collaborative office management system for documents based on big data behavior analysis, as described in this invention.

[0030] Figure 3 This is a radar diagram showing the coupling characteristics of the core performance indicators of the stream processing of this invention.

[0031] Figure 4 This is a flowchart of the task resource scheduling process guided by performance pressure in this invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Please see Figures 1-4 This invention provides a technical solution: a cloud document collaborative office management method and system based on big data behavior analysis, comprising: S1, collecting cloud document collaborative operation data, preprocessing the cloud document collaborative operation data, storing it, and constructing a document management database; S2, performing link performance diagnosis through stream processing operation data, and performing window span adjustment, partition parallelism shaping, and operator migration optimization operations based on the performance diagnosis results; S3, performing load pressure assessment through task and resource utilization data, and performing concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the pressure assessment results; S4, integrating operation logs and scheduling process data to construct a scheduling optimization model and perform scheduling optimization analysis, and realizing resource orchestration and scheduling backtracking closed-loop optimization based on the optimization analysis results.

[0034] Specifically, the steps for collecting cloud document collaborative operation data are as follows: Collect cloud document collaborative operation data by monitoring the data record volume and average time per data record through the stream processing framework monitoring interface; read the event time for each record entering the operator and write the processing time into the operator processing entry point; calculate the single-record latency by subtracting the event time from the processing time to obtain the data stream latency value; collect CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization through the operating system and cluster monitoring components; and collect task scheduling data through the resource scheduler interface to obtain task processing time, task resource requirements, task bandwidth requirements, and the total number of concurrent tasks. Resource utilization monitoring data is collected through the cluster monitoring platform interface to calculate overall load rate, overall memory utilization, I / O wait queue length, and packet loss rate. Task metadata is collected through the task orchestration and configuration management interface to obtain service level tags, task target end-to-end latency, enqueue time and current time, resource requirement tags, and bandwidth requirement tags. Real-time and non-real-time task identifiers are generated by comparing the task target end-to-end latency with latency thresholds, and queue waiting time is calculated. Based on the real-time task identifier, queue waiting time, and waiting threshold, tasks identified as real-time and with queue waiting times greater than the waiting threshold are marked as first priority. Tasks marked as real-time with a queue waiting time not exceeding a waiting threshold are designated as second priority. Tasks marked as non-real-time are designated as third priority and written to the task scheduling record. Resource control policies for subsequent scheduling phases are bound to tasks of different priorities: a real-time guarantee queue is configured for first-priority tasks, assigning a queue weight value w1 and a concurrent execution slot number of c1, with CPU and GPU resource quota limits set to r1 in the resource isolation policy; a balanced queue is configured for second-priority tasks, assigning a queue weight value w2 and a concurrent execution slot number of c2, with corresponding CPU and GPU resource quota limits set in the resource isolation policy. Limit r2; configure a buffer queue for the third priority task, assign a queue weight value w3 and a concurrent execution slot number c3. During the non-real-time task rate limiting buffering phase, prioritize the third priority task for enqueuing rate limiting and buffer queue import, and set the CPU and GPU resource quota upper limit r3 in the resource isolation strategy, where w1 is greater than w2 and w2 is greater than w3, c1 is greater than c2 and c2 is greater than c3, r1 is greater than r2 and r2 is greater than r3. Through the above binding relationship, subsequent concurrency scale control, non-real-time task rate limiting buffering and heterogeneous resource orchestration operations can achieve differentiated queue scheduling and resource allocation based on the first priority, second priority and third priority.

[0035] In this implementation plan, through the aforementioned cloud document collaborative operation data acquisition and preprocessing steps, the processing data record volume, average time per data item, data stream latency, CPU resource utilization, GPU resource utilization, I / O efficiency, network bandwidth utilization, task processing time, task resource requirements, task bandwidth requirements, overall load rate, overall memory utilization, I / O wait queue length, packet loss rate, as well as service level labels, task target end-to-end latency, enqueue time and current time, resource requirement labels, and bandwidth requirement labels are all uniformly incorporated into the same acquisition process. This not only provides a multi-dimensional operational view of the coverage link performance and resource consumption, but also generates real-time task identifiers and non-identifiers by comparing the task target end-to-end latency with latency thresholds. Real-time task identification and queue waiting time calculation are performed. Tasks are then classified into first, second, and third priorities and written into the task scheduling record. Differentiated queue weights, concurrent execution slots, and resource quotas are bound to tasks of different priorities. This allows subsequent concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration to be precisely differentiated at the decision-making level based on the fixed priority and resource control mapping. This achieves the effect of orderly yielding and resource shrinking of bufferable tasks while ensuring the end-to-end latency requirements of real-time tasks. It effectively solves the problem in existing technologies where coarse task priority division and decoupling of resource isolation and rate limiting strategies from priorities lead to real-time tasks being easily squeezed out by non-real-time tasks.

[0036] Specifically, the steps for collecting cloud document collaborative operation data, preprocessing it, storing it, and building a document management database are as follows: Service level tags, real-time task identifiers, non-real-time task identifiers, resource requirement tags, and bandwidth requirement tags are uniformly standardized and encoded. Tag value domain alignment and encoding mapping are completed, and the data is converted to an internally unified representation format to achieve dimensionless processing. Missing tags are filled in according to the consistency rules of the source field, and conflicting tags are overwritten and merged according to the latest collection timestamp under the same job identifier. The minimum-maximum normalization method is used to scale numerical attributes such as the number of processed data records, average time consumption per data item, task processing time, overall load rate, overall memory utilization, I / O wait queue length, and packet loss rate to a unified range. The correspondence between the original dimensional fields and the normalized fields is retained for the normalized numerical attributes, used for subsequent task scheduling records and traceability verification of resource utilization monitoring sequences. The standardized and normalized cloud document collaborative operation data is then appended with job identifiers, execution topology numbers, and collection timestamps before being stored and used to build document management data.

[0037] In this implementation plan, service level labels, real-time task identifiers, non-real-time task identifiers, resource requirement labels, and bandwidth requirement labels are uniformly standardized and encoded and converted into an internal unified representation format. At the same time, minimum and maximum normalization scaling is applied to the amount of data records processed, the average time for a single data item, the task processing time, the overall load rate, the overall memory utilization rate, the I / O wait queue length, and the packet loss rate. This ensures that data from different sources and with different dimensions are comparable and aggregable within the same metric space. Furthermore, traceable storage is achieved by adding job identifiers, execution topology numbers, and collection timestamps. This improves the data consistency, computational stability, and backtracking and location capabilities for subsequent link performance diagnosis, load pressure assessment, and scheduling optimization analysis.

[0038] Specifically, the steps for link performance diagnosis using stream processing runtime data are as follows: Obtain the number of successfully processed data records per unit time, average time per data record, data stream latency, CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization. Collect these metrics using stream processing monitoring and resource monitoring components. Use the number of successfully processed data records per unit time as the throughput measurement, and the average time per data record and data stream latency as the latency measurement. Multiply the average time per data record by the data stream latency to obtain the latency correction term, converging the latency-related impact into a single measurable dimension. Multiply the CPU resource utilization and GPU resource utilization to obtain the calculated latency. The resource utilization item characterizes the current concentration of computing resources; the data channel utilization item is obtained by adding I / O efficiency and network bandwidth utilization, which reflects the joint carrying capacity of storage channels and network channels; the resource matching correction item is obtained by dividing the computing resource utilization item by the data channel utilization item, which describes the matching relationship between computing power and data channels; the comprehensive time consumption benchmark is obtained by multiplying the latency correction time consumption item by the resource matching correction item, which combines latency factors and resource matching factors into a single evaluation benchmark; the data processing efficiency value is finally obtained by dividing the number of data records successfully processed per unit time by the comprehensive time consumption benchmark, which is used to quantify the comprehensive performance of the current stream processing link in completing data processing under given resource consumption and latency constraints.

[0039] The specific formula for calculating data processing efficiency is as follows:

[0040] ;

[0041] In the formula, This represents the data processing efficiency value, which is an overall performance indicator of the stream processing module. The higher the value, the stronger the system's ability to process data per unit time using given computing and I / O resources, and the better the overall efficiency. This indicates the data records being processed, reflecting the system's core throughput capacity; This represents the average time taken for a single data point, reflecting the complexity of the computational logic and execution efficiency. This represents the data stream latency value, reflecting the real-time nature of the data processing flow; This represents CPU resource utilization, reflecting the degree of saturation of CPU computing resources in stream processing tasks. This indicates GPU resource utilization, reflecting the efficiency and acceleration benefits of GPU-accelerated devices in streaming tasks; Indicates I / O efficiency, reflecting the speed and efficiency of reading and writing data in the storage subsystem; This indicates network bandwidth utilization, reflecting the actual proportion of bandwidth used in network data transmission.

[0042] In this embodiment, the first group of processed data records has a time of 0.818, an average time of 0.892 for a single data record, a data stream latency of 0.08, a CPU resource utilization of 0.75, a GPU resource utilization of 0.68, an I / O efficiency of 0.82, a network bandwidth utilization of 0.79, and a data processing efficiency of 0.86. The second group of processed data records has a time of 0.018, an average time of 0.0003 for a single data record, a data stream latency of 0.15, a CPU resource utilization of 0.88, a GPU resource utilization of 0.79, an I / O efficiency of 0.65, a network bandwidth utilization of 0.62, and a data processing efficiency of 0. 72; The third group of data records processed 0.436, with an average time of 0.649 for a single data record, a data stream latency of 0.10, CPU resource utilization of 0.78, GPU resource utilization of 0.72, I / O efficiency of 0.78, network bandwidth utilization of 0.75, and data processing efficiency of 0.81; The fourth group of data records processed 1.000, with an average time of 1.000 for a single data record, a data stream latency of 0.06, CPU resource utilization of 0.72, GPU resource utilization of 0.70, I / O efficiency of 0.85, network bandwidth utilization of 0.81, and data processing efficiency of 0.89.

[0043] Table 1. Data Table of Correlation Indicators for Data Processing Efficiency

[0044] Data group ID Processing data records Average time per data point Data stream latency value CPU resource utilization GPU resource utilization I / O efficiency Network bandwidth utilization Data processing efficiency value 1 0.818 0.892 0.08 0.75 0.68 0.82 0.79 0.86 2 0.018 0.0003 0.15 0.88 0.79 0.65 0.62 0.72 3 0.436 0.649 0.10 0.78 0.72 0.78 0.75 0.81 4 1.000 1.000 0.06 0.72 0.70 0.85 0.81 0.89

[0045] like Figure 3 The image shown is a radar chart illustrating the coupling characteristics of core performance indicators for stream processing provided in this application embodiment. Combined with the data in Table 1... Figure 3As can be seen, this radar chart normalizes the core metrics of stream processing to a unified range, intuitively presenting the coupling relationship between the distribution characteristics of each metric and the data processing efficiency value. Group 4 has the highest processed data record value of 1.000, the best average time per data record of 1.000, and the lowest data stream latency of 0.06. Simultaneously, I / O efficiency and network bandwidth utilization are the highest among all groups, while CPU and GPU resource utilization are within a reasonable range. The optimal synergy of these metrics supports this group's data processing efficiency value reaching the highest of the four groups at 0.89. Group 2 has the lowest normalized value of 0.018, and the lowest average time per data record is... The average value of 0.0003 is the worst among all groups, with relatively high data flow latency, low I / O efficiency, and low network bandwidth utilization. Even with high CPU and GPU resource utilization, the data processing efficiency value of this group is still the lowest among the four groups at 0.72. The core indicators of groups 1 and 3 are all at the middle level, and their data processing efficiency values ​​are also in the middle range of 0.81 to 0.86. The overall data and radar chart indicator distribution are highly consistent, which verifies that the data processing efficiency value formula can effectively integrate throughput, processing time, data flow latency, and multi-dimensional resource utilization indicators to achieve a scientific quantitative evaluation of the comprehensive performance of the streaming processing link.

[0046] In this implementation scheme, through the above calculation process, the number of data records successfully processed per unit time, the average time for a single data record, the data stream latency, CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization are compressed into a unified data processing efficiency value index. This allows for a quantitative characterization of the coupling relationship between throughput, latency, computational resource consumption, and data channel carrying capacity in the stream processing link. In subsequent stages, the data processing efficiency value can be used to identify performance bottlenecks in the operator link, determine the matching degree between resource allocation and channel utilization, and provide comparable numerical basis for window span adjustment, partition parallelism shaping, and operator migration strategies. This achieves the effect of reflecting multi-dimensional operating status with a single index and providing a unified quantitative benchmark for link performance diagnosis.

[0047] Specifically, the steps for performing window span adjustment, partition parallelism shaping, and operator migration optimization based on performance diagnostic results are as follows: By comparing data processing efficiency values ​​and efficiency thresholds in real time, when the data processing efficiency value is less than the efficiency threshold, the event monitoring window span is shrunk and the partition parallelism is increased based on the changing direction of the number of processed data records, the average time consumed by a single data item, and the data stream latency value, in order to reduce queue backlog within a single window and improve concurrent processing capacity per unit time; operators with CPU utilization greater than k% are sorted according to operator time consumption and resource usage characteristics and then migrated to the graphics accelerator card for execution, and a local solid-state cache link is enabled on the migration link side to shorten the cross-node data transfer path, while back pressure rate limiting and batch pull compression queue waiting latency are enabled to suppress the impact of sudden traffic on downstream operators; if I / O occurs... If the waiting queue length exceeds the length threshold and the packet loss rate exceeds the packet loss threshold, data compression and binary compact encoding are enabled for the corresponding link, and long-connection streaming is switched to reduce the transmission overhead per record and reduce the retransmission amplification effect. Operator chain merging and deduplication are performed to reduce the additional time consumption caused by intermediate serialization and repeated calculations. If the data processing efficiency value obtained from continuous calculation is still less than the efficiency threshold, the job is generated into a bottleneck queue in descending order of operator time. The parallelism limit of the front-end operator of the bottleneck queue is increased and elastic expansion is triggered to add computing nodes. At the same time, the triggering reason, the bottleneck queue sorting result and adjustment action are archived to the document management database to form a reversible configuration baseline. When the data processing efficiency value is greater than or equal to the efficiency threshold, the efficiency consolidation strategy is triggered to keep the window and backpressure configuration unchanged, and the effective configuration is archived to the document management database.

[0048] In this implementation scheme, the window span, partition parallelism, operator migration, and link transmission mode are adjusted in a coordinated manner by comparing the data processing efficiency value with the efficiency threshold in real time. When the data processing efficiency value deviates from the target, the queue waiting latency is compressed in a timely manner and the concurrent processing capability is improved. At the same time, when the I / O waiting queue length and packet loss rate exceed the limits, the link overhead is reduced by data compression, binary compact encoding, and long connection streaming transmission. Combined with operator chain merging and deduplication to reduce redundant calculations and intermediate state overhead, the bottleneck queue and elastic expansion are used to implement targeted gains for key operators when the efficiency value is consistently below the efficiency threshold. Finally, an effective configuration baseline that can be archived and rolled back is formed, thereby improving the stable throughput and latency controllability of the stream processing link under high load fluctuations.

[0049] Specifically, the steps for assessing carrying capacity pressure using task and resource utilization data are as follows:

[0050] The system obtains the total number of concurrent tasks, as well as the task priority, processing time, resource requirements, and bandwidth requirements for each task. It also obtains the overall load rate, CPU resource utilization, overall memory utilization, and data processing efficiency. Task priority is considered a comprehensive description of business urgency and service level, task processing time is considered the duration of computing resource usage per task, and task resource and bandwidth requirements are considered descriptions of computing power and network channel usage intensity. For each task, the task priority is divided by the task processing time to obtain the task's unit-time priority contribution value, reflecting the task's contribution to service assurance goals per unit time. This unit-time priority contribution value is then multiplied by the task resource and bandwidth requirements to obtain the single-task load contribution item, representing the workload of a task on the overall capacity under the current resource configuration. The load contribution of all concurrent tasks is summed to obtain the total load demand value of the current concurrent task set, which is used to characterize the overall carrying capacity requirements of the entire task set under the existing priority structure and resource demand structure. The overall load rate, CPU resource utilization rate and overall memory utilization rate are multiplied to obtain the basic carrying capacity term, which is used to represent the carrying capacity benchmark that the current resource side can maintain under the current occupancy level. Then, the data processing efficiency value is multiplied by the basic carrying capacity term to obtain the performance carrying capacity correction term, so that the comprehensive performance feedback of the data flow processing link is included in the carrying capacity assessment. The total load demand value is divided by the performance carrying capacity correction term to finally obtain the performance pressure value, which is used to quantify the tension between the current task demand side and the resource carrying side, and provide a unified pressure criterion for subsequent concurrency scale control, non-real-time task rate limiting buffering and heterogeneous resource orchestration.

[0051] The specific formula for calculating the performance pressure value is as follows:

[0052] ;

[0053] This represents the performance pressure value, indicating the overall demand pressure on various resources from the tasks currently being undertaken by the system; It indicates task priority, reflecting the relative importance and urgency of the task; This indicates the task processing time, reflecting the duration the task consumes computing resources. This indicates the task's resource requirements, reflecting the overall intensity of the task's demand for heterogeneous computing resources; This indicates the task's bandwidth requirements, reflecting the task's dependence on network I / O. It represents the overall load rate, reflecting the overall pressure caused by all running tasks; This indicates CPU resource utilization and reflects the overall saturation level of cluster CPU resources. This represents the overall memory utilization rate, reflecting the overall level of strain on cluster memory resources. This represents the total number of concurrent tasks, reflecting the system's concurrent processing scale. This represents the data processing efficiency value, and the overall performance index of the stream processing module.

[0054] In this implementation plan, the total number of concurrent tasks, task priorities, task processing time, task resource requirements, task bandwidth requirements, as well as overall load rate, CPU resource utilization, overall memory utilization, and data processing efficiency are uniformly converted into a single performance pressure value. This quantifies the tension between the current overall demand intensity on the task side and the carrying capacity on the resource side into a pressure indicator that can be directly compared and monitored. This allows for subsequent steps to determine whether the load is close to saturation based on the performance pressure value, identify resource shortage and redundancy intervals, and drive concurrent scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration strategy selection and scaling. Ultimately, this achieves the effect of using a single indicator to comprehensively reflect multi-dimensional load status and provide a clear quantitative basis for pressure-driven resource scheduling decisions. This effectively solves the problem in existing technologies where it is difficult to accurately perceive the global load pressure level, resulting in a lack of unified criteria for scheduling adjustments.

[0055] Specifically, the steps for implementing concurrency control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration based on the stress assessment results are as follows: Figure 4 This is a flowchart of the performance pressure-driven task resource scheduling process in this embodiment. By comparing the performance pressure value and the performance threshold in real time, when the performance pressure value is less than the performance threshold, the current heterogeneous resource allocation and task concurrency configuration are maintained. The idle status of resources is verified by combining the monitoring results of the total number of concurrent tasks, CPU resource utilization, and overall memory utilization. Graphics accelerator card instances and edge computing nodes with idle time exceeding the idle threshold are reclaimed to reduce idle occupation. The released CPU and memory are preferentially reserved for the upcoming real-time job queue to ensure the end-to-end latency constraint of the task target corresponding to the service level label. At the same time, the performance pressure value, task scheduling record, and operator migration of this round are recorded. The results are written into the document management database to form a traceable resource adjustment record. When the performance pressure value is greater than or equal to the performance threshold, the upper limit of the number of concurrent tasks is reduced to suppress concurrent crowding. Non-real-time tasks are enqueued and rate-limited, and new tasks are imported into the buffer queue to alleviate the impact of instantaneous peak on the real-time job queue. For tasks whose resource demand and bandwidth demand both exceed the demand threshold, resource isolation is enabled and the resource quota of a single task is limited to avoid excessive occupation of a single task leading to an increase in the overall load rate. For tasks whose queue waiting time exceeds the waiting threshold, tasks are split into enqueued tasks and delayed to reduce the pressure of a single enqueuing and shorten the waiting path of critical queues until the performance pressure value falls back below the performance threshold.

[0056] In this implementation scheme, dynamic shaping of heterogeneous resource allocation and task concurrency configuration is driven by the real-time comparison results of performance pressure value and performance threshold. When the performance pressure value is low, idle resources are reclaimed and the released CPU and memory are reserved for real-time job queues to ensure end-to-end latency constraints of task objectives. At the same time, the performance pressure value, task scheduling records and operator migration results of this round are recorded for traceability. When the performance pressure value exceeds the limit, the peak impact is suppressed by lowering the upper limit of the number of concurrent tasks, limiting the rate of non-real-time task enqueueing and introducing buffer queues. Resource isolation and single-task resource quota limits are implemented for tasks with resource and bandwidth requirements exceeding the limit. Combined with the splitting and delayed execution of tasks with queue waiting time exceeding the limit, congestion propagation is reduced. Ultimately, the resource carrying pressure can be quantified, controlled and reduced, improving the operational stability and scheduling sustainability in multi-resource competition scenarios.

[0057] Specifically, the steps for constructing a scheduling optimization model and conducting scheduling optimization analysis by integrating operation logs and scheduling process data are as follows: Using stream processing monitoring logs, task scheduling records, resource utilization monitoring sequences, and data processing efficiency and performance pressure values ​​as basic inputs, efficiency alarm records and pressure exceedance marker records are generated based on data processing efficiency values ​​and efficiency thresholds, and performance pressure values ​​and performance thresholds. A scheduling decision input set is constructed by combining service level tags, real-time and non-real-time task identifiers, resource requirements, and bandwidth requirements tags from task metadata for scheduling optimization analysis. A mixed-integer linear programming scheduling optimization algorithm is used to optimize the adjustment of the concurrent task limit, non-real-time task queuing rate limiting and buffer queue import, resource isolation and single-task resource quota limits, and splitting queuing and delayed execution strategies when queue waiting time exceeds the waiting threshold. A scheduling optimization model is constructed, and the resource scheduling optimization value is output. The resource scheduling optimization value is a scalar score calculated by the scheduling optimization model, representing the overall optimization level of the current task scheduling strategy. It is calculated using a mixed-integer linear programming algorithm combined with multiple scheduling decision parameters. The higher the resource scheduling optimization value, the better the scheduling strategy performs in terms of resource utilization, task processing efficiency, and load balancing. If the strategy vector fails to meet expectations, the resource scheduling optimization value will be correspondingly lower, indicating that the strategy needs further adjustment. Ultimately, the optimization value is used to evaluate the effectiveness of the current scheduling scheme and provide feedback for subsequent scheduling decisions.

[0058] In this implementation scheme, by combining stream processing monitoring logs, task scheduling records, resource utilization monitoring sequences, and data processing efficiency and performance pressure values, the system can monitor and evaluate the performance of task scheduling in real time, and alert the system to abnormal load by generating efficiency alarm records and pressure exceedance flag records. Utilizing service level tags, real-time and non-real-time task identifiers, and resource and bandwidth requirement tags from task metadata, combined with a mixed-integer linear programming scheduling optimization algorithm, the system can optimize task scheduling strategies based on real-time load conditions, adjust the number of concurrent tasks, non-real-time task queuing rate limiting, resource isolation, and queue waiting strategies, and output resource scheduling optimization values. This optimization value, as a scalar score, comprehensively reflects the performance of the current scheduling scheme, effectively guiding subsequent resource configuration and task scheduling adjustments, ensuring that the system can dynamically respond to changes in the priority and resource requirements of different tasks while efficiently utilizing resources, thereby improving overall system performance.

[0059] Specifically, the steps for achieving closed-loop optimization of resource orchestration and scheduling backtracking based on the optimization analysis results are as follows: By comparing the resource scheduling optimization value and the optimization threshold in real time, when the resource scheduling optimization value is greater than or equal to the optimization threshold, a strategy solidification record is generated, and the current round of constraint set, solution results, and resource scheduling optimization value change trajectory are written into the document management data for subsequent constraint set calibration and rollback baseline update, ensuring that the resource allocation decision during the optimization process is traceable and has a rollback baseline; when the resource scheduling optimization value is less than the optimization threshold, a scheduling backtracking record is generated, and the current round of task scheduling record and resource scheduling backtracking record are locked. The source uses monitoring sequences, efficiency alarm records, and pressure overrun marker records as training samples. These samples will be used to train a reinforcement learning model based on historical task scheduling and resource utilization, update the constraint set and penalty term configuration in the constraint programming solution, and adjust the penalty term by analyzing the negative effects of resource allocation imbalance and task delay in historical samples to penalize scheduling strategies that over-occupy resources. The next round of resource orchestration scheme is output and enters a fast recalculation closed loop until the resource scheduling optimization value rises back to the optimization threshold, ensuring that resource overload and task conflicts are gradually reduced in subsequent scheduling, thereby improving system operating efficiency.

[0060] In this implementation scheme, by comparing the resource scheduling optimization value with the optimization threshold in real time, when the resource scheduling optimization value reaches or exceeds the optimization threshold, the system will generate a strategy solidification record to ensure the traceability of optimization decisions and adjustment actions, while supporting subsequent constraint set calibration and rollback baseline updates. When the optimization value is lower than the optimization threshold, the system will use task scheduling records, resource utilization monitoring, efficiency alarms, and pressure over-limit markers as training samples, use these data to update the constraint set and penalty term configuration, continuously optimize the scheduling strategy through a reinforcement learning model, and then output a new resource orchestration scheme and start a fast recalculation closed loop until the optimization value rises back above the optimization threshold, ensuring that resource allocation and task scheduling reach the optimal state and are continuously improved.

[0061] Specifically, this embodiment provides a cloud-based collaborative office management system for documents based on big data behavior analysis. This system is applied to cloud-based collaborative office management using big data behavior analysis and includes: a data acquisition and preprocessing module responsible for collecting cloud document collaborative operation data and performing preliminary cleaning, transformation, and formatting preprocessing operations on the collected data. The preprocessed data is stored in a database to build a document management database, ensuring data standardization and structure for efficient subsequent module access and analysis. A stream processing module performs link performance diagnosis on the stream processing operation data to identify potential bottlenecks and performance issues. Based on the performance diagnosis results, the module performs window span adjustment, partition parallelism shaping, and operator migration optimization operations to improve the parallelism and overall processing efficiency of the data stream processing, ensuring the system can efficiently process massive amounts of data while maintaining low latency. A resource scheduling module uses task and resource utilization data to assess load capacity and accurately calculate the resource load required for each task. Based on the load assessment results, the module performs operations such as concurrency scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration to optimize resource allocation, ensure optimal use of system resources, and avoid performance bottlenecks caused by resource overload. The performance evaluation and monitoring module integrates runtime logs with scheduling process data to construct a scheduling optimization model and perform scheduling optimization analysis. Based on the results of the optimization analysis, the module can dynamically adjust resource orchestration schemes and perform scheduling backtracking closed-loop optimization to ensure that scheduling decisions always maximize system operating efficiency and stability under different loads and changing conditions.

[0062] In this implementation plan, the efficiency and stability of cloud document collaborative operation are comprehensively improved through the collaborative work of various modules. Data collection and preprocessing provide high-quality input for subsequent modules. Next, the stream processing module diagnoses link performance and optimizes the data stream processing process to ensure the system can handle large-scale data while maintaining low latency. The resource scheduling module rationally allocates computing resources based on load assessment and optimizes task execution strategies to avoid overload. The performance evaluation and monitoring module dynamically adjusts resource configuration and scheduling schemes based on real-time analysis results, achieving closed-loop optimization of scheduling backtracking, thereby improving the overall system's resource utilization, response speed, and stability.

[0063] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0064] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A cloud-based collaborative office management method for documents based on big data behavioral analysis, characterized in that: Includes the following steps: S1 collects cloud document collaborative operation data, preprocesses the cloud document collaborative operation data, stores it, and builds a document management database; S2 performs link performance diagnosis using stream processing data, and performs window span adjustment, partition parallelism shaping, and operator migration optimization operations based on the performance diagnosis results. S3 assesses the load capacity using task and resource utilization data, and performs concurrent scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the load assessment results. S4 integrates operation logs and scheduling process data to build a scheduling optimization model and perform scheduling optimization analysis. Based on the optimization analysis results, it realizes closed-loop optimization of resource orchestration and scheduling backtracking.

2. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for collecting cloud document collaborative operation data are as follows: Collect cloud document collaborative operation data: Collect the number of processed data records and the average time for a single data record through the stream processing framework monitoring interface; read the event time for each record entering the operator and write the processing time into the operator processing entry point; calculate the single-record latency by subtracting the event time from the processing time to obtain the data stream latency value; collect CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization through the operating system and cluster monitoring components; collect task scheduling data through the resource scheduler interface to obtain task processing time, task resource requirements, task bandwidth requirements, and the total number of concurrent tasks; collect resource utilization monitoring data through the cluster monitoring platform interface to statistically analyze the overall load rate, overall memory utilization, I / O wait queue length, and packet loss rate. Task metadata is collected through the task orchestration and configuration management interface to obtain service level tags, task target end-to-end latency, enqueue time and current time, resource requirement tags and bandwidth requirement tags; real-time task identifiers and non-real-time task identifiers are generated by comparing the task target end-to-end latency with latency thresholds, and queue waiting time is calculated; based on the real-time task identifier, queue waiting time and waiting threshold, tasks identified as real-time and with queue waiting time greater than the waiting threshold are marked as first priority, tasks identified as real-time and with queue waiting time not greater than the waiting threshold are marked as second priority, and tasks identified as non-real-time are marked as third priority and written to the task scheduling record.

3. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for preprocessing the collected cloud document collaborative operation data, storing it, and constructing a document management database are as follows: Service level labels, real-time task identifiers, non-real-time task identifiers, resource requirement labels, and bandwidth requirement labels are uniformly standardized and encoded, and converted into an internal unified representation format to achieve dimensionless processing. The minimum-maximum normalization method is used to scale numerical attributes such as the number of data records processed, the average time for a single data item, the task processing time, the overall load rate, the overall memory utilization rate, the I / O wait queue length, and the packet loss rate to a unified range. The standardized and normalized cloud document collaborative operation data is then stored after being appended with job identifiers, execution topology numbers, and collection timestamps, and a document management database is constructed.

4. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for diagnosing link performance using stream processing data are as follows: Obtain the number of data records successfully processed per unit time, the average time for a single data record, the data stream latency, CPU resource utilization, GPU resource utilization, I / O efficiency, and network bandwidth utilization; multiply the average time for a single data record by the data stream latency to obtain the latency correction term; multiply the CPU resource utilization by the GPU resource utilization to obtain the computational resource utilization term; add the I / O efficiency and network bandwidth utilization to obtain the data channel utilization term; divide the computational resource utilization term by the data channel utilization term to obtain the resource matching correction term. The overall time consumption baseline is obtained by multiplying the delay correction time item by the resource matching correction item; then the data processing efficiency value is obtained by dividing the number of data records successfully processed per unit time by the overall time consumption baseline.

5. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for performing window span adjustment, partition parallelism shaping, and operator migration optimization operations based on the performance diagnosis results are as follows: By comparing data processing efficiency values ​​and efficiency thresholds in real time, when the data processing efficiency value is less than the efficiency threshold, the event monitoring window span is shortened and the partition parallelism is increased. Operators with CPU utilization greater than k% are migrated to the graphics accelerator card for execution and local solid-state cache links are enabled. Back pressure rate limiting and batch pull compression queue waiting latency are enabled. If the I / O waiting queue length is greater than the length threshold and the packet loss rate is greater than the packet loss threshold, data compression and binary compact encoding are enabled for the corresponding link and it is switched to long connection streaming transmission. Operator chain merging and parsing redundancy removal are performed. If the data processing efficiency value obtained from continuous calculation is still less than the efficiency threshold, the job will be generated into a bottleneck queue in descending order of operator time, the parallelism limit of the front-end operator of the bottleneck queue will be increased and elastic expansion will be triggered to add computing nodes. At the same time, the triggering reason and adjustment action will be archived to the document management database. When the data processing efficiency value is greater than or equal to the efficiency threshold, the efficiency consolidation strategy is triggered: keep the window and backpressure configuration unchanged, and archive the effective configuration to the document management database.

6. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for assessing carrying capacity pressure using task and resource utilization data are as follows: The system retrieves the total number of concurrent tasks, as well as the task priority, processing time, resource requirements, and bandwidth requirements for each task. It also retrieves the overall load rate, CPU utilization, memory utilization, and data processing efficiency. For each task, the task priority is divided by the processing time to obtain its unit-time priority contribution. This unit-time priority contribution is then multiplied by the task resource requirements and bandwidth requirements to obtain the single-task load contribution. The load contributions of all concurrent tasks are summed to obtain the total load requirement for the current set of concurrent tasks. The overall load rate, CPU utilization, and memory utilization are multiplied to obtain the basic capacity. This is then multiplied by the data processing efficiency to obtain the performance capacity adjustment. Finally, the total load requirement is divided by the performance capacity adjustment to obtain the performance stress value.

7. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for performing concurrency control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the stress assessment results are as follows: By comparing the performance pressure value and the performance threshold in real time, when the performance pressure value is less than the performance threshold, the current heterogeneous resource allocation and task concurrency configuration are maintained, graphics accelerator card instances and edge computing nodes with idle time exceeding the idle threshold are reclaimed, and the released CPU and memory are preferentially reserved for the upcoming real-time job queue. The performance pressure value, task scheduling records and operator migration results of this round are written to the document management database. When the performance pressure value is greater than or equal to the performance threshold, the upper limit of concurrent tasks is reduced, non-real-time tasks are enqueued and rate-limited, and new tasks are imported into the buffer queue. For tasks whose resource and bandwidth requirements both exceed the demand threshold, resource isolation is enabled and the resource quota of a single task is limited. For tasks whose queue waiting time exceeds the waiting threshold, tasks are split into queues and delayed until the performance pressure value falls back below the performance threshold.

8. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for constructing a scheduling optimization model and performing scheduling optimization analysis by integrating operation logs and scheduling process data are as follows: Using stream processing monitoring logs, task scheduling records, resource utilization monitoring sequences, and data processing efficiency and performance pressure values ​​as basic inputs, efficiency alarm records and pressure exceedance flag records are generated based on data processing efficiency values ​​and efficiency thresholds, and performance pressure values ​​and performance thresholds. A scheduling decision input set is constructed by combining service level tags, real-time and non-real-time task identifiers, and resource and bandwidth requirement tags from task metadata for scheduling optimization analysis. A mixed-integer linear programming scheduling optimization algorithm is used to optimize the adjustment of the concurrent task limit, non-real-time task queuing rate limiting and buffer queue import, resource isolation and single-task resource quota limits, and splitting queuing and delayed execution strategies when queue waiting time exceeds the waiting threshold. A scheduling optimization model is constructed and resource scheduling optimization values ​​are output.

9. The cloud-based collaborative office management method for documents based on big data behavior analysis according to claim 1, characterized in that: The specific steps for achieving closed-loop optimization of resource orchestration and scheduling backtracking based on the optimization analysis results are as follows: By comparing the resource scheduling optimization value and the optimization threshold in real time, when the resource scheduling optimization value is greater than or equal to the optimization threshold, a strategy solidification record is generated and the current constraint set, solution results and resource scheduling optimization value change trajectory are written into the document management data for subsequent constraint set calibration and rollback baseline update. When the resource scheduling optimization value is less than the optimization threshold, a scheduling backtracking record is generated and the current round of task scheduling record, resource utilization monitoring sequence, efficiency alarm record, and pressure over-limit marker record are locked as training samples. The trigger source label is extracted and the constraint set and penalty term configuration of the constraint planning solution are updated. The next round of resource orchestration scheme is output and the fast recalculation closed loop is entered until the resource scheduling optimization value rises back to the optimization threshold.

10. A cloud-based collaborative office management system for documents based on big data behavior analysis, employing the cloud-based collaborative office management method for documents based on big data behavior analysis as described in any one of claims 1-9, characterized in that... ,include: The data acquisition and preprocessing module is used to collect cloud document collaborative operation data, preprocess the cloud document collaborative operation data, store it, and build a document management database. The stream processing module is used to perform link performance diagnosis through stream processing data, and perform window span adjustment, partition parallelism shaping and operator migration optimization operations based on the performance diagnosis results. The resource scheduling module is used to assess the load capacity based on task and resource utilization data, and to perform concurrent scale control, non-real-time task rate limiting buffering, and heterogeneous resource orchestration operations based on the load assessment results. The performance evaluation and monitoring module is used to integrate operation logs and scheduling process data to build a scheduling optimization model and perform scheduling optimization analysis. Based on the optimization analysis results, it realizes closed-loop optimization of resource orchestration and scheduling backtracking.

Images