Data real-time monitoring processing method and system of multi-service system, terminal and medium

Abstract

The invention relates to a data real-time monitoring and processing method and system of a multi-service system, a terminal and a medium, and belongs to the technical field of service data processing.The data real-time monitoring and processing method takes a client as an execution main body and comprises the steps that service data are received; determining service attributes of the service data and performing attribute marking; dynamically binding corresponding transmission parameters according to the service attributes; according to the transmission parameters, performing block processing on the service data and sending the service data; and after receiving the block data, the server integrates and restores the block data into the service data. The service data transmission method and the service data transmission device have the beneficial effects of improving the real-time performance and the integrity of service data transmission and facilitating monitoring of different service data.

Description

Technical Field

[0001] The present application relates to the technical field of business data processing, and in particular to a method, system, terminal and medium for real-time monitoring and processing of data of a multi-business system. Background Art

[0002] With the continuous growth and expansion of businesses, multiple business systems have gradually become the norm in enterprise operations. Different business systems each have distinct functions and tasks. For example, a customer relationship management system manages and maintains customer information, while an order processing system focuses on receiving, processing, and tracking orders. These business systems generate massive amounts of business data during operation, and the types and volumes of this data are becoming increasingly diverse and massive. For example, in e-commerce companies, in addition to common transaction data, this data also includes user behavior data and logistics information. This data includes not only structured data, such as table data in databases, but also a large amount of unstructured data, such as user reviews, images, and videos. As businesses expand in size and their business complexity increases, the amount of data is exploding.

[0003] In a multi-business system environment, accurate and timely acquisition and processing of business data is crucial for enterprise decision-making and operations. Real-time monitoring of business data can help companies promptly identify operational issues and anomalies, such as order backlogs and increased customer complaints, and quickly implement appropriate measures to resolve them, thereby improving business processing efficiency and customer satisfaction. For example, by monitoring transaction data in real time, companies can promptly detect transaction anomalies and prevent fraud. Furthermore, in-depth analysis of business data can provide valuable insights and decision-making support, helping companies optimize business processes and develop marketing strategies. However, due to differences in data formats, transmission protocols, and storage methods across multiple business systems, achieving real-time monitoring and effective processing of data from multiple business systems has become a pressing issue.

[0004] Currently, there are some shortcomings in data monitoring and processing across multiple business systems. On the one hand, traditional data monitoring methods often lack flexibility and are difficult to adapt to diverse business needs. Different business systems may require different monitoring strategies and processing methods, but existing methods generally adopt a unified processing model and cannot be customized to the characteristics of different business data. On the other hand, existing technologies also have certain issues with data transmission efficiency and accuracy. For example, during the data transmission process, due to excessive data volume or network instability, data loss or transmission delays may occur, affecting the real-time and integrity of the data. Summary of the Invention

[0005] In order to improve the real-time and integrity of business data transmission and facilitate the monitoring of different business data, the present application provides a data real-time monitoring and processing method, system, terminal and medium for a multi-business system.

[0006] In the first aspect, the present application provides a method for real-time monitoring and processing of data in a multi-business system, which adopts the following technical solutions: A method for real-time monitoring and processing of data in a multi-business system, with a client as the execution subject, includes: Receive business data; Determining the business attributes of the business data and performing attribute marking; Dynamically bind corresponding transmission parameters according to the service attributes; According to the transmission parameters, the business data is processed in blocks and sent; after receiving the block data, the server integrates the block data and restores it into the business data.

[0007] By employing the above technical solution, by identifying and labeling the service attributes of service data, the client can clearly distinguish different types of service data. Service attribute labeling enables the client to dynamically adjust transmission parameters based on real-time changes in the service. For example, if the traffic to a particular service suddenly increases, parameters such as the block size and retransmission interval can be adjusted accordingly to ensure efficient data transmission and processing, improving the system's adaptability and flexibility. Blocking service data reduces the amount of data transmitted at a time and reduces the probability of errors during transmission. When network fluctuations or congestion occur, smaller data blocks are easier to retransmit and recover, improving data transmission reliability. Service attribute labeling provides the foundation for accurate data identification and processing. On the server side, service attribute labeling enables accurate consolidation and restoration of block data into the original service data, avoiding data confusion and errors. Attribute labeling also facilitates subsequent data statistics and analysis, improving data processing accuracy and facilitating the monitoring of different service data. The block processing and consolidation and restoration mechanisms ensure data integrity. Even if some data blocks are lost or damaged during transmission, they can be repaired through retransmission, ensuring that the final consolidated and restored data is consistent with the original service data, thereby improving the real-time and integrity of service data transmission. This approach is highly scalable and can easily support the integration of new services. When a new service type emerges, simply define the corresponding service attributes and transmission parameters for it, and the client can follow the same process for data processing, transmission, and monitoring, eliminating the need for large-scale modifications to the entire system.

[0008] Optionally, the step of dynamically binding corresponding transmission parameters according to the service attributes includes: Obtain round-trip delay RTT and packet loss rate; determine the channel state index CSI based on the round-trip delay RTT and the packet loss rate, the channel state index CSI , is the packet loss rate; Mapping transmission parameters corresponding to current service attributes according to the channel state index CSI, the transmission parameters including fragment size and retransmission interval; Bind the service data with the transmission parameters.

[0009] By employing the above technical solution, clients can obtain round-trip time (RTT) and packet loss rate, enabling them to understand the current network transmission status in real time. The round-trip time (RTT) reflects the total time it takes for a data packet to be sent, received, and returned. A lower RTT indicates faster network response. The packet loss rate refers to the percentage of packets lost during network transmission compared to the total number of packets sent. Ideally, the packet loss rate should be 0%. These two metrics reflect network quality and stability from different perspectives, providing accurate data for subsequent processing. Determining the Channel State Index (CSI) based on the RTT and packet loss rate provides a comprehensive assessment of the current network channel status. The CSI takes into account network latency and packet loss, providing a more comprehensive picture of the actual network status. Compared to a single metric, the CSI provides more accurate network status information, helping clients better adjust their transmission strategies. The CSI maps transmission parameters corresponding to current service attributes, including fragment size and retransmission interval, to the CSI, enabling clients to dynamically adjust their transmission strategies based on network conditions. When network conditions are good (i.e., with a high CSI), the client can increase the fragment size and reduce the retransmission interval to improve data transmission efficiency. When network conditions are poor (i.e., with a low CSI), the client can reduce the fragment size and increase the retransmission interval to reduce the risk of data loss and ensure data transmission reliability. Different services have different requirements for network transmission. For example, real-time video services are more sensitive to latency, while file transfer services are more concerned with packet loss. By mapping transmission parameters corresponding to different service attributes based on the CSI, customized transmission solutions can be provided for different services, meeting diverse business needs and improving service quality. Dynamic adjustment of the fragment size and retransmission interval can reduce the number of data retransmissions.

[0010] Optionally, the step of processing the service data in blocks according to the transmission parameters includes: Assigning a unique task code to the business data according to the business attributes of the business data; Processing the service data in blocks according to the transmission parameters to form initial data blocks; Assigning corresponding sequence numbers to the initial data blocks according to the generation order of the initial data blocks; The sequence number corresponding to the initial data block is associated with the unique task code to generate a corresponding final data block.

[0011] By employing the above technical solution, business data is assigned unique task codes, enabling clear distinction between business data with different business attributes. In multi-business systems, a large amount of different types of business data coexist. Unique task codes can quickly identify the business type and task to which each piece of business data belongs, facilitating the monitoring of different business data. For example, on an e-commerce platform, order data, logistics data, user review data, and so on, each have their own unique task codes. This allows accurate location and processing of data for specific businesses during data processing, improving the accuracy and efficiency of data processing. Assigning sequence numbers to initial data blocks based on their generation order facilitates the orderly management of segmented business data. During data segmentation, the sequence number clearly identifies the position of each data block within the original business data, enabling the server to perform operations in the correct order when receiving and integrating segmented data, thus ensuring the integrity and accuracy of the business data. Associating the sequence numbers corresponding to the initial data blocks with the unique task codes to generate the final data blocks enhances data transmission reliability. During data transmission, problems such as data loss or data out of order may occur. However, this association mechanism allows the server to determine the business to which the data belongs based on the unique task code and then sort and consolidate the data blocks based on the sequence number. This ensures that even if data blocks encounter transmission errors, the original business data can be accurately restored. For example, if a data block arrives late during network transmission, the server can insert it into the correct location based on the association information, ensuring the integrity of the business data. The association between the unique task code and the sequence number also facilitates rapid error location and correction during data transmission. If the server discovers a missing or corrupted data block while receiving data, it can quickly determine the business data to which the data block belongs based on the unique task code and then locate the specific data block using the sequence number. It can then promptly request the client to retransmit the data block, improving data error correction efficiency and reducing the impact of data errors on the business system. Assigning unique task codes to business data is highly scalable and can adapt to the continuous expansion and changes of business. When a new business type is added to the system, it only needs to be assigned a new unique task code to integrate the new business data into the existing data processing and transmission system without requiring a major overhaul of the entire system. This approach enables the system to flexibly respond to business development and reduces the cost of system upgrades and maintenance.

[0012] Optionally, the real-time data monitoring and processing method further includes: Before sending the service data, detecting whether the current transmission network fluctuates; If so, the duration of the fluctuation is predicted based on the current fluctuation situation and combined with historical fluctuation data; generating a blank block group according to the fluctuation duration; The blank block group is associated with the first final data block in the business data.

[0013] By employing the above technical solution, the system detects network fluctuations before sending service data, enabling it to proactively identify potential data transmission risks. Network fluctuations are common and can be caused by a variety of factors, such as network congestion and equipment failure. Through real-time detection, the system can identify issues before data is transmitted, providing ample time for subsequent processing and avoiding data loss or interruptions caused by network fluctuations during the data transmission process. When network fluctuations are detected, the system predicts the duration of the fluctuation based on the current fluctuations and historical fluctuation data. This allows the system to more accurately assess network conditions. Historical fluctuation data contains information about past network fluctuations, such as frequency and duration. By analyzing this data and combining it with current fluctuations, the duration of the current fluctuation can be accurately predicted. Blank block groups are generated based on the duration of the fluctuations, and these blank blocks serve as a buffer. During periods of network fluctuations, blank blocks can fill gaps caused by transmission delays or loss, thereby enhancing data fault tolerance. Once the network returns to normal, service data can continue to be transmitted more smoothly, minimizing the impact of network fluctuations on data integrity and continuity. The blank block group is associated with the first and final data block in the service data, facilitating data recovery. By predicting network fluctuations and generating blank block groups to address them, the number of data retransmissions caused by network fluctuations is reduced. Data retransmissions not only increase network burden but also reduce data transmission efficiency. This method, however, avoids unnecessary data retransmissions by optimizing the data transmission structure, allowing service data to reach its destination faster and improving the data transmission efficiency of the entire service system.

[0014] Optionally, the step of generating a blank block group according to the fluctuation duration specifically includes: Inputting the transmission parameters corresponding to the current service data and the fluctuation duration into a pre-built blank block quantity model and a single blank block size model to obtain a predicted number of blank blocks to be generated and a predicted size of a single blank block; generating the blank block group according to the predicted number and the predicted size of a single blank block; The blank block quantity model is: , the single blank block size model is ; N is the number of blank blocks, is the size of a single blank block, R is the transmission speed, T is the fluctuation duration, I is the retransmission interval, and S is the fragment size.

[0015] By adopting the above technical solution, this model-based prediction method enables the generated blank block groups to more effectively respond to network fluctuations. Compared to traditional fixed blank block settings, this method can dynamically adjust based on real-time fluctuations and service data characteristics. When network fluctuations are mild, the model predicts fewer blank blocks of smaller size to avoid wasting resources. However, when network fluctuations are severe, sufficient blank blocks are generated to ensure stable data transmission, thereby improving the system's ability to cope with network fluctuations. Network conditions are dynamic, and the duration of fluctuations and transmission parameters also change accordingly. The model-based blank block group generation method can dynamically respond to these changes, adjusting the number and size of blank blocks in a timely manner. When network fluctuations intensify, the model adjusts the prediction results in real time, generating more or larger blank blocks. When network conditions improve, the generation of blank blocks is reduced, ensuring that the system maintains efficient and stable operation and enhancing system flexibility.

[0016] Optionally, the real-time data monitoring and processing method further includes: When retransmitting a data block, determining the number of remaining data blocks to be retransmitted based on the sequence number of the current data block to be retransmitted and the total number of data blocks; According to the service attribute mark of the service data, filter out key data blocks from the remaining data blocks to be retransmitted and mark them; The marked key data block is placed before the first non-key data block and the block processing is performed again.

[0017] By employing this technical solution, the number of remaining data blocks to be retransmitted is accurately determined, giving the system a clear understanding of retransmission tasks and enabling the appropriate allocation of retransmission resources and time based on actual conditions. For example, when the number of remaining data blocks to be retransmitted is small, resources can be concentrated to quickly complete retransmissions. If the number is large, a more rational retransmission plan can be developed to avoid blind retransmissions that waste resources or inefficiencies, thereby improving overall data retransmission efficiency. Critical data blocks may contain core business information and are crucial to the normal operation of the business. Prioritizing the retransmission of these critical data blocks allows the business system to quickly restore critical functions and minimize the impact of data loss on the business. For example, in a financial transaction system, prioritizing the retransmission of critical data blocks related to capital transactions allows for rapid resumption of normal transactions and avoids delays or errors caused by untimely data recovery. Pre-positioning and re-blocking critical data blocks ensures that critical data is restored and integrated first during the retransmission process. This helps ensure the integrity and accuracy of business data, as critical data often determines the correctness of business results. The system's dynamic adaptability is demonstrated by screening and processing critical data blocks based on the availability of remaining retransmitted blocks. For example, when network conditions are poor, critical data blocks can be more accurately identified and prioritized for retransmission to ensure that essential service functionality is not significantly impacted. When network conditions are good, data retransmission can be more comprehensive, improving data integrity. Re-blocking further optimizes and organizes data. Re-blocking of filtered and sorted data blocks can better meet network transmission and system processing requirements, improving data transmission and processing efficiency. This flexible data processing approach also helps the system better adapt to varying data sizes and service needs, enhancing its adaptability and scalability.

[0018] Optionally, after the re-blocking process, the following steps are included: Determining whether the marked key data block is located between non-key data blocks; If so, a link blank block is generated; Inserting the link blank block into the original position of the marked key data block; Merging the non-critical data block and the link blank block; If not, the non-critical data blocks are directly merged.

[0019] By employing the above technical solution, critical data blocks are ensured to remain relatively independent and intact within the data structure by determining whether a marked critical data block is located between non-critical data blocks and, if so, generating a linking blank block to insert in its place. Critical data blocks typically contain information critical to the business, such as transaction amounts and order numbers. Separating them from non-critical data blocks prevents errors or loss of critical data due to interference from non-critical data during data processing and transmission, ensuring the accuracy and security of critical data and providing a solid data foundation for the normal operation of the business. The generation and insertion of linking blank blocks connects critical and non-critical data blocks, making the entire data sequence more logically coherent. Even if critical data blocks are separated, the linking blank blocks allow the server to clearly identify the original data order and structure when receiving and integrating data, reducing the possibility of data confusion and errors and improving overall data continuity and readability. Merging non-critical data blocks with linking blank blocks, or directly merging non-critical data blocks, reduces the number of data blocks and simplifies the data processing process. Fewer data blocks mean less transmission overhead and shorter transmission time, improving data transmission efficiency. At the same time, the merged data blocks facilitate unified processing and storage on the server, reducing the system's processing burden and improving data processing efficiency. Merging non-critical data blocks and using linked blank blocks facilitates data maintenance and management. When updating or modifying data, only the critical data blocks and the merged non-critical data blocks need to be manipulated, reducing the workload of data maintenance. Furthermore, the presence of linked blank blocks facilitates data expansion and adjustment. The system can insert new data or adjust the data structure as needed in the location of the linked blank blocks, improving data management efficiency.

[0020] Secondly, the present application provides a real-time data monitoring and processing system for a multi-business system, which adopts the following technical solutions: A real-time data monitoring and processing system for a multi-business system, comprising: A data receiving module, used for receiving business data; A marking module, used to determine the business attributes of the business data and perform attribute marking; A parameter binding module, used to dynamically bind corresponding transmission parameters according to the service attributes; The data processing module is used to process and send the business data in blocks according to the transmission parameters; after the server receives the block data, it integrates the block data and restores it into business data.

[0021] In a third aspect, the present application provides a terminal that adopts the following technical solution: A terminal, comprising: Memory, which stores data real-time monitoring processing programs of multiple business systems; The processor is used to execute the program stored in the memory to implement the steps of the real-time data monitoring and processing method of the multi-business system.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the above-mentioned method for real-time monitoring and processing of data in a multi-business system.

[0023] In summary, this application has at least the following beneficial effects: By identifying and tagging service data's service attributes, the client can clearly distinguish different types of service data and dynamically adjust transmission parameters based on real-time service changes. For example, if a service's traffic suddenly increases, parameters such as the block size and retransmission interval can be adjusted accordingly to ensure efficient data transmission and processing, improving the system's adaptability and flexibility. Blocking service data reduces the amount of data transmitted at a time and reduces the probability of errors during transmission. When network fluctuations or congestion occur, smaller data blocks are easier to retransmit and recover, improving data transmission reliability. On the server side, service attribute tagging allows block data to be accurately reassembled and restored into the original service data, avoiding data confusion and errors. Attribute tagging also facilitates subsequent data statistics and analysis, improves data processing accuracy, and facilitates monitoring of different service data. The block processing and reassembly and restoration mechanisms ensure data integrity. Even if some data blocks are lost or damaged during transmission, they can be repaired through retransmission, ensuring that the final reassembled and restored data is consistent with the original service data, thereby improving the real-time and integrity of service data transmission. This method has good scalability and can easily support the access of new services. When a new service type appears, it is only necessary to define the corresponding service attributes and transmission parameters for it. The client can then perform data processing, transmission and corresponding monitoring according to the same processing flow without the need for large-scale transformation of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flowchart of an implementation method of Example 1 of the present application; Figure 2 It is a flowchart of the specific steps of S130; Figure 3 It is a flowchart for processing business data in blocks; Figure 4 It is a flowchart before sending business data; Figure 5It is a structural block diagram of another implementation method of the present application; Figure 6 It is a flowchart of the steps after the key data blocks are re-blocked; Figure 7 It is a structural block diagram of an embodiment of the system of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the appended drawings of the embodiments of the present invention. Figure 1 -Attached Figure 7 The technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] The first embodiment of the present application discloses a method for real-time monitoring and processing of data in a multi-business system, with a client as the execution subject. The client can be a computer, mobile terminal, user terminal or other device with data processing function. Figure 1 As an implementation of the data real-time monitoring processing method, the data real-time monitoring processing method may include S110-S140: S110, receiving business data; S120, determining the business attributes of the business data and marking the attributes; S130, dynamically binding corresponding transmission parameters according to service attributes; S140, processing and sending the business data in blocks according to the transmission parameters; after receiving the block data, the server integrates the block data and restores it into business data.

[0027] Specifically, business data can include financial data, human resources data, sales data, and so on. Appropriate communication protocols and interfaces can be used to receive business data. For example, if the business data originates from other devices or systems on the network, TCP or UDP sockets can be used for data reception. If the data is being communicated between local devices, serial communication can be used. The received business data is parsed, and its business attributes are determined based on its content, format, source, and other information. Business attributes can include a first-level data type (such as order data, inventory data, user information data), a second-level sensitivity (privacy / encryption / callability), a third-level importance, and a fourth-level timeliness. These business attributes are then tagged with corresponding attribute tags for subsequent processing. Based on the business attributes, corresponding transmission parameters are searched and bound from a pre-set parameter table. The parameter table includes historical channel state indices, historical business attributes, and historical transmission parameters. Each business attribute corresponds to multiple channel state indices, and each channel state index corresponds to a transmission parameter. Transmission parameters include fragment size and retransmission interval.

[0028] Reference Figure 2 Specifically, S130 may include S131-S134: S131, obtaining the round-trip delay RTT and packet loss rate; S132, determining a channel state index CSI based on the round-trip delay RTT and the packet loss rate; S133, mapping transmission parameters corresponding to the current service attributes according to the channel state index CSI; S134: Bind the service attributes with the transmission parameters.

[0029] Specifically, the client sends a data packet to the server and records the sending time. This can be repeated multiple times to calculate the average RTT. After receiving the data packet, the server immediately sends a response packet back to the client. When the client receives the response packet, it records the reception time. The sending time is subtracted from the receiving time to obtain the RTT for that packet. After multiple packets are sent, the average of these RTTs is calculated to more accurately reflect network latency.

[0030] Specify the number of packets sent by the client, count the number of successfully received packets on the server, and use (number of packets sent - number of successfully received packets) / number of packets sent to calculate the current packet loss rate.

[0031] According to the round-trip delay RTT and packet loss rate, the channel state index CSI is calculated. , is the packet loss rate. After determining the service attributes of the service data, the same historical service attributes are located. Then, based on the determined channel state index (CSI), the same historical channel state index (CSI) is located. The historical transmission parameters corresponding to the historical channel state index (CSI) are the current transmission parameters. The service data is then processed in blocks based on the current transmission parameters.

[0032] Reference Figure 3 The block processing steps may include S210-S240: S210, assigning a unique task code to the business data according to the business attributes of the business data; S220, processing the service data into blocks according to the transmission parameters to form initial data blocks; S230, assigning corresponding sequence numbers to the initial data blocks according to the generation order of the initial data blocks; S240 , associating the sequence number corresponding to the initial data block with the unique task code to generate a corresponding final data block.

[0033] Specifically, a feature code can be defined based on the primary data type in the business attributes. For example, if the data type is order data, the feature code could be "OD." A unique task code is then generated by combining information such as the timestamp and random number. For example, if the current time is 2025-04-08 21:34:54 and the business data is order data, the generated task code could be "OD-20250408213454-789," where "789" is a randomly generated number to ensure the uniqueness of the task code.

[0034] The business data is then divided into blocks according to the shard size in the transmission parameters. If the size of the business data is exactly an integer multiple of the data block size, it is directly divided into equal parts; if it is not an integer multiple, the size of the last data block is smaller than the set data block size. During the block processing, each initial data block is assigned a sequence number according to the order of generation. The sequence number starts from 1 and increases. For example, the sequence number of the first generated initial data block is 1, the second is 2, and so on. The sequence number of each initial data block is associated with the unique task code previously generated for the business data. The task code and sequence number information can be added to the header of the initial data block to form the final data block. After the server receives the final data block, it integrates the block data into the business data according to the sequence number.

[0035] Reference Figure 4 After block processing, S310-S350 can be executed: S310, before sending service data, detecting whether the current transmission network fluctuates; S320, if not, send directly; S330, if yes, then predict the duration of the fluctuation based on the current fluctuation situation and combined with historical fluctuation data; S340, generating a blank block group according to the fluctuation duration; S350: Associating the blank block group with the first final data block in the business data.

[0036] Specifically, network monitoring software can be used to monitor client network fluctuations. This software can display client network latency and bandwidth usage in graphical form, allowing for quick identification of network fluctuations. When network fluctuations are detected, the current fluctuation conditions, such as latency and packet loss rate, are recorded. Historical fluctuation data, including information such as the start and end time of the fluctuation, latency change during the fluctuation period, and packet loss rate, are stored in a historical fluctuation database. Time series analysis algorithms (such as ARIMA and LSTM) are used to model the historical fluctuation data. The current fluctuation conditions are then used as input to predict the duration of the fluctuation using the trained model. The transmission parameters corresponding to the current service data and the duration of the fluctuation are input into a pre-built blank block number model and a single blank block size model to obtain the predicted number of blank blocks required and the predicted size of each blank block. Based on these predicted numbers and predicted sizes, blank block groups are generated. Blank blocks can be padded with all zeros or specific padding characters. Within the service data, the first and final data block is located based on the data structure and identification information. The generated blank block group is then associated with the first and final data block. The association can be at the head of the data block, or by inserting a blank block group directly before the first final data block.

[0037] The blank block quantity model is , the single blank block size model is ; N is the number of blank blocks, is the size of a single blank block, R is the transmission speed, T is the fluctuation duration, I is the retransmission interval, and S is the fragment size. In addition, .

[0038] Reference Figure 5 As another implementation of the real-time data monitoring process, the real-time data monitoring process may further include S410-S430: S410, when retransmitting a data block, determining the number of remaining data blocks to be retransmitted based on the sequence number of the current data block to be retransmitted and the total number of data blocks; S420, based on the service attribute tag of the service data, filter out key data blocks from the remaining data blocks to be retransmitted and mark them; S430: Place the marked key data block before the first non-key data block and re-perform the block division process.

[0039] Specifically, during data segmentation and transmission, it's necessary to record the sequence number of each data block and the total number of data blocks. This information can be stored in a data structure (such as an array, list, or database table). For example, an array can be used to store the status of each data block (transmitted, pending retransmission, etc.), with the array index corresponding to the data block's sequence number. When a data block needs to be retransmitted, a simple mathematical calculation is performed based on the sequence number of the current data block to be retransmitted and the total number of data blocks to determine the number of remaining data blocks to be retransmitted. The calculation formula is: Number of remaining data blocks to be retransmitted = Total number of data blocks - Sequence number of the current data block to be retransmitted + 1. Based on business requirements, define association rules between business attribute tags and key data blocks. For example, data blocks with business attribute tags such as "critical order data" (level 3 importance) and "privacy" (level 2 sensitivity) are defined as key data blocks. Specific tags are added to selected key data blocks to facilitate subsequent processing. Tags can be additional fields or identification bits, such as adding a "key" flag to the header of a key data block. The remaining data blocks to be retransmitted are divided into two parts: key data blocks and non-key data blocks. The marked key data block is moved before the first non-key data block, and then the key data block is divided into blocks. The size of the key data block is smaller than the size of a single data block.

[0040] Reference Figure 6 After the key data blocks are re-blocked, S510-S550 can be executed: S510, determining whether the marked key data block is located between non-key data blocks; S520, if yes, generate a link blank block; S530, inserting the link blank block into the original position of the marked key data block; S540, merging the non-critical data blocks and the link blank blocks; S550: If not, directly merge the non-critical data blocks.

[0041] Specifically, traverse from the starting position of the data structure storing the data blocks, and check whether the data blocks before and after each key data block are non-key data blocks. If there is a key data block whose previous and next data blocks are both non-key data blocks, then the key data block is determined to be located between non-key data blocks. Then, based on the size of the key data block, use specific padding characters (such as all zeros) to fill the blank blocks to make them the same size to form linked blank blocks; based on the previously stored data block position information, find the original position of the marked key data block in the data structure, and insert the generated linked blank block at this position. According to the order of the data blocks, the filtered non-key data blocks and linked blank blocks are merged into a new data block in sequence. The merge operation can be implemented using string splicing, byte array splicing, etc.

[0042] If the key data block is located before or after the non-key data block, the non-key data block can be directly merged.

[0043] Based on the above method embodiment, the second embodiment of the present application discloses a real-time data monitoring and processing system for a multi-business system. Figure 7 As an implementation of the data real-time monitoring and processing system, the data real-time monitoring and processing system is embedded in the client and may include: A data receiving module, used for receiving business data; A marking module is used to determine the business attributes of business data and mark the attributes; Parameter binding module, used to dynamically bind corresponding transmission parameters according to business attributes; The data processing module is used to process and send the business data in blocks according to the transmission parameters; after the server receives the block data, it integrates the block data and restores it into business data.

[0044] The modules of the real-time data monitoring and processing system for the multi-business system correspond one to one with the real-time data monitoring and processing method for the multi-business system, and no further details will be given here.

[0045] The third embodiment of the present application provides a terminal. As an implementation of the terminal, the terminal may include: a memory and a processor; wherein, The memory is used to store the data real-time monitoring processing program of the multi-business system; The processor is used to execute the program stored in the memory to implement the steps of the above-mentioned data real-time monitoring and processing method of the multi-business system.

[0046] The memory may be communicatively connected to the processor via a communication bus, and the communication bus may be an address bus, a data bus, a control bus, or the like.

[0047] In addition, the memory may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0048] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0049] A fourth embodiment of the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned method for real-time data monitoring and processing of a multi-business system.

[0050] Computer-readable storage media can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media. Available media can include magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0051] The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Unless otherwise specified, any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features. In other words, unless otherwise specified, each feature is merely an example of a series of equivalent or similar features.

Claims

1. A method for real-time monitoring and processing of data in a multi-business system, characterized in that: The client is the execution subject, including: Receive business data; Determining the business attributes of the business data and performing attribute marking; Dynamically bind corresponding transmission parameters according to the service attributes; According to the transmission parameters, the business data is processed in blocks and sent; after receiving the block data, the server integrates the block data and restores it into the business data.

2. The method for real-time monitoring and processing of data in a multi-business system according to claim 1, characterized in that: The step of dynamically binding corresponding transmission parameters according to the service attributes includes: Get the round-trip time (RTT) and packet loss rate; According to the round trip delay RTT and the packet loss rate, a channel state index CSI is determined. , is the packet loss rate; Mapping transmission parameters corresponding to current service attributes according to the channel state index CSI, the transmission parameters including fragment size and retransmission interval; Bind the service data with the transmission parameters.

3. The method for real-time monitoring and processing of data in a multi-business system according to claim 2, characterized in that: The step of processing the service data in blocks according to the transmission parameters includes: Assigning a unique task code to the business data according to the business attributes of the business data; Processing the service data in blocks according to the transmission parameters to form initial data blocks; Assigning corresponding sequence numbers to the initial data blocks according to the generation order of the initial data blocks; The sequence number corresponding to the initial data block is associated with the unique task code to generate a corresponding final data block.

4. The method for real-time monitoring and processing of data in a multi-business system according to claim 3, characterized in that: The data real-time monitoring and processing method further includes: Before sending the service data, detecting whether the current transmission network fluctuates; If so, the duration of the fluctuation is predicted based on the current fluctuation situation and combined with historical fluctuation data; generating a blank block group according to the fluctuation duration; The blank block group is associated with the first final data block in the business data.

5. The method for real-time monitoring and processing of data in a multi-business system according to claim 4, characterized in that: The step of generating a blank block group according to the fluctuation duration specifically includes: Inputting the transmission parameters corresponding to the current service data and the fluctuation duration into a pre-built blank block quantity model and a single blank block size model to obtain a predicted number of blank blocks to be generated and a predicted size of a single blank block; generating the blank block group according to the predicted number and the predicted size of a single blank block; The blank block quantity model is: , the single blank block size model is ; N is the number of blank blocks, is the size of a single blank block, R is the transmission speed, T is the fluctuation duration, I is the retransmission interval, and S is the fragment size.

6. The method for real-time monitoring and processing of data in a multi-business system according to claim 1, characterized in that: The data real-time monitoring and processing method further includes: When retransmitting a data block, determining the number of remaining data blocks to be retransmitted based on the sequence number of the current data block to be retransmitted and the total number of data blocks; According to the service attribute mark of the service data, filter out key data blocks from the remaining data blocks to be retransmitted and mark them; The marked key data block is placed before the first non-key data block and the block processing is performed again.

7. The method for real-time monitoring and processing of data in a multi-business system according to claim 6, characterized in that: After the re-blocking process, the following steps are included: Determining whether the marked key data block is located between non-key data blocks; If so, a link blank block is generated; Inserting the link blank block into the original position of the marked key data block; Merging the non-critical data block and the link blank block; If not, the non-critical data blocks are directly merged.

8. A method, system, terminal and medium for real-time monitoring and processing of data in a multi-business system, characterized in that: include: A data receiving module, used for receiving business data; A marking module, used to determine the business attributes of the business data and perform attribute marking; A parameter binding module, used to dynamically bind corresponding transmission parameters according to the service attributes; A data processing module, configured to process and send the service data in blocks according to the transmission parameters; After receiving the block data, the server integrates the block data and restores it into business data.

9. A terminal, characterized in that: include: Memory, which stores data real-time monitoring processing programs of multiple business systems; The processor is used to execute the program stored in the memory to implement the steps of the real-time data monitoring and processing method of the multi-business system as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer program is stored which can be loaded by a processor and executes the real-time data monitoring and processing method of the multi-business system according to any one of claims 1 to 7.

Images