A lightweight method for reporting massive amounts of data collection points from cloud phones

Abstract

The application discloses a cloud mobile phone mass point data lightweight reporting method, comprising: classifying original point data according to business value and timeliness, combining reporting strategies and network states to aggregate and compress to generate pre-processing batches; converting to compact byte streams through binary coding by field name mapping, enumeration value coding and incremental difference recording; generating load data by removing redundancy through default value omission, adjacent difference and common prefix, and encapsulating protocol headers containing version number, compression mode and other meta information into self-explaining data packets; adaptively scheduling reporting according to real-time network detection and user activity state, dynamically adjusting transmission timing and retry strategy, and completing cleaning, aggregation and hierarchical storage on the server side. The application reduces transmission load and storage cost through hierarchical collection and end-side deep compression, guarantees real-time performance and reachability of core data by combining adaptive scheduling and hierarchical retry, and reduces the occupation of cloud mobile phone main business resources.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a lightweight reporting method for massive amounts of data collected by cloud mobile phones. Background Technology

[0002] With the maturity of cloud computing and virtualization technologies, cloud phones, a technology that runs a complete Android operating system in a virtualized and service-oriented form on cloud servers, are being widely used in scenarios such as game streaming, application hosting, mobile office, automated testing, and marketing promotion. In the operation and service of cloud phones, the collection and reporting of event tracking data has become a fundamental and crucial task in order to accurately understand user behavior, evaluate product functionality, monitor virtual device performance, and quickly locate online faults.

[0003] Compared to traditional mobile applications, cloud phone event tracking data has its unique characteristics. First, a single cloud phone instance continuously generates a massive data stream during operation, including user interaction events, application lifecycle events, system performance metrics, and debugging logs. When tens of thousands of cloud phone instances run concurrently, the scale of event tracking data reaches an astonishing level. Second, data needs to be transmitted from the cloud virtual device to the analysis server via the network. This link may traverse virtual networks and the public internet, and is subject to the shared bandwidth of the cloud server, resulting in fluctuating and unstable network conditions. Furthermore, frequent, high-volume data reporting consumes valuable virtual network I / O resources, potentially impacting the real-time performance and smoothness of the cloud phone's main services, and increasing operating costs.

[0004] Existing technologies have explored two main aspects. One approach proposes a queue-based method for reporting event tracking data. This involves creating a data storage queue on the terminal side, setting a running node for each event tracking data point, and storing the data in the queue sequentially. When the running conditions are met, the data is retrieved from the queue in order for reporting. This method aims to alleviate the concurrency pressure of real-time reporting by scheduling the reporting sequence and to ensure the order of data reporting using the first-in, first-out (FIFO) characteristic of the queue. However, this approach lacks differentiated processing of the data's inherent value, treating critical business events the same as ordinary performance indicators. This makes it difficult to prioritize the real-time performance and reliability of core data during network congestion. Furthermore, this approach does not effectively lightweight the original event tracking data, directly transmitting the original redundant text format data, resulting in low network transmission efficiency, making it particularly unsuitable for cloud phone scenarios with massive amounts of data.

[0005] Another approach focuses on expanding the data collection dimensions in the cloud phone environment. It collects deeper-level system events by intercepting system broadcasts and monitoring window focus changes, and reports the data by periodically polling local files. While this approach addresses the breadth of data collection in cloud phone scenarios, its reporting mechanism is relatively crude and inefficient, relying on fixed polling intervals and plaintext transmission, lacking adaptive scheduling based on network conditions. Furthermore, the data is simply stored on the terminal side without any aggregation, compression, or encoding preprocessing, resulting in a high transmission load. This makes it unsuitable for core business monitoring with high real-time requirements and struggles to handle data surges under large-scale instances.

[0006] In summary, existing technical solutions either focus on the timing control of reporting in general scenarios or on the collection of specific data from cloud phones, but neither has been able to build a complete, lightweight reporting system for the massive amounts of embedded data from cloud phones from an end-to-end perspective. Summary of the Invention

[0007] The main objective of this invention is to provide a lightweight method for reporting massive amounts of data collected by cloud mobile phones.

[0008] Another objective of this invention is to provide a lightweight reporting device for massive amounts of data collected by cloud mobile phones.

[0009] The third objective of this invention is to provide a computer device.

[0010] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0011] To achieve the above objectives, a first aspect of the present invention proposes a lightweight reporting method for massive amounts of cloud phone data tracking information, comprising: Based on business value and timeliness requirements, the collected raw data points are classified and processed, and reporting strategies for different levels of data are determined. The classified data points are then intelligently aggregated and compressed according to the reporting strategies and the current network status to generate pre-processed data batches. The preprocessed data batch is binary encoded based on a predefined protocol. By digitally mapping field names, replacing enumerated values ​​with codes, and recording incremental differences in status data, the data batch is converted into a compact binary byte stream. The compact binary byte stream is subjected to field-level redundancy elimination processing. The payload data is generated by omitting default value fields, recording the difference between adjacent data, and extracting common prefix and suffix separation methods. The payload data is then encapsulated with a protocol header containing protocol metadata to form a self-interpreting data packet. Based on real-time network detection results and user activity status, the self-interpreting data packets are adaptively scheduled and reported. The transmission timing and retry strategy are dynamically adjusted according to the data level and network conditions. The successfully reported data streams are cleaned, aggregated, and stored hierarchically on the server side.

[0012] In one embodiment of the present invention, the step of intelligently aggregating and compressing the graded embedded data according to the reporting strategy and the current network status to generate a preprocessed data batch includes: Before the data is stored in the cloud phone virtual environment, the hierarchical results and the current network conditions are integrated in real time in the terminal memory buffer to dynamically determine the packaging timing and batch composition strategy. The batch processing window size is determined based on data priority and network speed. The time window is then used to aggregate the graded data points according to the batch processing window size to generate a preliminary aggregated data batch. Semantic compression is performed on the aggregated data batches. Common fields are preprocessed using columnar storage reorganization and dictionary encoding. Specific fields are further compressed using a general compression algorithm to generate preprocessed data batches.

[0013] In one embodiment of the present invention, the process of performing field-level redundancy elimination on the compact binary byte stream, which generates load data by omitting default value fields, recording differences between adjacent data, and extracting common prefixes and suffixes, includes: The field default value omission operation is performed. During the serialization process, for fields whose values ​​are the same as the default values ​​defined in the protocol, the encoding of the field is directly omitted in the payload so that the receiving end can automatically fill in the default value during parsing. Perform adjacent record difference recording operation. For numeric fields between consecutive records in the same batch, starting from the second record, store the difference between the current value of the field and the value of the previous record. Perform a common prefix extraction operation. For long string prefixes that are common to all records in the same batch, extract them in the payload header and store them only once. In each individual record, store only the difference suffix after removing the common prefix, and generate payload data with redundancy elimination completed.

[0014] In one embodiment of the present invention, the step of combining the protocol header containing protocol metadata to form a self-interpreting data packet includes: For load data that has undergone field-level redundancy elimination processing, a lightweight fixed-length protocol header is appended before the load body to form a self-interpreting data packet; Write all the metadata necessary for the receiver to independently and correctly parse the payload into the protocol header, including the protocol version number, data compression method identifier, sampling and aggregation strategy description, and start and end timestamps of the data within the batch.

[0015] In one embodiment of the present invention, the adaptive scheduling and reporting of the self-interpreting data packets based on real-time network detection results and user activity status includes: Before uploading, the current network status is detected in real time, and differentiated scheduling rules are executed based on the detection results. When the network status is good, non-urgent data is allowed to be uploaded in batches. When the network status is weak, the core protection and traffic control strategy is activated, allowing only high-priority data to be uploaded immediately, and the generated non-high-priority data is temporarily stored in the local cache queue to wait for the network to recover. By sensing user activity levels, and taking advantage of periods of inactivity when the device screen is off or the application is in the background, historical backlog data that does not have high real-time requirements can be uploaded in batches.

[0016] In one embodiment of the present invention, the step of dynamically adjusting the transmission timing and retry strategy according to data level and network conditions includes: Implement a progressive delay retry and degradation processing strategy associated with the data level for failed data packets; For failures in reporting high-priority data, a retry strategy with short intervals and high priority is adopted. For failures in reporting low-priority data, a backoff algorithm is used to gradually extend the retry interval. Set a total quota limit for data in the local cache. When the amount of cached data exceeds the quota limit, discard the oldest and lowest-level data in the cache according to the first-in-first-out (FIFO) strategy.

[0017] In one embodiment of the present invention, the step of cleaning, aggregating, and hierarchically storing the successfully reported data stream on the server side includes: In the real-time computing layer, the received raw event stream is rapidly aggregated within a preset window according to predefined aggregation logic, and the aggregated statistical indicators generated are written into the online analysis system in real time. Implement differentiated storage strategies for data with different values ​​and access patterns. Automatically route and store data in storage media with different performance and cost based on data popularity, granularity and purpose, and implement corresponding data lifecycle management strategies.

[0018] To achieve the above objectives, a second aspect of the present invention provides a lightweight reporting device for massive amounts of cloud phone data, comprising: The data classification and preprocessing module is used to classify the collected raw data points according to business value and timeliness requirements, determine the reporting strategy for different levels of data, and intelligently aggregate and compress the classified data points according to the reporting strategy and the current network status to generate preprocessed data batches. The binary encoding module is used to perform binary encoding processing on the preprocessed data batch based on a predefined protocol. It converts the data batch into a compact binary byte stream by digital mapping of field names, code replacement of enumeration values, and recording of incremental differences in status data. The redundancy elimination and encapsulation module is used to perform field-level redundancy elimination processing on the compact binary byte stream. It generates payload data by omitting default value fields, recording the difference between adjacent data, and extracting common prefix and suffix separation methods, and encapsulates it with a protocol header containing protocol metadata to form a self-interpreting data packet. The adaptive reporting module is used to adaptively schedule and report the self-interpreting data packets based on real-time network detection results and user activity status. It dynamically adjusts the transmission timing and retry strategy according to the data level and network conditions, and cleans, aggregates and stores the successfully reported data streams on the server.

[0019] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement a lightweight reporting method for massive cloud phone data collection points as described in the first aspect embodiment.

[0020] To achieve the above objectives, the fourth aspect of this application proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a lightweight reporting method for massive cloud phone data collection points as described in the first aspect embodiment.

[0021] The embodiments of the present invention have the following beneficial effects: This invention maximizes data volume reduction in both structure and content through a progressive deep compression process, significantly reducing network bandwidth consumption and cloud storage costs when reporting from massive cloud phone clusters. Through an adaptive scheduling strategy based on value grading and network status, batch transmission is efficient when the network is good, while ensuring the real-time and delivery of high-priority core data only when the network is congested, ensuring reliable support for real-time business monitoring and decision-making even with limited overall network resources. Intelligent batch processing and idle-time transmission mechanisms proactively sense user activity levels, centrally reporting non-urgent data during periods when users are unaware, effectively avoiding interference with cloud phone operation smoothness and frame rate during peak reporting periods, thus ensuring a superior end-user experience. Real-time hot updates of sampling and aggregation strategies are achieved through a dynamic configuration center on the server side, and the self-interpreting data packet protocol completely decouples client-server version dependencies, improving the maintainability and stability of the data system. Attached Figure Description

[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a lightweight reporting method for massive data collection points on cloud phones, provided in an embodiment of the present invention; Figure 2 This is a structural diagram of a lightweight reporting device for massive cloud phone data points provided in an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] The following describes a lightweight reporting method for massive data collection points on cloud phones according to an embodiment of the present invention, with reference to the accompanying drawings.

[0026] Example 1 This embodiment provides a lightweight reporting method for massive amounts of event tracking data from cloud phones, such as... Figure 1 As shown, the method includes the following steps: S1. Based on business value and timeliness requirements, the collected raw data points are classified and processed, and the reporting strategies for different levels of data are determined. Based on the reporting strategies and the current network status, the classified data points are intelligently aggregated and compressed to generate pre-processed data batches.

[0027] S2, based on a predefined protocol, performs binary encoding processing on the preprocessed data batch, converting the data batch into a compact binary byte stream through field name digital mapping, enumeration value code replacement, and status data incremental difference recording.

[0028] S3, perform field-level redundancy elimination processing on the compact binary byte stream, generate payload data by omitting default value fields, recording the difference between adjacent data and extracting common prefix and suffix separation methods, and encapsulate it with a protocol header containing protocol metadata to form a self-interpreting data packet.

[0029] S4. Based on real-time network detection results and user activity status, the self-interpreting data packets are adaptively scheduled and reported. The transmission timing and retry strategy are dynamically adjusted according to the data level and network conditions. The successfully reported data streams are cleaned, aggregated, and stored hierarchically on the server side.

[0030] This embodiment establishes a four-level data tracking strategy based on value and urgency. Before data is reported, it performs local preprocessing through intelligent dynamic batch processing and semantic compression technology. Then, it sequentially performs binary encoding, field-level redundancy elimination, and self-explanatory data packet encapsulation. Finally, it performs adaptive scheduling and reporting based on real-time network conditions and data priority. The data is then cleaned, aggregated, and stored hierarchically on the server side, forming a lightweight data reporting system that spans the entire chain from the terminal to the server.

[0031] Specifically, step S1 includes: First, a four-level tracking system is established based on the business value and timeliness requirements of tracking points.

[0032] P0 level is defined as key behavior tracking points, including behaviors that directly affect core business indicators and revenue, such as login, payment, and access to core functions, and adopts 100% full real-time reporting.

[0033] Level P1 is defined as user path tracking, including interactive events such as page browsing and function clicks used to analyze user behavior paths and product experience, and uses dynamic sampling and reporting of 10% to 30%.

[0034] To accurately calculate the dynamic sampling rate of P1-level user path tracking points, the real-time sampling rate is determined using the following formula, taking into account current business priorities and terminal network conditions:

[0035] in, The real-time sampling rate is P1 level. The base sampling rate is 10%-30%, and N is the current terminal network bandwidth. This represents the average network bandwidth across the entire network.

[0036] Level P2 is defined as performance monitoring points, including indicators that reflect the operating status of applications and systems, such as frame rate, network latency, and CPU utilization. It adopts a local client-side bucket aggregation reporting strategy, that is, the average value, quantile and other statistical values ​​are pre-calculated on the cloud mobile client according to a preset time window before being reported.

[0037] P3 level is defined as a debug log entry point, which is triggered to report when the user actively reports a problem or when the server dynamically issues a specific collection command.

[0038] The aforementioned four-level data tracking strategy clarifies the processing priority, reporting frequency, and granularity of different data types from the source of data collection. To achieve flexibility and adaptability, the dynamic sampling rate of P1-level data, the aggregation window size and statistical method of P2-level data, and the triggering conditions for each level are all dynamically adjusted and updated in real time through a unified configuration center issued by the server. The server's dynamic configuration and adjustment of data tracking parameters are based on real-time decisions made according to the current business development stage and priorities, the real-time network status of the terminal, user value stratification, and other operational strategy factors.

[0039] Next, before the event tracking data leaves the cloud phone's virtual environment, efficient local preprocessing is performed to transform the raw, discrete event tracking data into data units with a better structure, smaller size, and better suitability for network transmission on the terminal side. First, intelligent aggregation is implemented, introducing the aforementioned intelligent dynamic batch processing based on priority and network status. In the terminal's memory buffer, the system integrates two key decision-making criteria in real time: one is the real-time data priority from the hierarchical results, and the other is the currently detected network status. Combined with the global scheduling decision intention, the optimal packaging timing and batch composition strategy are dynamically determined. The batch processing window size needs to be dynamically adjusted based on data priority and network status. The basic calculation logic is as follows:

[0040] in, To set the size of the real-time batch processing window, The maximum window threshold, The current data priority coefficients are (P0=1.0, P1=0.7, P2=0.5, P3=0.1). , where is the highest priority coefficient (1.0), and N is the current network speed. To optimize network speed, when network conditions are good, increase the batch processing window size for P1-level user path tracking and P2-level performance monitoring tracking, accumulating more data before compression and reporting; when network conditions are weak, use small window or immediate reporting for P0-level critical behavior tracking to ensure data real-time performance; for P3-level debug log tracking, do not perform batch processing, but directly encapsulate and report when trigger conditions are met.

[0041] After data is batched by intelligent batch processing, semantic compression is performed. A differentiated processing approach based on data structure is adopted. First, highly structured and highly repetitive common fields are identified and processed. These fields are reorganized using columnar storage and preprocessed using techniques such as dictionary encoding. Second, highly redundant unique fields for each event within the batch are further compressed using a general, efficient compression algorithm. To quantify the effect of semantic compression (dictionary encoding + general compression), the compression ratio of a single batch of data is calculated using the following formula:

[0042] Where C is the semantic compression ratio. The data volume before compression. The final data batch is generated after aggregation and compression, representing the volume of semantically compressed data.

[0043] Specifically, step S2 includes: To effectively reduce network transmission load, the data batches output from step S1, which have undergone aggregation and compression, are optimized in terms of transmission medium and serialization format, and binary encoding is adopted as the unified medium for data transmission.

[0044] Within the framework of the aforementioned binary encoding scheme based on a predefined protocol, a field-level simplified encoding strategy is implemented. Readable field names in the data are replaced with short, predefined numeric identifiers in the protocol descriptor of the binary encoding scheme. A compact binary encoding format is adopted, using variable-length encoding for integers and a compact format for floating-point numbers. For discrete value fields, their enumeration mapping relationships are predefined in the protocol descriptor of the binary encoding scheme, and short numeric enumeration codes are directly used in the data instead of lengthy strings. For state data that needs to be reported periodically and changes slowly, an incremental difference reporting strategy is adopted: within the same batch, subsequent reports only transmit the fields that have changed compared to the previous state and their new values, thus completing the conversion from logically structured data batches to compact binary byte streams.

[0045] The protocol descriptor includes: a field definition table, which records the numeric identifier, field type, default value, and whether each field can be omitted; an enumeration mapping table, which records the correspondence between the string value and the numeric enumeration code of each enumeration field; a data type table, which defines the binary encoding format of data types such as integer, floating point, string, array, and object; and version management information, which records the current protocol version number and compatibility descriptions with historical versions.

[0046] Specifically, step S3 includes: After completing the binary encoding of a single data record in step S2, the payload data that will constitute the main body of the final network data packet undergoes a deeper level of encoding and redundancy elimination processing.

[0047] First, the default value omission operation is performed. For fields in the data structure that have explicitly defined default values, optimization is performed. During serialization, if the actual value of a field is exactly the same as its defined default value, the encoding and storage of that field are directly omitted in the payload of that record. When the receiving end parses, for such fields missing in the payload, it will automatically fill them with their protocol-defined default values.

[0048] Secondly, the difference recording operation between adjacent records is performed. For numerical fields between consecutive records within the same batch, compression is performed using their sequence correlation. The numerical fields mainly include two types: one is a strictly monotonically increasing timestamp sequence, and the other is a slowly changing or continuous index sequence. During processing, instead of storing the complete original value of the field for each record independently, starting from the second record, the difference between the current value of the field and the value of the previous record of the same field is stored.

[0049] Finally, a common prefix extraction operation is performed. For fields containing the same long string prefix that are present in all records within the same batch, optimization is performed. During processing, the common prefix is ​​first extracted from the batch header of the payload and stored only once. Then, in the corresponding field of each individual record, only the difference suffix after removing the common prefix is ​​stored. When parsing each record, the receiving end reassembles the suffix stored in the record with the common prefix stored in the batch header to restore the complete field value, generating payload data with redundancy elimination completed.

[0050] Next, the payload data that has undergone field-level redundancy elimination processing is encapsulated at the application layer to form a complete self-interpreting data packet. This data packet consists of two parts: first, the payload body carrying the actual data of all records in the batch; and second, a lightweight, fixed-length protocol header appended to the payload body.

[0051] The protocol header contains all the metadata necessary for the receiver to independently and correctly parse the payload. This metadata includes: the protocol version number of the data structure, the data compression method identifier, the description of the sampling and aggregation strategy, and the start and end timestamps of the data within the batch. By carrying the complete protocol header, any server component, upon receiving a data packet, can complete the decompression, parsing, validity verification, and subsequent distribution of the data solely based on the metadata carried by the data packet itself. This effectively solves the problem of inconsistent data link parsing caused by the coexistence of old and new client versions, canary releases, or asynchronous upgrades of server components.

[0052] Specifically, step S4 includes: Before uploading, the system uses a lightweight network probe to detect the current network status in real time and executes differentiated scheduling rules based on the detection results: when the network status is good, backlogged, non-urgent P1 / P2 level data is allowed to be uploaded in large batches with a relaxed upload process; when the network status is weak, a core protection and traffic control strategy is activated, allowing only P0 level critical data to attempt to upload immediately, while temporarily storing the generated non-P0 level data in a local cache queue to wait for the network to recover. To reduce resource contention and potential perceptual interference for users when using the application, the system senses the user's activity status and takes advantage of inactive periods such as when the device screen is off or the application is in the background to centrally and in batches upload historical backlog data at the P1 and P2 levels that do not have high real-time requirements. For failed data packets, a progressive delayed retry and degradation processing strategy strongly correlated with data level is implemented. Specifically: for failed reporting of P0 level critical data, a short-interval, high-priority retry strategy is adopted, and retry is performed immediately or quickly; for failed reporting of low-level data such as P1 and P2, a backoff algorithm is used to gradually extend the retry interval. This strategy includes setting a total quota limit for data in the local cache. When the amount of cached data continues to grow and exceeds the quota limit, the system will discard the oldest and lowest-level data in the cache according to the FIFO strategy. After P1 and P2 level data reporting fails, the backoff algorithm is used to calculate the retry interval. The basic formula is as follows:

[0053] in, The interval for the k-th retry is... The initial retry interval is 1 second (default value), k is the number of retries (k≥1) and does not exceed the preset maximum number of retries.

[0054] On the server side, it receives continuous data streams from the entire processing chain and successfully reported data. For core business metrics that require high-frequency observation but do not need to be permanently retained at the original detailed granularity, such as business monitoring and real-time dashboards, the system performs rapid aggregation calculations on the received raw event streams in the real-time computing layer within a sliding or fixed window according to predefined aggregation logic, and writes the calculated aggregated statistical metrics into the online analysis system in real time.

[0055] Differentiated storage strategies are implemented for data with different values ​​and access patterns. The system automatically routes and stores data in storage media with different performance and costs based on the data's popularity, granularity, and purpose, forming a hierarchical storage system. For example, the latest detailed data and frequently queried aggregation results are stored in high-performance hot storage; earlier data with lower access frequency can be moved to warm storage; and historical detailed data used only for compliance audits or long-term trend analysis is archived in low-cost cold storage. At the same time, a data lifecycle management strategy is implemented to define the data retention period based on business and compliance requirements, and automatically clean up or delete the data from the corresponding storage after the expiration.

[0056] After the above streaming processing, aggregation computing, and hierarchical storage, the original terminal event tracking data is transformed into different forms of data products serving different scenarios, including cleaned high-quality detailed data, near real-time aggregated metrics, and historical datasets for in-depth analysis. This step serves as the terminal of the entire data pipeline, and its output directly drives business analysis, product iteration, operational decision-making, system monitoring, and user insights.

[0057] Example 2 This invention also provides a lightweight reporting device for massive amounts of data collected from cloud phones, such as... Figure 2 As shown, the device 10 includes: The data classification and preprocessing module 100 is used to classify the collected raw embedded data according to business value and timeliness requirements, determine the reporting strategy for different levels of data, and perform intelligent aggregation and compression processing on the classified embedded data according to the reporting strategy and the current network status to generate preprocessed data batches.

[0058] The binary encoding module 200 is used to perform binary encoding processing on the preprocessed data batch based on a predefined protocol. By digitally mapping field names, replacing enumeration values ​​with codes, and recording incremental differences in status data, the data batch is converted into a compact binary byte stream.

[0059] The redundancy elimination and encapsulation module 300 is used to perform field-level redundancy elimination processing on the compact binary byte stream. It generates load data by omitting default value fields, recording the difference between adjacent data, and extracting common prefix and suffix separation methods, and encapsulates it with a protocol header containing protocol metadata to form a self-interpreting data packet.

[0060] The adaptive reporting module 400 is used to adaptively schedule and report the self-interpreting data packets based on real-time network detection results and user activity status, dynamically adjust the transmission timing and retry strategy according to data level and network conditions, and clean, aggregate and hierarchically store the successfully reported data streams on the server side.

[0061] Example 3 To implement the methods of the above embodiments, the present invention also provides a computer device, which includes a memory and a processor; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, so as to implement the various steps of the methods described above.

[0062] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0064] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A lightweight reporting method for massive amounts of data collected by cloud phones, characterized in that, Includes the following steps: Based on business value and timeliness requirements, the collected raw data points are classified and processed, and reporting strategies for different levels of data are determined. The classified data points are then intelligently aggregated and compressed according to the reporting strategies and the current network status to generate pre-processed data batches. The preprocessed data batch is binary encoded based on a predefined protocol. By digitally mapping field names, replacing enumerated values ​​with codes, and recording incremental differences in status data, the data batch is converted into a compact binary byte stream. The compact binary byte stream is subjected to field-level redundancy elimination processing. The payload data is generated by omitting default value fields, recording the difference between adjacent data, and extracting common prefix and suffix separation methods. The payload data is then encapsulated with a protocol header containing protocol metadata to form a self-interpreting data packet. Based on real-time network detection results and user activity status, the self-interpreting data packets are adaptively scheduled and reported. The transmission timing and retry strategy are dynamically adjusted according to the data level and network conditions. The successfully reported data streams are cleaned, aggregated, and stored hierarchically on the server side.

2. The method according to claim 1, characterized in that, The step of intelligently aggregating and compressing the graded embedded data according to the reporting strategy and the current network status to generate preprocessed data batches includes: Before the data is stored in the cloud phone virtual environment, the hierarchical results and the current network conditions are integrated in real time in the terminal memory buffer to dynamically determine the packaging timing and batch composition strategy. The batch processing window size is determined based on data priority and network speed. The time window is then used to aggregate the graded data points according to the batch processing window size to generate a preliminary aggregated data batch. Semantic compression is performed on the aggregated data batches. Common fields are preprocessed using columnar storage reorganization and dictionary encoding. Specific fields are further compressed using a general compression algorithm to generate preprocessed data batches.

3. The method according to claim 1, characterized in that, The process of eliminating field-level redundancy in the compact binary byte stream, which generates payload data by omitting default value fields, recording differences between adjacent data, and extracting common prefixes and suffixes, includes: The field default value omission operation is performed. During the serialization process, for fields whose values ​​are the same as the default values ​​defined in the protocol, the encoding of the field is directly omitted in the payload so that the receiving end can automatically fill in the default value during parsing. Perform adjacent record difference recording operation. For numeric fields between consecutive records in the same batch, starting from the second record, store the difference between the current value of the field and the value of the previous record. Perform a common prefix extraction operation. For long string prefixes that are common to all records in the same batch, extract them in the payload header and store them only once. In each individual record, store only the difference suffix after removing the common prefix, and generate payload data with redundancy elimination completed.

4. The method according to claim 1, characterized in that, The process of combining a protocol header containing protocol metadata to form a self-interpreting data packet includes: For load data that has undergone field-level redundancy elimination processing, a lightweight fixed-length protocol header is appended before the load body to form a self-interpreting data packet; Write all the metadata necessary for the receiver to independently and correctly parse the payload into the protocol header, including the protocol version number, data compression method identifier, sampling and aggregation strategy description, and start and end timestamps of the data within the batch.

5. The method according to claim 1, characterized in that, The adaptive scheduling and reporting of the self-interpreting data packets based on real-time network detection results and user activity status includes: Before uploading, the current network status is detected in real time, and differentiated scheduling rules are executed based on the detection results. When the network status is good, non-urgent data is allowed to be uploaded in batches. When the network status is weak, the core protection and traffic control strategy is activated, allowing only high-priority data to be uploaded immediately, and the generated non-high-priority data is temporarily stored in the local cache queue to wait for the network to recover. By sensing user activity levels, and taking advantage of periods of inactivity when the device screen is off or the application is in the background, historical backlog data that does not have high real-time requirements can be uploaded in batches.

6. The method according to claim 5, characterized in that, The method of dynamically adjusting transmission timing and retry strategies based on data level and network conditions includes: Implement a progressive delay retry and degradation processing strategy associated with the data level for failed data packets; For failures in reporting high-priority data, a retry strategy with short intervals and high priority is adopted. For failures in reporting low-priority data, a backoff algorithm is used to gradually extend the retry interval. Set a total quota limit for data in the local cache. When the amount of cached data exceeds the quota limit, discard the oldest and lowest-level data in the cache according to the first-in-first-out (FIFO) strategy.

7. The method according to claim 1, characterized in that, The process of cleaning, aggregating, and storing the successfully reported data streams on the server side includes: In the real-time computing layer, the received raw event stream is rapidly aggregated within a preset window according to predefined aggregation logic, and the aggregated statistical indicators generated are written into the online analysis system in real time. Implement differentiated storage strategies for data with different values ​​and access patterns. Automatically route and store data in storage media with different performance and cost based on data popularity, granularity and purpose, and implement corresponding data lifecycle management strategies.

8. A lightweight reporting device for massive amounts of data collected from cloud mobile phones, characterized in that, include: The data classification and preprocessing module is used to classify the collected raw data points according to business value and timeliness requirements, determine the reporting strategy for different levels of data, and intelligently aggregate and compress the classified data points according to the reporting strategy and the current network status to generate preprocessed data batches. The binary encoding module is used to perform binary encoding processing on the preprocessed data batch based on a predefined protocol. It converts the data batch into a compact binary byte stream by digital mapping of field names, code replacement of enumeration values, and recording of incremental differences in status data. The redundancy elimination and encapsulation module is used to perform field-level redundancy elimination processing on the compact binary byte stream. It generates payload data by omitting default value fields, recording the difference between adjacent data, and extracting common prefix and suffix separation methods, and encapsulates it with a protocol header containing protocol metadata to form a self-interpreting data packet. The adaptive reporting module is used to adaptively schedule and report the self-interpreting data packets based on real-time network detection results and user activity status. It dynamically adjusts the transmission timing and retry strategy according to the data level and network conditions, and cleans, aggregates and stores the successfully reported data streams on the server.

9. A computer device, characterized in that, Including processor and memory; The processor reads the executable program code stored in the memory to run the program corresponding to the executable program code, so as to implement a lightweight reporting method for massive cloud mobile phone data collection points as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a lightweight reporting method for massive data collection points of cloud mobile phones as described in any one of claims 1-7.

Images