Integrated data reporting system and method based on Android platform

Abstract

The invention discloses an integrated data reporting system and method based on an Android platform, and relates to the technical field of data processing, and the method comprises the steps: calculating the activeness of users, dividing the users into different user levels, and designing a sampling rate for each user level; performing anomaly detection on the sampled data, calculating a quality score of the sampled data of each user, and filtering abnormal data; acquiring real-time network delay and network bandwidth, and designing a self-adaptive compression ratio control function to obtain compressed coded data; the emergency degree of the coded data is calculated, and when the coded data is larger than the emergency degree threshold value of the user level where the coded data is located, the coded data is reported immediately; when the coded data is smaller than or equal to the emergency degree threshold value of the user level where the coded data is located, the optimization objective function is defined as the minimum data reporting amount and the maximum data real-time performance, the optimal reporting strategy of the coded data is searched, and the method and the device do not occupy excessive client resources or network bandwidth while meeting the real-time performance requirement of the data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an integrated data reporting system and method based on an Android platform. Background Art

[0002] With the rapid development of the mobile game market, game data analysis plays an increasingly important role in game operations. By collecting and analyzing various types of data generated during the game operation, it can help game developers and operators to timely understand the health of the game, user behavior characteristics, etc., so as to optimize game design, improve operational efficiency and monetization capabilities.

[0003] The patent application with publication number CN119065942A discloses a reporting and checking method, system, device and medium for game advertising media packages. The reporting and checking method ensures the effective display of advertising content through the interaction between the game terminal, advertising platform and SDK. The method also involves the sending and responding of advertising requests, the inspection and analysis of log data, the prompting of abnormal content and the packaging and installation of advertising content. In the process of advertising content packaging and installation, a stress testing scheme is also introduced to simulate the actual environment to perform performance testing on the SDK and generate a performance report. This method not only ensures the accurate display of advertising content, but also improves the stability and reliability of the SDK through stress testing, providing strong support for the efficient dissemination of game advertising.

[0004] The integrated data reporting based on the Android platform has high requirements for the real-time performance of the data. The existing game integrated data methods may ignore the problem of occupying too many client resources or network bandwidth when reporting data because they have to meet the real-time requirements of the data, thus affecting the user's gaming experience. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose an integrated data reporting system and method based on the Android platform, which solves the impact on the user's gaming experience when reporting data.

[0006] One aspect of the present application provides an integrated data reporting method based on an Android platform, comprising:

[0007] Step S100: obtaining the original data and user activity index generated when the user plays the game, calculating the user activity, dividing the user into K different user levels according to the user activity, and designing a different sampling rate for each user level;

[0008] Step S200: sampling the original data of each user level to obtain the user's sampled data, performing anomaly detection on the sampled data, calculating the quality score of each user's sampled data, filtering the abnormal data, and obtaining filtered game integrated data;

[0009] Step S300: obtaining real-time network delay and network bandwidth, designing an adaptive compression rate control function, designing compression strategies according to different data types, and obtaining compressed encoded data;

[0010] Step S400: Calculate the urgency of the coded data, and when the coded data is greater than the urgency threshold of the user level to which it belongs, immediately report the coded data;

[0011] Step S500: when the coded data is less than or equal to the urgency threshold of the user level to which it belongs, the reporting strategy is modeled as an optimization objective function, the optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance, the decision variables and constraints of the optimization objective function are defined, and the optimal reporting strategy for the coded data is searched;

[0012] The specific method of obtaining the original data and user activity index generated when the user plays the game, calculating the user activity, dividing the user into K different user levels according to the user activity, and designing different sampling rates for each user level is as follows:

[0013] Step S110: collecting raw data generated by the user when playing games in real time in the client SDK, the raw data includes user behavior data, device information, and game events, wherein the user behavior data includes the number of monthly logins, monthly online time, monthly consumption amount, and the timestamp of each game record;

[0014] Step S120: Calculate the real-time index of the latest game record based on the difference between the timestamp of the latest game record and the current time , taking the monthly login times, monthly online time, monthly consumption amount and real-time index as user activity indicators, calculate user activity ;

[0015] Step S130: Divide the users into K user levels based on the clustering algorithm according to the user activity;

[0016] Step S140: For each user level , design the sampling rate according to the average user activity in this user level ;

[0017] The specific method of dividing users into K user levels based on the clustering algorithm according to user activity is:

[0018] Step S131: randomly selecting multiple users as initial cluster centers, each cluster center being composed of various user activity indicators of the user;

[0019] Step S132: For each user, calculate the Euclidean distance between the user and each cluster center, and assign the user to the cluster where the cluster center closest to the user is located;

[0020] Step S133: For each cluster, the mean user activity of all users within it is calculated as a new cluster center, and the Euclidean distance between each user and the new cluster center is recalculated to re-divide the clusters;

[0021] Step S134: repeating steps S132 to S133 until the cluster center of each cluster no longer changes, and finally obtaining K clusters, and sorting the cluster centers of the clusters from large to small to obtain K user levels;

[0022] The specific method of sampling the original data of each user level to obtain the sampled data of the user is:

[0023] Step S210: for each user level, sampling is performed on the user level according to a sampling rate, and the amount of sampled data of each user level is the product of the total amount of original data of the user level and the sampling rate;

[0024] Step S220: Preset sliding window length and sliding step length , use a sliding window to sample the original data in the time dimension, sort the original data of users in the kth user level according to timestamps, and obtain the data stream;

[0025] Step S230: Initialize the starting position of the sliding window , remove from the starting position arrive The original data of the position is used as the sampling data in the current sliding window;

[0026] Step S240: Add the sampled data to the sampled data set of the current user, and move the starting position backward by the sliding step length Length, loop sampling until the end of the data stream is reached;

[0027] The specific method of sampling the original data of each user level to obtain the user's sampled data, performing anomaly detection on the sampled data, calculating the quality score of each user's sampled data, filtering the abnormal data, and obtaining the filtered game integrated data is as follows:

[0028] Step S250: Select the LOF algorithm as the unsupervised anomaly detection algorithm to perform anomaly detection on the user sampled data, extract the user behavior features from the original data of each user, calculate the Euclidean distance between the user and the neighbor users based on the user behavior features, and select the ka neighbor users closest to each user as the neighborhood of the user;

[0029] Step S260: For each user p, calculate the reachable distance between the user and the neighboring user q, and for each user p, calculate its local reachable density ;

[0030] Step S270: Calculate the local outlier factor of each user based on the local reachability density ;

[0031] Step S280: Use the local outlier factor as the user's quality score and preset anomaly threshold , users whose quality scores exceed the abnormal threshold are marked as abnormal data;

[0032] Step S290: marking abnormal data in the sample data set to obtain filtered game integration data;

[0033] The specific method of obtaining the real-time network delay and network bandwidth, designing an adaptive compression rate control function, and designing a compression strategy according to different data types to obtain the compressed encoded data is as follows:

[0034] Step S310: Designing an adaptive compression rate control function , based on the current network delay and network bandwidth b, calculate the target compression rate r;

[0035] Preset the first preset threshold of network delay , the second preset threshold , the third preset threshold , divide the definition domain of network delay into four intervals, and preset the first preset threshold of network bandwidth , the second preset threshold , the third preset threshold , the definition domain of network bandwidth is divided into four intervals;

[0036] The adaptive compression rate control function is a piecewise function. When , the target compression ratio takes the minimum value;

[0037] when When , the target compression rate increases linearly with the network delay and decays logarithmically with the network bandwidth;

[0038] when When , the target compression rate increases sinusoidally with network delay and decays exponentially with network bandwidth;

[0039] when When , the target compression ratio takes the highest value;

[0040] The adaptive compression rate control function The function expression is: ,in, , , are a first preset threshold, a second preset threshold, and a third preset threshold of network delay, , , are the first preset threshold, the second preset threshold, and the third preset threshold of the network bandwidth, , are the lowest and highest compression rates respectively. , , , , , is the fitting parameter;

[0041] Step S320: Design compression strategies for different data types in the game integrated data, the data types include numerical data and string data. For numerical data, Delta encoding is used for compression to obtain the numerical data sequence in the game integrated data. , Delta-encoded numerical data sequence ,in, is the nth numeric data, is the nth numeric data after Delta encoding, where n is the number of data in the numeric data sequence. , ;

[0042] Step S330: for the string data, dictionary coding and Huffman coding are used, firstly a character frequency dictionary is constructed, and then Huffman coding is used to encode the string data to obtain compressed string data;

[0043] Step S340: according to the current target compression rate and compression strategy, compress the game integrated data of different data types in each user level according to different compression strategies to obtain encoded data, and integrate all the encoded data into an encoded data set;

[0044] The specific method of calculating the urgency of the coded data and immediately reporting the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs is:

[0045] Step S410: predefine a data type urgency mapping table and a business semantic urgency mapping table for coded data, obtain the data type urgency and business semantic urgency of coded data j from the data type urgency mapping table and the business semantic urgency mapping table, calculate the timeliness urgency of coded data j, and and business semantic urgency , Timeliness and Urgency The weighted summation is used to obtain the urgency of the coded data j ;

[0046] Step S420: Preset the initial value of the urgency threshold of each user level, calculate the urgency of the coded data j, and if the coded data j is greater than the urgency threshold of the user level, , then mark the coded data j as urgent data and report it immediately;

[0047] Step S430: Setting the upper limit of network delay When the average network delay of the k-th user level is greater than the upper limit of the network delay, the urgency threshold of the k-th level is increased. improve , when the average network delay of the kth user level is restored to be less than or equal to the upper limit of the network delay, the current urgency threshold is restored to the initial urgency threshold;

[0048] When the coded data is less than or equal to the urgency threshold of the user level to which it belongs, the reporting strategy is modeled as an optimization objective function, the optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance, the decision variables and constraints of the optimization objective function are defined, and the specific method for searching the optimal reporting strategy for the coded data is as follows:

[0049] Step S510: If the urgency of the coded data j is less than or equal to the urgency threshold of the user level to which it belongs, the optimization objective function is used to make a reporting decision on the coded data j, and whether the coded data j is reported within the time window z is determined. Defined as a decision variable, the value of whether the coded data j is reported within the time window z is 0 or 1, 0 means no reporting, 1 means reporting, and the objective function is defined as the data reporting amount and data real-time , the optimization goal is to minimize the amount of data reported and maximize data real-time performance ;

[0050] The optimization objective function expression for minimizing the amount of data reported is: ,in, is the size of the encoded data j, is the number of data blocks of the encoded data, is the number of time windows, represents the network transmission cost of the encoded data j in time window z, is the network transmission cost coefficient, is the priority of the coded data j, Indicates whether the coded data j has not been reported in the current time window z. Indicates the cumulative number of unreported times of coded data j from the first time window to the current time window z, Indicates whether the coded data j is reported within the time window nz. Reporting delay penalty coefficient for data blocks, is the penalty intensity coefficient;

[0051] The optimization objective function expression for maximizing data real-time performance is: ,in, represents the urgency of the coded data j, represents the time attenuation coefficient, Indicates the length of time that the encoded data j waits in the time window z;

[0052] Step S520: defining constraints for optimizing the objective function, wherein the constraints include data quality constraints and abnormal data filtering constraints;

[0053] Step S530: Use a genetic algorithm to search for the optimization objective function, obtain the reporting decision of the current coded data j, and execute the reporting decision of whether to report the coded data j within the time window z.

[0054] One aspect of the present application provides an integrated data reporting system based on the Android platform, including:

[0055] The hierarchical sampling module is used to obtain the original data and user activity indicators generated when users play games, calculate user activity, divide users into K different user hierarchies according to user activity, and design different sampling rates for each user hierarchy;

[0056] An integrated data filtering module is used to sample the original data of each user level to obtain the user's sampled data, perform anomaly detection on the sampled data, calculate the quality score of each user's sampled data, filter the abnormal data, and obtain the filtered game integrated data;

[0057] The compression strategy execution module is used to obtain real-time network delay and network bandwidth, design an adaptive compression rate control function, design compression strategies according to different data types, and obtain compressed encoded data;

[0058] An emergency data reporting module is used to calculate the urgency of the coded data and immediately report the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs;

[0059] The non-emergency data reporting module is used to model the reporting strategy as an optimization objective function when the encoded data is less than or equal to the urgency threshold of the user level to which it belongs. The optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance. The decision variables and constraints of the optimization objective function are defined to search for the optimal reporting strategy for the encoded data.

[0060] One aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the integrated data reporting method based on the Android platform are implemented.

[0061] One aspect of the present application provides a readable storage medium, which stores a computer program, and the computer program is suitable for loading by a processor to execute the steps in an integrated data reporting method based on an Android platform.

[0062] Compared with the prior art, the integrated data reporting system and method based on the Android platform proposed in this application has the following advantages:

[0063] This application clusters and stratifies users according to their activity levels, and adopts different sampling rates for users at different levels, rather than simply reporting the entire data or using fixed sampling. This ensures the real-time nature of key data, while reducing the reporting of redundant data and reducing the load on the client and network.

[0064] This application introduces an unsupervised anomaly detection method based on the LOF algorithm, which identifies and filters abnormal data through behavioral feature analysis and quality scoring, reduces the reporting of invalid and abnormal data, further reduces the amount of data reporting, and improves data quality and analysis accuracy.

[0065] This application dynamically adjusts the target compression rate according to the real-time network status, and adopts differentiated compression encoding strategies for different data types. When the network condition is poor, the compression rate is increased, the packet size is reduced, and the real-time transmission of data is guaranteed; when the network condition is good, the compression rate is appropriately reduced, the compression overhead is reduced, and the client performance is improved.

[0066] This application introduces an urgency assessment mechanism based on data type, business semantics, and timeliness. It immediately reports data that exceeds the urgency threshold and dynamically adjusts the urgency threshold based on network latency, ensuring real-time reporting of critical data and improving the timeliness of data analysis and business decisions. At the same time, dynamic threshold control is used to avoid excessive resource usage caused by real-time reporting.

[0067] This application balances the costs and benefits of data reporting on a global scale, maximizes the data reporting performance under limited resource constraints, reduces the occupancy of client and network resources by data reporting through intelligent optimization, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A method flow chart of the integrated data reporting method based on the Android platform provided in this application;

[0069] Figure 2 A flow chart of a method for filtering game integrated data provided in this application;

[0070] Figure 3 A flow chart of the method for reporting emergency data provided for this application;

[0071] Figure 4 This is a functional module diagram of the integrated data reporting system based on the Android platform provided in this application. DETAILED DESCRIPTION

[0072] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0073] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0074] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0075] Unless otherwise defined, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that unless there is a clear explanation in this application, words defined in commonly used dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0076] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0077] Example 1

[0078] like Figure 1 As shown, the integrated data reporting method based on the Android platform provided by this application includes:

[0079] Step S100: obtaining the original data and user activity index generated when the user plays the game, calculating the user activity, dividing the user into K different user levels according to the user activity, and designing a different sampling rate for each user level;

[0080] The specific method of obtaining the original data and user activity index generated when the user plays the game, calculating the user activity, dividing the user into K different user levels according to the user activity, and designing different sampling rates for each user level is as follows:

[0081] Step S110: collecting raw data generated by the user when playing games in real time in the client SDK, the raw data includes user behavior data, device information, and game events, wherein the user behavior data includes the number of monthly logins, monthly online time, monthly consumption amount, and the timestamp of each game record;

[0082] Step S120: Calculate the real-time index of the latest game record according to the difference between the timestamp of the latest game record and the current time, and use the monthly login times, monthly online time, monthly consumption amount and real-time index as user activity indicators to calculate user activity;

[0083] The calculation formula of the real-time index of the latest game record is: ,in, Real-time indicators recorded for the latest games, Timestamp for the latest game record, is the current time;

[0084] The user activity The calculation formula is: ,in, is the nth user activity indicator, is the weight coefficient of each user activity indicator; NA is the total number of user activity indicators.

[0085] The sum of the weight coefficients of the user activity index is 1, and the specific value of the weight coefficient of each user activity index is set by those skilled in the art based on experience.

[0086] Step S130: Divide the users into K user levels based on the clustering algorithm according to the user activity;

[0087] The specific method of dividing users into K user levels based on the clustering algorithm according to user activity is:

[0088] Step S131: randomly selecting multiple users as initial cluster centers, each cluster center being composed of various user activity indicators of the user;

[0089] Step S132: For each user, calculate the Euclidean distance between the user and each cluster center, and assign the user to the cluster where the cluster center closest to the user is located;

[0090] Step S133: For each cluster, the mean user activity of all users within it is calculated as a new cluster center, and the Euclidean distance between each user and the new cluster center is recalculated to re-divide the clusters;

[0091] Step S134: repeating steps S132 to S133 until the cluster center of each cluster no longer changes, and finally obtaining K clusters, and sorting the cluster centers of the clusters from large to small to obtain K user levels;

[0092] Step S140: For each user level , design the sampling rate according to the average user activity in this user level ;

[0093] The sampling rate The calculation formula is: ,in, is the total sampling rate, is the sampling rate adjustment factor, is the mean user activity in the kth user level, log(·) is the logarithmic function, K is the number of user levels, is the influencing factor of user activity;

[0094] The total sampling rate represents the ratio of the final sampled data set to the original data set;

[0095] The adjustment factor of the sampling rate and the influencing factor of the user activity are set by those skilled in the art based on experience.

[0096] The calculation formula of the total sampling rate is: ,in, is the adjustment coefficient, is the baseline data volume, is the current amount of raw data, is the current available resource capacity, is the baseline system resource capacity, For data processing delay requirements, They are the historical minimum and maximum values ​​of the sampling rate respectively;

[0097] The adjustment coefficient, benchmark data volume, benchmark system resource capacity, and data processing delay requirement are set by those skilled in the art based on experience and the data processing capability of the system.

[0098] The above steps divide users into several different user levels according to their activity levels, and assign different sampling rates to each user level, thereby avoiding reporting of redundant data and wasting of network resources.

[0099] Step S200: sampling the original data of each user level to obtain the user's sampled data, performing anomaly detection on the sampled data, calculating the quality score of each user's sampled data, filtering the abnormal data, and obtaining filtered game integrated data;

[0100] The specific method of sampling the original data of each user level to obtain the sampled data of the user is:

[0101] Step S210: for each user level, sampling is performed on the user level according to a sampling rate, and the amount of sampled data of each user level is the product of the total amount of original data of the user level and the sampling rate;

[0102] Step S220: Preset sliding window length and sliding step length , use a sliding window to sample the original data in the time dimension, sort the original data of users in the kth user level according to timestamps, and obtain the data stream;

[0103] The calculation formula of the sliding window length is: ;

[0104] The calculation formula of the sliding step length is: ;

[0105] Step S230: Initialize the starting position of the sliding window , remove from the starting position arrive The original data of the position is used as the sampling data in the current sliding window;

[0106] Step S240: Add the sampled data to the sampled data set of the current user, and move the starting position backward by the sliding step length Length, loop sampling until the end of the data stream is reached;

[0107] See also Figure 2 The specific method of sampling the original data of each user level to obtain the user's sampled data, performing anomaly detection on the sampled data, calculating the quality score of each user's sampled data, filtering the abnormal data, and obtaining the filtered game integrated data is:

[0108] Step S250: Select the LOF algorithm as the unsupervised anomaly detection algorithm to perform anomaly detection on the user sampled data, extract the user behavior features from the original data of each user, calculate the Euclidean distance between the user and the neighbor users based on the user behavior features, and select the ka neighbor users closest to each user as the neighborhood of the user;

[0109] Step S260: For each user p, calculate the reachable distance between the user and the neighboring user q, and for each user p, calculate its local reachable density ;

[0110] The calculation formula of the reachable distance is: ,in, q is the Euclidean distance between neighbor user q and its k-th nearest neighbor user, is the actual Euclidean distance between user p and neighbor user q; is the reachable distance between user p and neighbor user q;

[0111] The calculation formula of the local reachable density is: ,in, is the neighborhood of user p;

[0112] Step S270: Calculate the local outlier factor of each user based on the local reachability density;

[0113] The calculation formula of the local outlier factor is: ,in, is the local reachability density of neighbor user q;

[0114] The above steps introduce an anomaly detection and data quality assessment mechanism based on the LOF algorithm to identify and filter out abnormal data, effectively improving the quality and reliability of reported data.

[0115] Step S280: Use the local outlier factor as the user's quality score and preset anomaly threshold , users whose quality scores exceed the abnormal threshold are marked as abnormal data;

[0116] The calculation formula of the abnormal threshold is: ,in, is the mean of historical user quality ratings, The standard deviation of historical user quality scores;

[0117] Step S290: marking abnormal data in the sample data set to obtain filtered game integration data;

[0118] Step S300: obtaining real-time network delay and network bandwidth, designing an adaptive compression rate control function, designing compression strategies according to different data types, and obtaining compressed encoded data;

[0119] The specific method of obtaining the real-time network delay and network bandwidth, designing an adaptive compression rate control function, and designing a compression strategy according to different data types to obtain the compressed encoded data is as follows:

[0120] Step S310: Designing an adaptive compression rate control function , based on the current network delay and network bandwidth b, calculate the target compression rate r;

[0121] Preset the first preset threshold of network delay , the second preset threshold , the third preset threshold , divide the definition domain of network delay into four intervals, and preset the first preset threshold of network bandwidth , the second preset threshold , the third preset threshold , the definition domain of network bandwidth is divided into four intervals;

[0122] The adaptive compression rate control function is a piecewise function. When , the target compression ratio takes the minimum value;

[0123] when When , the target compression rate increases linearly with the network delay and decays logarithmically with the network bandwidth;

[0124] when When , the target compression rate increases sinusoidally with network delay and decays exponentially with network bandwidth;

[0125] when When , the target compression ratio takes the highest value;

[0126] The adaptive compression rate control function The function expression is: ,in, , , are a first preset threshold, a second preset threshold, and a third preset threshold of network delay, , , are the first preset threshold, the second preset threshold, and the third preset threshold of the network bandwidth, , are the lowest and highest compression rates respectively. , , , , , is the fitting parameter;

[0127] The current network delay and network bandwidth b are monitored in real time by the client SDK.

[0128] The fitting parameters are obtained by those skilled in the art through experimental fitting based on historical data;

[0129] The first preset threshold, the second preset threshold, the third preset threshold of the network delay and the first preset threshold, the second preset threshold, the third preset threshold of the network bandwidth are set by those skilled in the art according to different user levels and experience, wherein: , ;

[0130] The user levels range from 1 to K, and the sizes of their cluster centers are sorted from large to small, that is, the higher the user level, the lower the user activity. Different thresholds are assigned to the game integration data of each user level. The higher the user level, the larger the first preset threshold, the second preset threshold, and the third preset threshold of the network delay and the smaller the first preset threshold, the second preset threshold, and the third preset threshold of the network bandwidth.

[0131] The minimum compression rate and the maximum compression rate are the minimum and maximum values ​​of all compression rates in the history of each user level;

[0132] Step S320: Design compression strategies for different data types in the game integrated data, the data types include numerical data and string data. For numerical data, Delta encoding is used for compression to obtain the numerical data sequence in the game integrated data. , Delta-encoded numerical data sequence ,in, is the nth numeric data, is the nth numeric data after Delta encoding, where n is the number of data in the numeric data sequence. , ;

[0133] Step S330: for the string data, dictionary coding and Huffman coding are used, firstly a character frequency dictionary is constructed, and then Huffman coding is used to encode the string data to obtain compressed string data;

[0134] The dictionary coding and Huffman coding are commonly used technical means in this field and will not be described in detail here.

[0135] Step S340: according to the current target compression rate and compression strategy, compress the game integrated data of different data types in each user level according to different compression strategies to obtain encoded data, and integrate all the encoded data into an encoded data set;

[0136] The above adaptive compression rate control function combines multiple nonlinear functions to better fit the complex relationship between network conditions and compression rate. By setting reasonable fitting parameters, a lower compression rate can be used in low latency and high bandwidth to ensure data quality; a higher compression rate can be used in high latency and low bandwidth to reduce data transmission volume. At the same time, a smooth transition can be achieved under medium network conditions to avoid drastic fluctuations in compression rate.

[0137] Step S400: Calculate the urgency of the coded data, and when the coded data is greater than the urgency threshold of the user level to which it belongs, immediately report the coded data;

[0138] See also Figure 3 The specific method of calculating the urgency of the coded data and immediately reporting the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs is:

[0139] Step S410: predefine a data type urgency mapping table and a business semantic urgency mapping table for coded data, obtain the data type urgency and business semantic urgency of coded data j from the data type urgency mapping table and the business semantic urgency mapping table, calculate the timeliness urgency of coded data j, and and business semantic urgency , Timeliness and Urgency The weighted sum is used to obtain the urgency of the coded data j;

[0140] The calculation formula of the urgency is: ,in, , , They are the weight coefficients of data type urgency, business semantic urgency, and timeliness urgency respectively;

[0141] The calculation formula of the timeliness urgency is: ,in, is the time when the encoded data j is generated, is the current time, is the time-dependent attenuation coefficient;

[0142] Step S420: Preset the initial value of the urgency threshold of each user level, calculate the urgency of the coded data j, and if the coded data j is greater than the urgency threshold of the user level, , then mark the coded data j as urgent data and report it immediately;

[0143] The initial value of the urgency threshold of each user level is set by those skilled in the art based on experience.

[0144] Step S430: Setting the upper limit of network delay When the average network delay of the k-th user level is greater than the upper limit of the network delay, the urgency threshold of the k-th level is increased. Increase Δu. When the average network delay of the k-th user level is restored to be less than or equal to the upper limit of the network delay, the current urgency threshold is restored to the initial urgency threshold.

[0145] The Δu is the adjustment degree of the emergency threshold, which is set by those skilled in the art based on experience.

[0146] The dynamic adjustment of the above urgency threshold enables the intelligent reporting strategy to adaptively optimize the timeliness and quality of data reporting according to the real-time status and data characteristics of the system. This is of great significance for ensuring the real-time and reliability of key data and improving business decision-making and user experience.

[0147] Step S500: when the coded data is less than or equal to the urgency threshold of the user level to which it belongs, the reporting strategy is modeled as an optimization objective function, the optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance, the decision variables and constraints of the optimization objective function are defined, and the optimal reporting strategy for the coded data is searched;

[0148] When the coded data is less than or equal to the urgency threshold of the user level to which it belongs, the reporting strategy is modeled as an optimization objective function, the optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance, the decision variables and constraints of the optimization objective function are defined, and the specific method for searching the optimal reporting strategy for the coded data is as follows:

[0149] Step S510: If the urgency of the coded data j is less than or equal to the urgency threshold of the user level to which it belongs, the optimization objective function is used to make a reporting decision on the coded data j, and whether the coded data j is reported within the time window z is determined. Defined as a decision variable, the value of whether the coded data j is reported within the time window z is 0 or 1, 0 means no reporting, 1 means reporting, and the objective function is defined as the data reporting amount and data real-time ,The optimization goal is to minimize the amount of data reported and maximize the real-time performance of data;

[0150] The optimization objective function expression for minimizing the amount of data reported is: ,in, is the size of the encoded data j, is the number of data blocks of the encoded data, is the number of time windows, represents the network transmission cost of the encoded data j in time window z, is the network transmission cost coefficient, is the priority of the coded data j, Indicates whether the coded data j has not been reported in the current time window z. Indicates the cumulative number of unreported times of coded data j from the first time window to the current time window z, Indicates whether the coded data j is reported within the time window nz. Reporting delay penalty coefficient for data blocks, is the penalty intensity coefficient;

[0151] The values ​​of the network transmission cost coefficient, the data block reporting delay penalty coefficient, and the penalty intensity coefficient are set by those skilled in the art based on experience.

[0152] The optimization objective function expression for maximizing data real-time performance is: ,in, represents the urgency of the coded data j, represents the time attenuation coefficient, Indicates the length of time that the encoded data j waits in the time window z;

[0153] The weight coefficients of the data type urgency, business semantic urgency, timeliness urgency, timeliness decay coefficient, and time decay coefficient are set by those skilled in the art based on experience.

[0154] Step S520: defining constraints for optimizing the objective function, wherein the constraints include data quality constraints and abnormal data filtering constraints;

[0155] The constraint expression of the data quality constraint is: ,in, is the quality score of the encoded data j, represents the minimum quality requirement of the encoded data in the kth user level, represents the encoded data set in the kth user level, M is a constant term;

[0156] The quality score of the coded data j refers to the quality score of the sampled data corresponding to the coded data;

[0157] The constant term M is a sufficiently large positive number, and its function is to relax the data quality constraint condition when no coded data is reported, so that the data quality constraint condition always holds.

[0158] The minimum quality requirement of the encoded data is set according to business needs and historical data quality distribution.

[0159] The constraint expression of the abnormal data filtering constraint is: , where AJ is the set of abnormal data;

[0160] Step S530: Use a genetic algorithm to search for the optimization objective function, obtain the reporting decision of the current coded data j, and execute the reporting decision of whether to report the coded data j within the time window z.

[0161] The above steps form a complete intelligent reporting decision framework. The urgency judgment plays a role in preprocessing and rapid response before the data enters the optimization model, and the optimization objective function systematically optimizes the reporting strategy at the macro level. Through dynamic adjustment, the urgency threshold, the intelligent reporting strategy can adaptively respond to changes in data characteristics and business needs, and continuously optimize the timeliness, quality and cost of data reporting.

[0162] In general, this application significantly reduces the resource consumption of data reporting and improves data quality and user experience while meeting the real-time requirements of game data through a series of innovative mechanisms such as user stratified sampling, data anomaly filtering, adaptive compression, urgency assessment, and multi-objective optimization. These innovations not only solve the contradiction between real-time and resource usage in existing game data reporting, but also provide useful optimization ideas and solutions for other similar data reporting scenarios.

[0163] Example 2

[0164] like Figure 4 As shown, the integrated data reporting system based on the Android platform provided by this application includes:

[0165] The hierarchical sampling module is used to obtain the original data and user activity indicators generated when users play games, calculate user activity, divide users into K different user hierarchies according to user activity, and design different sampling rates for each user hierarchy;

[0166] An integrated data filtering module is used to sample the original data of each user level to obtain the user's sampled data, perform anomaly detection on the sampled data, calculate the quality score of each user's sampled data, filter the abnormal data, and obtain the filtered game integrated data;

[0167] The compression strategy execution module is used to obtain real-time network delay and network bandwidth, design an adaptive compression rate control function, design compression strategies according to different data types, and obtain compressed encoded data;

[0168] An emergency data reporting module is used to calculate the urgency of the coded data and immediately report the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs;

[0169] The non-emergency data reporting module is used to model the reporting strategy as an optimization objective function when the encoded data is less than or equal to the urgency threshold of the user level to which it belongs. The optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance. The decision variables and constraints of the optimization objective function are defined to search for the optimal reporting strategy for the encoded data.

[0170] Furthermore, the integrated data reporting system based on the Android platform provided by this application reports the game data of a multiplayer online game in real time, and the game client SDK collects the original game data of the players in real time, including the number of monthly logins, monthly online time, monthly consumption amount, and the timestamp of the game records, etc. The hierarchical sampling module calculates the activity of each player based on the real-time and other indicators of the latest game records, and divides the players into three activity levels through a clustering algorithm: high-activity players, medium-activity players, and low-activity players, and designs different sampling rates according to the average activity of each level.

[0171] Furthermore, the integrated data filtering module samples the original game data according to the sampling rates at different levels. For example, for a moderately active player, a sliding window is used to sample every 10 minutes, and 50% of the data is randomly selected each time. Next, the LOF algorithm is used to detect anomalies in the sampled data, extract the behavioral characteristics of the players, calculate the distance between the players, find the abnormal players whose local outliers are higher than the threshold, and mark their data as abnormal.

[0172] Furthermore, the compression strategy execution module designs an adaptive compression rate control function according to the real-time network status. When the network delay is high or the bandwidth is low, the compression rate is appropriately increased. Player data at different activity levels use different delay and bandwidth thresholds. Different compression strategies are applied to different types of game data. For numerical data such as levels and coins, Delta encoding is used; for string data such as player names and task names, dictionary encoding and Huffman encoding are used. After compression, an encoded data set is obtained, which includes multiple encoded data blocks.

[0173] Furthermore, the emergency data reporting module calculates the urgency of each coded data block, taking into account factors such as its data type, business semantics, and timeliness. If the urgency of a data block exceeds the urgency threshold of its player level, it is marked as emergency data and reported immediately without waiting for an optimization decision.

[0174] Furthermore, for non-urgent data blocks whose urgency is not high enough, the non-urgent data reporting module models the reporting decision problem as a multi-objective optimization problem, uses a genetic algorithm to search for the optimal solution of this optimization problem, and obtains the optimal decision on whether to report each data block within different time windows.

[0175] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0176] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0177] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An integrated data reporting method based on the Android platform, characterized in that: include: Obtain the original data and user activity indicators generated by users when playing games, calculate user activity, divide users into K different user levels according to user activity, and design different sampling rates for each user level; The original data of each user level is sampled to obtain the user's sampled data, anomaly detection is performed on the sampled data, the quality score of each user's sampled data is calculated, the abnormal data is filtered, and the filtered game integrated data is obtained; Obtain real-time network delay and network bandwidth, design an adaptive compression rate control function, design compression strategies according to different data types, and obtain compressed encoded data; Calculate the urgency of the coded data, and when the coded data is greater than the urgency threshold of the user level to which it belongs, immediately report the coded data; When the coded data is less than or equal to the urgency threshold of its user level, the reporting strategy is modeled as an optimization objective function, which is defined as minimizing the amount of data reported and maximizing data real-time performance. The decision variables and constraints of the optimization objective function are defined to search for the optimal reporting strategy for the coded data.

2. The integrated data reporting method based on the Android platform as claimed in claim 1, characterized in that: The specific method of obtaining the original data and user activity index generated when the user plays the game, calculating the user activity, dividing the user into K different user levels according to the user activity, and designing different sampling rates for each user level is as follows: Collect the original data generated by users when playing games in real time in the client SDK. The original data includes user behavior data, device information, and game events. The user behavior data includes the number of monthly logins, monthly online time, monthly consumption amount, and the timestamp of each game record; Calculate the real-time index of the latest game record based on the difference between the timestamp of the latest game record and the current time , taking the number of monthly logins, monthly online time, monthly consumption amount and real-time indicators as user activity indicators, calculate user activity ; According to user activity, users are divided into K user levels based on clustering algorithm; For each user level , design the sampling rate according to the average user activity in this user level .

3. The integrated data reporting method based on the Android platform as claimed in claim 2, characterized in that: The specific method of sampling the original data of each user level to obtain the sampled data of the user is: For each user level, the user level is sampled according to the sampling rate, and the amount of sampled data for each user level is the product of the total amount of original data of the user level and the sampling rate; Preset sliding window length and sliding step length , use a sliding window to sample the original data in the time dimension, sort the original data of users in the kth user level according to timestamps, and obtain the data stream; Initialize the starting position of the sliding window , remove from the starting position arrive The original data of the position is used as the sampling data in the current sliding window; Add the sampled data to the current user's sampled data set and move the starting position backward by the sliding step size. The length of the data stream is looped until the end of the data stream is reached.

4. The integrated data reporting method based on the Android platform as claimed in claim 3, characterized in that: The specific method of sampling the original data of each user level to obtain the user's sampled data, performing anomaly detection on the sampled data, calculating the quality score of each user's sampled data, filtering the abnormal data, and obtaining the filtered game integrated data is as follows: Select the LOF algorithm as the unsupervised anomaly detection algorithm to perform anomaly detection on user sampling data, extract user behavior features from each user's original data, calculate the Euclidean distance between the user and the neighboring users based on the user behavior features, and select the ka nearest neighboring users to each user as the neighborhood of the user; For each user p, calculate the reachable distance between the user and the neighbor user q, and for each user p, calculate its local reachable density ; Calculate the local outlier factor of each user based on the local reachability density ; Use the local outlier factor as the user's quality score and preset the abnormal threshold , users whose quality scores exceed the abnormal threshold are marked as abnormal data; The abnormal data in the sampled data set is marked to obtain the filtered game integration data.

5. The integrated data reporting method based on the Android platform as claimed in claim 4, characterized in that: The specific method of obtaining the real-time network delay and network bandwidth, designing an adaptive compression rate control function, and designing a compression strategy according to different data types to obtain the compressed encoded data is as follows: Designing an adaptive compression rate control function , based on the current network delay and network bandwidth b, calculate the target compression rate r; Preset the first preset threshold of network delay , the second preset threshold , the third preset threshold , divide the definition domain of network delay into four intervals, and preset the first preset threshold of network bandwidth , the second preset threshold , the third preset threshold The domain of network bandwidth is divided into four intervals; the adaptive compression rate control function is a piecewise function. When , the target compression ratio takes the minimum value; when When , the target compression rate increases linearly with the network delay and decays logarithmically with the network bandwidth; when When , the target compression rate increases sinusoidally with network delay and decays exponentially with network bandwidth; when When , the target compression ratio takes the highest value; Design compression strategies for different data types in game integrated data, including numeric data and string data. For numeric data, use Delta encoding for compression to obtain the numeric data sequence in game integrated data. , Delta-encoded numerical data sequence ,in, is the nth numeric data, is the nth numeric data after Delta encoding, where n is the number of data in the numeric data sequence. , ; For string data, dictionary coding and Huffman coding are used. First, a character frequency dictionary is constructed, and then Huffman coding is used to encode the string data to obtain compressed string data. According to the current target compression rate and compression strategy, the game integrated data of different data types in each user level are compressed according to different compression strategies to obtain encoded data, and all the encoded data are integrated into an encoded data set.

6. The integrated data reporting method based on the Android platform as claimed in claim 5, characterized in that: The specific method of calculating the urgency of the coded data and immediately reporting the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs is: Predefine the data type urgency mapping table and business semantic urgency mapping table of the coded data, obtain the data type urgency and business semantic urgency of the coded data j from the data type urgency mapping table and the business semantic urgency mapping table, calculate the timeliness urgency of the coded data j, and and business semantic urgency , Timeliness and Urgency The weighted summation is used to obtain the urgency of the coded data j ; Preset the initial value of the urgency threshold of each user level, calculate the urgency of the coded data j, and if the coded data j is greater than the urgency threshold of the user level , then mark the coded data j as urgent data and report it immediately; Set a limit on network latency When the average network delay of the k-th user level is greater than the upper limit of the network delay, the urgency threshold of the k-th level is increased. improve , when the average network delay of the kth user level is restored to be less than or equal to the upper limit of the network delay, the current urgency threshold is restored to the initial urgency threshold.

7. The integrated data reporting method based on the Android platform as claimed in claim 6, characterized in that: When the coded data is less than or equal to the urgency threshold of the user level to which it belongs, the reporting strategy is modeled as an optimization objective function, the optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance, the decision variables and constraints of the optimization objective function are defined, and the specific method for searching the optimal reporting strategy for the coded data is as follows: If the urgency of the coded data j is less than or equal to the urgency threshold of the user level, the optimization objective function is used to make a reporting decision, and whether the coded data j is reported within the time window z is determined. Defined as a decision variable, the value of whether the coded data j is reported within the time window z is 0 or 1, 0 means no reporting, 1 means reporting, and the objective function is defined as the data reporting amount and data real-time , the optimization goal is to minimize the amount of data reported and maximize data real-time performance ; Defining constraints for optimizing the objective function, wherein the constraints include data quality constraints and abnormal data filtering constraints; A genetic algorithm is used to search for the optimization objective function, obtain the reporting decision of the current coded data j, and execute the reporting decision of whether the coded data j is reported within the time window z.

8. An integrated data reporting system based on an Android platform, which is used to implement the integrated data reporting method based on an Android platform as described in any one of claims 1 to 7, characterized in that: include: The hierarchical sampling module is used to obtain the original data and user activity indicators generated when users play games, calculate user activity, divide users into K different user hierarchies according to user activity, and design different sampling rates for each user hierarchy; An integrated data filtering module is used to sample the original data of each user level to obtain the user's sampled data, perform anomaly detection on the sampled data, calculate the quality score of each user's sampled data, filter the abnormal data, and obtain the filtered game integrated data; The compression strategy execution module is used to obtain real-time network delay and network bandwidth, design an adaptive compression rate control function, design compression strategies according to different data types, and obtain compressed encoded data; An emergency data reporting module is used to calculate the urgency of the coded data and immediately report the coded data when the coded data is greater than the urgency threshold of the user level to which it belongs; The non-emergency data reporting module is used to model the reporting strategy as an optimization objective function when the encoded data is less than or equal to the urgency threshold of the user level to which it belongs. The optimization objective function is defined as minimizing the data reporting amount and maximizing the data real-time performance. The decision variables and constraints of the optimization objective function are defined to search for the optimal reporting strategy for the encoded data.

9. An electronic device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the integrated data reporting method based on the Android platform as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the steps in the integrated data reporting method based on the Android platform as described in any one of claims 1-7.

Images