A code mapping-based client log batch uploading method and system

Abstract

The application discloses a kind of client log batch upload method and system based on Code mapping, comprising: defining the corresponding relation of each kind of log and Code;Continuously obtain the log to be uploaded, and the log to be uploaded is divided into mappable and unmappable log;Mappable log is mapped into Code one by one, and unmappable log carries Code, obtains the Code log to be uploaded, simultaneously continuously Code is added to Code list, obtains new Code list, and is saved to flash space;Code log to be uploaded is uploaded to server according to normal log, for new Code list, it is judged whether to meet trigger condition, if meet, it is uploaded to server in batch;After receiving is completed in server, it is directly stored in the form of Code.The application achieves the purpose that the carrying capacity of server can be significantly improved without expanding log system.

Description

Technical Field

[0001] This invention relates to the field of batch log uploading technology, specifically to a client-side batch log uploading method and system based on code mapping. Background Technology

[0002] Currently, many terminal devices, including mobile phones, set-top boxes, and televisions, require log uploads from their system software. Logs uploaded to a log server can be cleaned and used for big data analysis (user profiling, habits, preferences, etc.), message chain tracing, and locating software problems through logs from set-top box anomalies or crashes. Existing technologies generally achieve these functions through server-side logging tools. Taking ELK as an example: "ELK" is an acronym for three open-source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server-side data processing pipeline that can simultaneously collect and transform data from multiple sources and then send it to "repositories" such as Elasticsearch. Kibana allows users to visualize data using graphs and charts within Elasticsearch. The display, querying, statistics, and big data analysis of server-side logs can all be implemented using ELK. Message chain tracing can be achieved through SkyWalking.

[0003] For a single terminal, the amount of logs that need to be uploaded is not large, so the server-side logging system does not experience much pressure. However, if the number of terminals reaches millions, tens of millions, or even more, the pressure on the server in terms of bandwidth, concurrent processing, and storage becomes enormous.

[0004] The current approach is to increase the server's capacity by expanding the log system's servers, but this method has the problem of high usage and maintenance costs. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides a method and system for batch uploading client logs based on code mapping. This method addresses the technical problem of high usage and maintenance costs in improving the server's carrying capacity, thereby achieving the goal of significantly improving the server's carrying capacity without expanding the log system.

[0006] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0007] A method for batch uploading client logs based on code mapping includes the following steps:

[0008] Define the correspondence between each type of log and the code;

[0009] Continuously acquire logs to be uploaded and divide the logs to be uploaded into mappable logs and unmappable logs;

[0010] According to the correspondence, the mappable logs are mapped one by one to Code, and the unmappable logs are made to carry Code, so as to obtain the Code logs to be uploaded. At the same time, the Codes are continuously added to the Code list to obtain a new Code list.

[0011] On the client, a flash space is allocated for storing logs and code, and the code log to be uploaded and the new code list are saved to the flash space;

[0012] The code logs to be uploaded are uploaded to the server as normal logs. For the new code list, it is determined whether the triggering conditions are met. If they are met, the codes in the new code list are uploaded to the server in batches.

[0013] After the server completes the reception, it obtains the uploaded code log and the uploaded code list, and stores the codes in the uploaded code list directly in code form.

[0014] In a preferred embodiment of the present invention, the definition of the correspondence between each type of log and code includes:

[0015] If the log type is without parameters, the mapping will be performed directly according to the preset correspondence.

[0016] If the log type has parameters, then further determine the type of the parameters.

[0017] If the parameter type is a non-floating-point number, the correspondence between the log and the Code is: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number...;

[0018] If the parameter type is a floating-point number, determine whether the floating-point number is necessary. If yes, convert the floating-point number to two integers; otherwise, convert the floating-point number to integers.

[0019] In a preferred embodiment of the present invention, when further determining the type of the parameters, the method further includes:

[0020] If the parameter type is an exhaustive list of characters, then the character list is converted into an enumeration. The correspondence between the log and the Code is: Code (2 bytes) + quantity (1 byte) + number (4 bytes) + number...;

[0021] If the parameter type is not a fixed character parameter, the processing flow is as follows:

[0022] If the log format corresponding to Code 12102 is: mps recv unknown skip url.[%s];

[0023] A log file was retrieved: mps recv unknown skip url.[https: / / www.google.com];

[0024] Where https: / / www.google.com is the non-fixed character parameter, i.e. the unmappable log, then the correspondence between the log and the code is: 12102.[https: / / www.google.com].

[0025] In a preferred embodiment of the present invention, when allocating flash space on the client for storing logs and code, the following steps are included:

[0026] A flash space containing a ringbuffer array is allocated on the client, and the flash space format is: head+body;

[0027] The head includes: ringbuffer size + ringbufferwrite offset + ringbufferread offset + Code num; the body includes: a ringbuffer array.

[0028] In a preferred embodiment of the present invention, after saving the new code list to the flash space, the following steps are included:

[0029] A code list upload size threshold is preset based on the MTU size, and the average size of each log is obtained based on historical data;

[0030] Based on the upload size threshold of the Code list and the average size of each log entry, an upload limit is obtained, as shown in Formula 1:

[0031]

[0032] In the formula, S is the upload limit, Y is the upload size threshold of the Code list, and P is the average size of each log entry;

[0033] The code list upload size threshold must reserve space for log format and transmission protocol.

[0034] In a preferred embodiment of the present invention, after obtaining an upload limit, the method includes:

[0035] By setting the upload limit for the new code list, the number of codes in the new code list is controlled during each batch upload, and the upload limit is used as the trigger number.

[0036] In a preferred embodiment of the present invention, determining whether the triggering condition is met includes:

[0037] Obtain the number of codes in the code list and compare the number of codes with the set trigger count;

[0038] If the number of Codes is greater than the set number of triggers, then the triggering condition is considered met.

[0039] If the number of Codes is less than or equal to the set trigger number, then obtain the time node of the last upload of the new Code list, and get an interval time based on the current time node;

[0040] Determine whether the interval time is greater than the set trigger time. If so, the trigger condition is considered to be met.

[0041] In a preferred embodiment of the present invention, after storing the Codes in the new Code list directly as Codes, the method includes:

[0042] When the server executes a specific task, it restores the logs based on the uploaded code logs and the uploaded code list.

[0043] The specific tasks include: displaying, querying, statistically analyzing, and tracking server-side logs.

[0044] In a preferred embodiment of the present invention, log recovery includes:

[0045] Map each code in the uploaded code list to the original log, and find the order relationship of the logs based on the codes carried in the uploaded code logs;

[0046] Based on the stated order, the logs obtained by mapping the uploaded Code list are combined with the uploaded Code logs to complete log recovery.

[0047] A method for batch uploading client logs based on code mapping, comprising:

[0048] Definition Unit: Used to define the correspondence between each type of log and the code;

[0049] Mapping unit: used to continuously acquire logs to be uploaded, and divide the logs to be uploaded into mappable logs and unmappable logs; according to the correspondence, map the mappable logs one by one into Code, and let the unmappable logs carry the Code to obtain the Code logs to be uploaded, and at the same time continuously add the Code to the Code list to obtain a new Code list;

[0050] Storage unit: used to allocate flash space on the client for storing logs and code, and save the code log to be uploaded and the new code list to the flash space;

[0051] Batch upload unit: used to upload the code logs to be uploaded to the server as normal logs. For the new code list, it is determined whether the triggering condition is met. If the condition is met, the codes in the new code list are uploaded to the server in batches. After the server completes the reception, it obtains the uploaded code logs and the uploaded code list, and stores the codes in the uploaded code list directly in code form.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] (1) This invention divides each log into mappable logs and unmappable logs, maps the mappable logs to Code and adds them to the Code list, and allows the unmappable logs to carry Code. Only a small number of parameters, such as non-fixed character parameters, are unmappable, while most of the logs are mappable. Therefore, most logs are mapped to Code. When uploading logs, the unmappable logs are uploaded with Code as normal logs. The Code list is uploaded in batches after the trigger conditions are met, which can greatly reduce the pressure on concurrency, storage and bandwidth. In actual tests, it can reduce by 90%. Therefore, the server's carrying capacity can be significantly improved without expanding the log system.

[0054] (2) In this invention, the Code list is stored directly in the form of Code on the server. Only when the server performs operations such as log display, query, statistics, big data analysis, and message link tracing, the Code is mapped to log for processing, thereby greatly reducing the storage pressure.

[0055] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the steps of the client-side batch log upload method based on code mapping provided by this invention. Detailed Implementation

[0057] The client-side log batch upload method based on code mapping provided by this invention, such as... Figure 1 As shown, it includes the following steps:

[0058] Step S1: Define the correspondence between each type of log and the code;

[0059] Step S2: Continuously acquire logs to be uploaded and divide them into mappable logs and unmappable logs;

[0060] Step S3: Map the mappable logs to Code one by one according to the correspondence, and let the unmappable logs carry the Code to obtain the Code logs to be uploaded. At the same time, continuously add the Code to the Code list to obtain a new Code list.

[0061] Step S4: Allocate flash space on the client to store logs and code, and save the code logs to be uploaded and the new code list to the flash space;

[0062] Step S5: Upload the Code logs to be uploaded to the server as normal logs. For the new Code list, determine whether the triggering conditions are met. If they are met, upload the Codes in the new Code list to the server in batches.

[0063] Step S6: After the server completes the reception, it obtains the uploaded Code log and the uploaded Code list, and stores the Code in the uploaded Code list directly in Code format.

[0064] In step S1 above, when defining the correspondence between each type of log and the Code, the following is included:

[0065] If the log type is without parameters, the mapping will be performed directly according to the preset correspondence.

[0066] If the log type has parameters, then further determine the type of the parameters.

[0067] If the parameter type is a non-floating-point number, the correspondence between the log and the Code is: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number...;

[0068] If the parameter type is a floating-point number, determine whether the floating-point number is necessary. If it is, convert the floating-point number to two integers; otherwise, convert the floating-point number to integers.

[0069] Furthermore, when further determining the type of the parameters, the following steps are also included:

[0070] If the parameter type is an exhaustive list of characters, then the character list is converted into an enumeration. The correspondence between the log and the Code is: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number...;

[0071] If the parameter type is not a fixed character parameter, the processing flow is as follows:

[0072] If the log format corresponding to Code 12102 is: mps recv unknown skip url.[%s];

[0073] A log file was retrieved: mps recv unknown skip url.[https: / / www.google.com];

[0074] Where https: / / www.google.com is the non-fixed character parameter, i.e., the log cannot be mapped. The correspondence between the log and the code is: 12102.[https: / / www.google.com].

[0075] In step S4 above, when allocating flash space on the client for storing logs and code, the following is included:

[0076] Allocate flash space containing the ringbuffer array on the client side. The flash space format is: head+body;

[0077] The head includes: ringbuffer size + ringbufferwrite offset + ringbufferreadoffset + Code num; the body includes: the ringbuffer array.

[0078] In step S4 above, after saving the new code list to the flash space, the following steps are included:

[0079] A code list upload size threshold is preset based on the MTU size, and the average size of each log is obtained based on historical data;

[0080] Based on the code list upload size threshold and the average size of each log entry, an upload limit is obtained, as shown in Formula 1:

[0081]

[0082] In the formula, S is the upload limit, Y is the code list upload size threshold, and P is the average size of each log entry;

[0083] The code list upload size threshold needs to reserve space for log format and transmission protocol.

[0084] Furthermore, after obtaining an upload limit, it includes:

[0085] By setting an upload limit for the new code list, the number of codes in the new code list can be controlled during each batch upload, and the upload limit can be used as the trigger number.

[0086] In step S5 above, determining whether the triggering condition is met includes:

[0087] Get the number of codes in the code list and compare it with the set trigger count;

[0088] If the number of Codes is greater than the set number of triggers, the triggering condition is considered met.

[0089] If the number of codes is less than or equal to the set trigger number, then obtain the time node of the last new code list upload and get an interval time based on the current time node;

[0090] Determine whether the interval time is greater than the set trigger time. If so, the trigger condition is considered to be met.

[0091] In step S6 above, after storing the Codes in the new Code list directly as Codes, the following steps are included:

[0092] When the server executes a specific task, log recovery is performed based on the uploaded code logs and the uploaded code list;

[0093] Specific tasks include: displaying, querying, statistically analyzing, and tracking server-side logs.

[0094] Furthermore, log recovery includes:

[0095] Map each code in the uploaded code list to the original log, and find the order of the logs based on the codes carried in the uploaded code logs.

[0096] Based on the order of the uploaded code list, the logs obtained by mapping are combined with the uploaded code logs to complete the log recovery.

[0097] The client-side log batch upload method based on code mapping provided by this invention includes: a definition unit, a mapping unit, a saving unit, and a batch upload unit.

[0098] Definition Unit: Used to define the correspondence between each type of log and the code.

[0099] Mapping Unit: Used to continuously acquire logs to be uploaded and divide them into mappable logs and unmappable logs; map mappable logs one by one to Code according to the correspondence, and let unmappable logs carry Code to obtain Code logs to be uploaded. At the same time, the Code is continuously added to the Code list to obtain a new Code list.

[0100] Storage Unit: Used to allocate flash space on the client to store logs and code, and save the code logs to be uploaded and the new code list to the flash space.

[0101] Batch Upload Unit: Used to upload code logs to the server as normal logs. For new code lists, it checks whether the triggering conditions are met. If they are met, the codes in the new code list are uploaded to the server in batches. After the server completes the reception, it obtains the uploaded code logs and the uploaded code list, and stores the codes in the uploaded code list directly as codes.

[0102] The following embodiments are further illustrations of the present invention, but the scope of the present invention is not limited thereto.

[0103] Mapping rules between code and logs:

[0104] 1. In this embodiment, the correspondence between each log and the code is first defined. This is determined in the design phase and adjusted during the development process.

[0105] For example, during the design phase, logs corresponding to Code (0-10) were planned, but during development, it was found that the definitions were missing, and Code (10-15) needed to be added, or Code (0-5) needed to be modified.

[0106] The following list of enumeration values ​​is the Code, half of which was added during development because it could not be fully included in the design phase.

[0107] PLAYING_NORMAL = 0;

[0108] PLAYING_CATON = 1;

[0109] BLACK_SCREEN = 2;

[0110] NO_AUDIO = 3;

[0111] AU_AND_VI_ASYN = 4;

[0112] PLAYER_ERROR_INIT = 5;

[0113] URL_IS_ILLEGAL = 6;

[0114] URL_CANNOT_ACCESSIBLE = 7;

[0115] URL_PROTOCAL_NOT_SUPPORT=8;

[0116] URL_DOWNLOAD_TIMEOUT = 9;

[0117] BITRATE_TOO_BIGGER = 10;

[0118] FORMAT_PARSE_ERROR = 11;

[0119] AUDIO_ERROR_DECode = 12;

[0120] VIDEO_ERROR_DECode = 13;

[0121] SUBTITLES_ERROR_DECode=14;

[0122] VOLUME_SET_FAILED = 15;

[0123] REPLAY_FAILED = 16;

[0124] OUT_OF_MEMORY = 17;

[0125] THROW_FRAME = 18;

[0126] PLAYING_MOSAIC = 19;

[0127] LOAD_PROGRAM_LIST_FAILED=20.

[0128] 2. In this embodiment, there are two main categories of logs. One type of log has no parameters and can be mapped directly according to preset rules. The other type of log has parameters, which can be further divided into several subcategories, including: parameters are numbers (floating-point and non-floating-point), exhaustive character lists, and non-fixed characters.

[0129] 3. For non-floating-point number formats, the log correspondence is: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number...

[0130] 4. For floating-point number formats, there are two processing methods:

[0131] One option is that it is not necessary (the server does not need high-precision floating-point data), in which case the floating-point number format is converted to an integer;

[0132] If necessary, the floating-point number format will be converted to two integers.

[0133] 5. For a list of characters that can be enumerated, convert the character list of the parameters into an enumeration (an enumeration is an integer). Then the log format is: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number...

[0134] 6. For logs with non-fixed character parameters, the processing flow is as follows:

[0135] First, map the mappable portions of the log to Code:

[0136] This mapping specifies which log entry corresponds to the code, for example:

[0137] The log format corresponding to Code(12102) is: mps recvunknown skip url.[%s].

[0138] This example illustrates the possible upload scenarios for this log entry: mps recv unknown sk ipurl.[https: / / www.google.com]. The URL length is not fixed, but is generally limited to 1024 characters.

[0139] After mapping according to the rules of this embodiment, it becomes: 12102.[https: / / www.google.com]. At the same time, 12102 is also included in the batch uploaded code list, so that the order of messages can still be guaranteed after the code is mapped back to the log.

[0140] Logs that cannot be mapped, such as "12102.[https: / / www.google.com]", are uploaded as normal logs along with a code. Because the code list upload is done in batches and has a certain delay, there is a certain order between it and the normally uploaded logs. Including the code here makes it easier for the server to find the log order relationship when performing cleanup later.

[0141] In this embodiment, except for logs with non-fixed character parameters, all other logs are converted to code. When uploading, the code is simply added to the code list. Similarly, for logs with non-fixed parameters, only the non-fixed parameters cannot be converted to code; the rest of the log can be converted to code.

[0142] The above outlines the mapping rules between logs and code.

[0143] Client upload processing flow:

[0144] 1. The client first allocates a flash space, roughly in the format: head(ringbuffer size + ringbuffer write offset + ringbuffer read offset + Code num) + body(ringbuffer array).

[0145] 2. In this embodiment, if the logs are all formatted as: Code (2 bytes) + Quantity (1 byte) + Number (4 bytes) + Number (4 bytes), then the size of a log is 11 bytes. The average size of each log can be obtained from historical data.

[0146] 3. In this embodiment, if each upload does not exceed the MTU, it is calculated based on 1400 (leaving some space for log format and transmission protocol). 1400 / 11Byte = 127, so an upper limit of 120 can be set for the code list.

[0147] The resulting flash memory size is typically twice the MTU, leaving ample space.

[0148] 5. In this embodiment, the flash space is contiguous and can be understood as an array. It is generally managed by using a ring buffer.

[0149] This ringbuffer is a circular buffer implemented using an array. It is controlled by read / write pointers and the total size to achieve the effect of a circular buffer.

[0150] 6. The terminal saves the logs that need to be uploaded to the flash space. Upload is triggered under two conditions:

[0151] First is the time: generally, it will be every certain period of time (30 minutes). This time is set according to the actual needs. If the concurrent pressure is high, the time can be extended appropriately; if there are certain requirements for the delay of logs, the time can be shortened appropriately, thereby speeding up the upload frequency.

[0152] Secondly, the quantity: once the Code reaches the upper limit (100), the upload will still be triggered.

[0153] 7. In this embodiment, the upload protocol can be customized and can be either the existing common HTTP protocol or other protocols.

[0154] 8. In this embodiment, based on actual test results: compared to normal log upload, mapping most logs to code before uploading reduces server bandwidth and storage by approximately 90% (this percentage is directly related to terminal logs).

[0155] Server-side receiving and processing flow:

[0156] 1. After receiving the code list, the server stores it directly according to the code list, instead of storing the previous logs. Storing it directly according to the code list means that the code is not converted into logs. This is because converting it into logs would put pressure on the storage system.

[0157] 2. The server only maps code to logs for processing when performing log display, querying, statistics, big data analysis, message chain tracing, etc.

[0158] The display, query, statistics, and big data analysis of server-side logs can all be done on ELK, and message chain tracing can be achieved through SkyWalking.

[0159] In this embodiment, the logs are displayed over a period of time, or the logs of a specific day for a particular serial number (SN). ELK retrieves a list of all codes uploaded by the SN on a specific day, then calls a custom script in this embodiment to reverse-map the codes back to logs, and then ELK displays them normally.

[0160] 3. Mapping code to logs is simply the reverse parsing process of the rules described earlier:

[0161] The mapping rules described earlier involve the client converting logs that would normally be uploaded into code, based on its defined code and logs. The server-side defines a script that converts the code back into logs. Essentially, the rules are the same; this set of rules is essentially a standard. The server-side processing is as follows:

[0162] 1. The log format corresponding to Code(12102) is: mps recv unknown skip url.[%s];

[0163] 2. After client-side mapping, it becomes: 12102.[https: / / www.google.com];

[0164] 3. The server-side translation script will list all the logs and formats corresponding to the code;

[0165] 4. When parsing the code list, code 12102 is found to be a non-fixed parameter log.

[0166] 5. As mentioned earlier, logs with non-fixed parameters will be uploaded separately, and ELK will store these logs separately from the Codelist;

[0167] 6. Locate the separately uploaded non-fixed parameter log using the code: 12102. [https: / / www.google.com]

[0168] 7. Then the script uses the mapping relationship between Code: 12102 and mps recvunknown skip url.[%s]

[0169] 8. Then you can output "mps recvunknown skip url.[https: / / www.google.com]"

[0170] 9. Logs with other fixed parameters do not require steps 5-7.

[0171] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for batch uploading client logs based on code mapping, characterized in that, Includes the following steps: Define the correspondence between each type of log and the code; Continuously acquire logs to be uploaded and divide the logs to be uploaded into mappable logs and unmappable logs; According to the correspondence, the mappable logs are mapped one by one to Code, and the unmappable logs are allowed to carry Code to obtain Code logs to be uploaded. At the same time, the Code mapped one by one from the mappable logs and the Code carried by the unmappable logs are continuously added to the Code list to obtain a new Code list. On the client, a flash space is allocated for storing logs and code, and the code log to be uploaded and the new code list are saved to the flash space; The code logs to be uploaded are uploaded to the server as normal logs. For the new code list, it is determined whether the triggering conditions are met. If they are met, the codes in the new code list are uploaded to the server in batches. After the server completes the reception, it obtains the uploaded Code log and the uploaded Code list, and stores the Code in the uploaded Code list directly in Code form; When defining the mapping between each type of log and the code, the following is included: If the log type is without parameters, the mapping will be performed directly according to the preset correspondence. If the log type has parameters, then further determine the type of the parameters. If the parameter type is a non-floating-point number, the correspondence between the log and the Code is: Code + quantity + number + number..., where the Code is 2 bytes, the quantity is 1 byte, and the number is 4 bytes; If the parameter type is a floating-point number, determine whether the floating-point number is necessary. If yes, convert the floating-point number to two integers; otherwise, convert the floating-point number to an integer. Further determination of the parameter type includes: If the parameter type is an exhaustive list of characters, then the character list is converted into an enumeration. The correspondence between the log and the Code is: Code + quantity + number + number..., where the Code is 2 bytes, the quantity is 1 byte, and the number is 4 bytes. If the parameter type is not a fixed character parameter, the processing flow is as follows: If the log format corresponding to Code 12102 is: mps recv unknown skip url.[%s]; A log file was retrieved: mps recv unknown skip url. [ https: / / www.google.com]; Where https: / / www.google.com is the non-fixed character parameter, i.e., the unmappable log, then the correspondence between the log and the code is: 12102. [ https: / / www.google.com].

2. The method for batch uploading client logs based on code mapping according to claim 1, characterized in that, When allocating flash space on the client side for storing logs and code, the following should be included: A flash space containing a ringbuffer array is allocated on the client, and the flash space format is: head+body; The head includes: ringbuffer size + ringbuffer write offset + ringbuffer read offset + Code num; the body includes: a ringbuffer array.

3. The method for batch uploading client logs based on code mapping according to claim 1, characterized in that, After saving the new code list to the flash space, the process includes: A code list upload size threshold is preset based on the MTU size, and the average size of each log is obtained based on historical data; Based on the upload size threshold of the Code list and the average size of each log entry, an upload limit is obtained, as shown in Formula 1: （1）； In the formula, The upload limit is mentioned above. The upload size threshold for the Code list. The average size of each log entry; The code list upload size threshold must reserve space for log format and transmission protocol.

4. The method for batch uploading client logs based on code mapping according to claim 3, characterized in that, After obtaining an upload limit, the following is included: By setting the upload limit for the new Code list, the number of Codes in the new Code list is controlled during each batch upload, and the upload limit is used as the trigger number.

5. The method for batch uploading client logs based on code mapping according to claim 1 or 4, characterized in that, When determining whether the triggering condition is met, the following should be included: Obtain the number of codes in the code list and compare the number of codes with the set trigger count; If the number of Codes is greater than the set number of triggers, then the triggering condition is considered met. If the number of Codes is less than or equal to the set trigger number, then obtain the time node of the last upload of the new Codelist, and get an interval time based on the current time node; Determine whether the interval time is greater than the set trigger time. If so, the trigger condition is considered to be met.

6. The method for batch uploading client logs based on code mapping according to claim 1, characterized in that, After storing the codes in the new code list directly as codes, the process includes: When the server executes a specific task, it restores the logs based on the uploaded Code logs and the uploaded Codelist. The specific tasks include: displaying, querying, statistically analyzing, and tracking server-side logs.

7. The method for batch uploading client logs based on code mapping according to claim 6, characterized in that, When performing log recovery, the following are included: Map each code in the uploaded code list to the original log, and find the order relationship of the logs based on the codes carried in the uploaded code logs; Based on the stated order, the logs obtained by mapping the uploaded Code list are combined with the uploaded Code logs to complete log recovery.

8. A client-side batch log upload system based on code mapping, characterized in that, include: Definition Unit: Used to define the correspondence between each type of log and the code; Mapping Unit: Used to continuously acquire logs to be uploaded, and divide the logs to be uploaded into mappable logs and unmappable logs; map the mappable logs one by one into Code according to the correspondence, and let the unmappable logs carry Code to obtain Code logs to be uploaded, and continuously add the Code mapped one by one from the mappable logs and the Code carried by the unmappable logs to the Code list to obtain a new Code list. Storage unit: used to allocate flash space on the client for storing logs and code, and save the code log to be uploaded and the new code list to the flash space; Batch upload unit: used to upload the code logs to be uploaded to the server as normal logs. For the new code list, it is determined whether the triggering condition is met. If the condition is met, the codes in the new code list are uploaded to the server in batches. After the server completes the reception, it obtains the uploaded code logs and the uploaded code list, and stores the codes in the uploaded code list directly in code form. When defining the mapping between each type of log and the code, the following is included: If the log type is without parameters, the mapping will be performed directly according to the preset correspondence. If the log type has parameters, then further determine the type of the parameters. If the parameter type is a non-floating-point number, the correspondence between the log and the Code is: Code + quantity + number + number..., where the Code is 2 bytes, the quantity is 1 byte, and the number is 4 bytes; If the parameter type is a floating-point number, determine whether the floating-point number is necessary. If yes, convert the floating-point number to two integers; otherwise, convert the floating-point number to an integer. Further determination of the parameter type includes: If the parameter type is an exhaustive list of characters, then the character list is converted into an enumeration. The correspondence between the log and the Code is: Code + quantity + number + number..., where the Code is 2 bytes, the quantity is 1 byte, and the number is 4 bytes. If the parameter type is not a fixed character parameter, the processing flow is as follows: If the log format corresponding to Code 12102 is: mps recv unknown skip url.[%s]; A log file was retrieved: mps recv unknown skip url. [ https: / / www.google.com]; Where https: / / www.google.com is the non-fixed character parameter, i.e., the unmappable log, then the correspondence between the log and the code is: 12102. [ https: / / www.google.com].

Images