Traffic data processing method and device of computer service platform and electronic equipment

Abstract

The application discloses a traffic data processing method and device of a computer service platform, electronic equipment and a storage medium. The method comprises the following steps: obtaining original traffic data in the computer service platform; determining the field type of each field in the original traffic data to obtain the field type of at least one field; determining a target simplification strategy matched with the field type from a preset field expression simplification strategy set, and simplifying the expression form of the corresponding field in the original traffic data based on the target simplification strategy to obtain target traffic data; and executing a target service task according to the target traffic data based on a trained platform task processing model to obtain a task processing result. The method can make the expression form of the field closer to the big model word segmentation coding specification under the premise of retaining the core information of the task and without changing the data coding mode, reduce the number of invalid word units generated in the word segmentation coding process from the root, and avoid the excessive total amount of word units.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus and electronic device for processing traffic data of a computer service platform. Background Technology

[0002] With the increasing application of Large Language Models (LLMs) in fields such as security risk control and behavioral analysis, there is a need to input massive amounts of network traffic data into LLMs for intelligent analysis and in-depth mining. Traffic data is mostly transmitted in the form of standardized protocol messages and structured interface messages, mainly containing core content such as request parameters, response data, device identifiers, user identifiers, and job status codes.

[0003] Existing traffic data formats are designed for machine parsing and generally suffer from issues such as symbol redundancy and excessively long fields. This causes the LLM's built-in tokenizer to easily generate a large number of invalid tokens when segmenting and encoding traffic data, resulting in the total number of tokens exceeding the model's preset context window length limit, making it impossible to fully input the data into the model for analysis. To avoid exceeding the token limit, the industry commonly uses methods such as simple front-end truncation and random sampling to compress traffic data before feeding it into the LLM, thereby controlling the token size. However, this data processing method is prone to losing key operational features and core operational fields in the traffic data, resulting in incomplete model input information and incomplete feature representation. This leads to problems such as large semantic understanding biases and reduced inference and analysis capabilities, ultimately affecting the model's task processing performance. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for processing traffic data of a computer service platform, which can reduce the number of invalid tokens and avoid exceeding the total number of tokens while retaining the core information of the operation and without changing the data encoding method.

[0005] In a first aspect, embodiments of this application provide a method for processing traffic data on a computer service platform, including: Obtain raw traffic data from the computer service platform; Determine the field type of each field in the original traffic data to obtain the field type of at least one field; A target simplification strategy matching the field type is determined from a preset set of field expression simplification strategies, and the expression of the corresponding field in the original traffic data is simplified based on the target simplification strategy to obtain the target traffic data. Based on the trained platform task processing model, target service tasks are executed according to the target traffic data to obtain task processing results.

[0006] In conjunction with the first aspect, in some possible implementations, determining the field type of each field in the original traffic data to obtain the field type of at least one field includes: The raw traffic data is input into a trained field type detection model for processing to detect the field type of at least one field. The field type detection model is trained based on a first set of traffic data samples, and each field in each first traffic data sample is labeled with a corresponding field type label.

[0007] In conjunction with the first aspect, in some possible implementations, determining the field type of each field in the original traffic data to obtain the field type of at least one field includes: Extract the field names and values ​​of each field from the original traffic data; The job type matching the field name is determined from the preset job type mapping table, and the data format matching the field value is determined based on the preset data format determination rules; Based on the matching job type and data format, the field type of the corresponding field is determined from a preset field type mapping table.

[0008] In conjunction with the first aspect, in some possible implementations, the method further includes: Determine the task type of the target service task; Determine the importance level of each field in the raw traffic data based on the task type; The step of determining the target simplification strategy that matches the field type from the preset set of field expression simplification strategies includes: determining the target simplification strategy that matches the field type and the importance level from the preset set of field expression simplification strategies.

[0009] In conjunction with the first aspect, in some possible implementations, determining the importance level of each field in the raw traffic data based on the task type includes: Determine the target field level classification table corresponding to the task type from multiple preset field level classification tables; The importance of each field in the original traffic data is classified according to the target field classification table to obtain the importance level of each field.

[0010] In conjunction with the first aspect, in some possible implementations, determining the importance level of each field in the raw traffic data based on the task type includes: Based on the trained field importance assessment model, the importance of each field in the original traffic data is classified according to the task type to obtain the importance level of each field. The field importance assessment model is trained on the second traffic data sample set corresponding to different task types, and each field in each second traffic data sample is labeled with a corresponding importance level label.

[0011] In conjunction with the first aspect, in some possible implementations, after obtaining the target traffic data, the method further includes: In response to a restoration instruction for a target field in the target traffic data, metadata generated during the simplification of the representation is obtained; Based on the metadata and the target simplification strategy corresponding to the target field, the expression of the target field is restored.

[0012] In conjunction with the first aspect, in some possible implementations, before simplifying the representation of the corresponding fields in the original traffic data based on the target simplification strategy, the method further includes: Determine the maximum number of input lexical units for the platform's task processing model; Based on the target simplification strategy, estimate the total number of encoded tokens in the original traffic data after field simplification; If the estimated total number of encoded tokens is greater than the maximum number of input tokens, the target simplification strategy will be adjusted.

[0013] Secondly, embodiments of this application provide a traffic data processing apparatus for a computer service platform, comprising: The acquisition unit is used to acquire raw traffic data from the computer service platform. A determining unit is used to determine the field type of each field in the original traffic data, so as to obtain the field type of at least one field; A simplification unit is used to determine a target simplification strategy that matches the field type from a preset set of field expression simplification strategies, and to simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data. The execution unit is used to execute target service tasks based on the target traffic data according to the trained platform task processing model, so as to obtain task processing results.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory; The processor is connected to the memory. The aforementioned memory is used to store executable program code; The processor reads the executable program code stored in the memory to run the program corresponding to the executable program code, so as to execute the method provided by the first aspect of the embodiments of this application or any possible implementation of the first aspect.

[0015] Fourthly, embodiments of this application provide a computer storage medium storing a plurality of instructions, which are adapted to be loaded by a processor and executed by the method provided in the first aspect of the embodiments of this application or any possible implementation thereof.

[0016] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the method provided by the first aspect of the embodiments of this application or any possible implementation thereof.

[0017] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following: In one or more embodiments of this application, raw traffic data from a computer service platform is acquired, and the field types of each field in the raw traffic data are determined to obtain the field type of at least one field. Then, a target simplification strategy matching the field type is determined from a preset set of field expression simplification strategies, and the expression form of the corresponding field in the raw traffic data is simplified based on the target simplification strategy to obtain the target traffic data. Finally, based on the trained platform task processing model, the target service task is executed according to the target traffic data to obtain the task processing result. That is, firstly, based on the word segmentation encoding characteristics of the large language model, targeted field expression simplification strategies are designed for different types of fields, and then the field-level expression form is converted in combination with the actual field type of each field in the raw traffic data. Thus, on the one hand, while retaining the core information of the task and without changing the data encoding method, the expression form of the field is made closer to the word segmentation encoding specification of the large model, reducing the number of invalid tokens generated during the word segmentation encoding process from the root and avoiding the total number of tokens exceeding the limit. On the other hand, through the standardized and concise field expression form, the interference of redundant noise data on model judgment is weakened, the identification of key features of traffic data is strengthened, and the efficiency and accuracy of task processing are improved. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1A schematic diagram of the system architecture of a traffic data processing method for a computer service platform provided as an exemplary embodiment of this application; Figure 2 A flowchart illustrating a traffic data processing method for a computer service platform provided as an exemplary embodiment of this application; Figure 3 A flowchart illustrating another method for processing traffic data in a computer service platform, provided as an exemplary embodiment of this application; Figure 4 A flowchart illustrating another method for processing traffic data in a computer service platform, provided as an exemplary embodiment of this application; Figure 5 A flowchart illustrating another method for processing traffic data in a computer service platform, provided as an exemplary embodiment of this application; Figure 6 A schematic diagram of the structure of a traffic data processing device for a computer service platform provided as an exemplary embodiment of this application; Figure 7 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0020] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0021] The terms "first," "second," "third," etc., used in this specification, claims, and the foregoing drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0022] This application provides a method, apparatus, electronic device, storage medium, and computer program product for processing traffic data of a computer service platform.

[0023] Please see Figure 1 , Figure 1 This is a schematic diagram of the system architecture of a traffic data processing method for a computer service platform provided as an exemplary embodiment of this application. For example... Figure 1As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0024] Terminal 101 is a user-side electronic device, including but not limited to laptops, mobile phones, tablets, laptops, and desktop computers. Server 103 can be an electronic device on the computer service platform side, which can deploy a Large Language Model (LLM) for various platform service tasks such as intelligent traffic analysis, network security risk control, risk situation early warning, anti-fraud, and user behavior analysis. These scenarios require the LLM to process massive amounts of traffic data. Traffic data is network interaction message data generated during the interaction between terminal 101 and the computer service platform, mainly including core content such as request parameters, response data, device identifiers, user identifiers, and job status codes.

[0025] In this embodiment, terminal 101 can achieve bidirectional data interaction with server 103 via network 103. It can upload job interaction data and behavioral access data to server 103, and also receive analysis results, risk control instructions, and job push messages from server 103. The core task of this computer service platform is: First, the raw traffic data from the computer service platform is acquired, and the field types of each field in the raw traffic data are determined to obtain the field type of at least one field. This raw traffic data includes various message data such as interface request parameters, job response data, user identification, device terminal information, and job status codes. Next, a target simplification strategy matching the field type is determined from a pre-defined set of field expression simplification strategies. Based on the target simplification strategy, the expression form of the corresponding field in the raw traffic data is transformed to obtain target traffic data. The target traffic data has a smaller data volume than the raw traffic data and is compatible with the LLM word segmentation encoding standard. Finally, based on a trained platform task processing model, a target service task is executed according to the target traffic data to obtain the task processing result. This platform task processing model is an artificial intelligence model built on LLM. The target service task can be determined according to actual task requirements, such as risk warning tasks, anti-fraud tasks, or user behavior analysis tasks. During the execution of the target service task, the built-in word segmenter of the model first segments and encodes the target traffic data to obtain a token sequence. Then, feature extraction and semantic understanding are performed based on the token sequence, and finally the task processing result is output.

[0026] It should be understood that Figure 1 The number of terminals 101, networks 102, and servers 103 in the diagram is only illustrative. Depending on the implementation requirements, there can be any number of terminals 101, networks 102, and servers 103.

[0027] Based on the above system architecture, this application provides a method for processing traffic data on a computer service platform. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating a traffic data processing method for a computer service platform, provided as an exemplary embodiment of this application. The execution entity of this traffic data processing method for the computer service platform can be an electronic device on the computer service platform side. Specifically, the traffic data processing method for the computer service platform includes at least the following steps S202-S208, wherein: S202. Obtain raw traffic data from the computer service platform.

[0028] Raw traffic data refers to the interactive message data generated between user devices and the computer service platform during the operation interaction process. It covers all types of transmission data generated throughout the entire operation interaction process, including but not limited to core content such as request parameters, response data, device identifiers, user identifiers, and operation status codes. The computer service platform can provide users with various service tasks such as traffic data analysis, network security risk control, risk situation early warning, anti-fraud, and user behavior analysis.

[0029] In this embodiment, raw traffic data can be collected through various methods such as platform gateway traffic mirroring, job interface log capture, network port packet listening, and job interaction log retrieval. After the raw traffic data is collected, preprocessing operations such as structured parsing, dirty data cleaning, duplicate data deduplication, abnormal data removal, and basic format standardization can be performed on the collected data to filter out meaningless, invalid, and redundant data, unify the basic data structure, and ensure data standardization and validity, so as to efficiently carry out field type determination and field expression form conversion for the raw traffic data in the future.

[0030] S204. Determine the field type of each field in the original traffic data, and obtain the field type of at least one field.

[0031] In this context, a field is the smallest independent data unit that constitutes traffic data, used to carry single-dimensional job information. A field includes a field name and a field value. The field name identifies the job meaning of the current data unit, and the field value represents the specific content carried by the current data unit. Typically, after structured parsing of the raw traffic data, at least one independent field name and corresponding field value can be separated. At this point, the field type can be determined based on a learning model or mapping table matching. The field type can be configured according to the job attribute represented by the field name and the data format of the field value, including but not limited to key identifier fields, numeric identifier (ID) fields, job enumeration fields, dynamic parameter fields, and encoded string fields (such as strings encoded using hash, base64, etc.).

[0032] S206. Determine the target simplification strategy that matches the field type from the preset set of field expression simplification strategies, and simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data.

[0033] The original traffic data is adapted to the existing machine parsing protocol and interface interaction specifications, while the target traffic data is directly adapted to the LLM word segmentation encoding logic specifications. The set of field expression simplification strategies is designed based on the word segmentation rules and character encoding characteristics (word segmentation encoding specifications) of the Large Language Model (LLM). These are simplification rules specifically designed for the expression of field names and / or field values ​​for different types of fields (which can be considered a data compression strategy that does not change the encoding method). Simplifying the expression of fields is a data lightweighting operation performed while preserving effective information such as job semantics, key features, and interaction logic. The original data encoding method remains unchanged throughout the process; only the expression of field names and field values ​​is converted to adapt to the LLM word segmentation encoding specifications while reducing the data volume.

[0034] Specifically, considering that the root cause of the excessive token problem in LLM when processing traffic data is that the original traffic data is designed for machine parsing and does not conform to the natural language word segmentation encoding logic of the LLM's word segmenter, the word segmenter generates many invalid tokens during the encoding process of non-linguistic traffic data. This is mainly manifested in the following ways: long concatenated strings in the traffic data are split and encoded character by character; individual numbers in numerical strings are encoded independently; recurring job enumeration values ​​are fully encoded each time; and low-information-density data such as dynamic random parameters and encrypted strings are fully encoded. Based on this, the field expression simplification strategy set constructed in this embodiment is a set of lightweight simplification rules with differentiated configurations for the above-mentioned different types of fields. It is used to specifically improve various fields that are prone to generating invalid tokens, ensuring that without losing key job features and job semantics, and without changing the data encoding method, the fields are made to conform to the LLM's word segmentation encoding specifications only by converting the expression of field names and / or field values. This reduces the number of invalid tokens generated during the subsequent LLM encoding process, thereby effectively compressing the total number of tokens to adapt to the LLM context window limitations.

[0035] Optionally, the set of field expression simplification strategies includes at least two of the following strategies: ① Field name abbreviation mapping: Specifically, this involves establishing a mapping dictionary from the field names of key fields to abbreviation identifiers, shortening long field names to shorter ones. For example, for the original fields {"user_identity_id": "account name"} and {"operation_type": "interface name"}, after abbreviation, they become {"uid": "account name"} and {"op": "interface name"} respectively. In this way, the original field names are reduced from approximately 15 tokens to approximately 3 tokens.

[0036] ② Simplify hash or base64 encoded strings. Specifically, simplify long encoded strings by using a prefix + ellipsis truncation method, or by using length + encoding method annotation. For example, for the original hash encoded field: {"session_id":"a1b2c3d4e5f6g7h8xxxxxxl2m3n4o5p6q7r8s9t0"} (the field value is approximately 20 tokens), after truncation it becomes {"session_id":"a1b2c3d4e5f6g7h8..."} (the field value is approximately 8 tokens), and after using length + encoding method annotation it becomes {"session_id":"[HASH_40_CHARS]"} (the field value is approximately 3-4 tokens).

[0037] For raw fields encoded in base64: {"encrypted_payload":"SGVsbG8gV29ybGQxxxxxxxxxxxx9yIGRlbW9uc3RyYXRpb24="} (the field value is approximately 30 tokens), after being annotated using the length + encoding method, becomes {"encrypted_payload":"[BASE64_136_CHARS]"} (the field value is approximately 3-4 tokens).

[0038] ③ Dynamic parameter simplification: Specifically, dynamically changing parameters are replaced with uniform placeholders. For example, the original fields {"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."} (approximately 50 tokens) and {"trace_id":"0a1b2c3d4e5f6789..."} (approximately 20 tokens) are simplified using placeholders to become {"token":"[ENCRYPTED_DATA]"} (approximately 2 tokens) and {"trace_id":"[DYNAMIC_ID]"} (approximately 2 tokens) respectively.

[0039] ④ Convert numeric IDs to integers. Specifically, convert numeric IDs in string format to integer types. For example, the original field is {"uid":"account name"}. Each number in the string value of this field is encoded separately during word segmentation, which is approximately 4 tokens. After simplification, the field becomes {"uid":account name}. This field value is encoded as a whole as an integer during word segmentation, which is 2 tokens, thus saving 50% of the tokens.

[0040] ⑤ Integer mapping of job enumeration values: Specifically, a mapping dictionary is established to map job enumeration values ​​(job codes) to integer numbers, converting recurring string job codes into short numbers. For example, the original fields {"op":"interface name"} (field value is 8 tokens) and {"status":"PROCESSING_SUCCESS"} (field value is 3 tokens) become {"op":42} (field value is 1 token) and {"status":5} (field value is 1 token) after integer mapping, thus saving 70%-90% of tokens. An example mapping dictionary is: {"interface name 1":42, "interface name 2":43, "PROCESSING_SUCCESS":5, "PROCESSING_FAILED":6......}.

[0041] S208. Based on the trained platform task processing model, execute the target service task according to the target traffic data to obtain the task processing result.

[0042] The platform task processing model is an AI model built on LLM (Library Management Model). This model leverages LLM's powerful semantic understanding, text feature extraction, and contextual reasoning capabilities. This model can handle at least one platform service task, including but not limited to: risk identification and early warning tasks for the platform's network environment; violation identification and intelligent anti-fraud tasks for user interaction scenarios; behavior trajectory tracing and situation analysis tasks based on the entire user operation record; and abnormal traffic capture and comprehensive security risk control tasks based on network-wide traffic data. The task processing results depend on the specific task. For example, for network risk early warning tasks, it can accurately output hierarchical risk level judgments, system potential intrusion behavior prompts, vulnerability exposure alarms, and attack trend prediction information; for intelligent anti-fraud tasks, it can automatically generate qualitative classification labels for fraudulent behavior, quantitative scores for interactive security risks, and conclusions on violation tracing; for user behavior analysis tasks, it can completely reconstruct the user's entire operation trajectory, accurately output abnormal operation identification conclusions, behavior deviation judgments, and high-risk behavior markers.

[0043] It should be noted that the platform's task processing model needs to be trained and deployed on the platform in advance. During the model training phase, historical traffic data from various task scenarios on the computer service platform can be collected first to construct training sample data. This historical traffic data covers various typical and edge operation scenarios, such as interface calls, account logins, data queries, and abnormal access. Then, based on the judgment rules, risk control standards, and judgment logic of different platform service tasks, scenario-differentiated data annotation is carried out. Simultaneously, data preprocessing operations such as sample deduplication, noisy data cleaning, balanced allocation of category samples, and sparse sample enhancement are implemented to remove invalid and redundant data and optimize the overall sample quality. On this basis, combined with task-specific training objectives and training sample data, model-oriented parameter fine-tuning, weight optimization, and multi-round iterative convergence training are conducted to drive the model's deep learning of the semantic features of built-in fields in traffic data under different task scenarios, the logic of normal user interaction behavior, the characteristics and patterns of hidden violations, and the evolution logic of potential security risks.

[0044] The platform's task processing model, after thorough training and iteration, possesses comprehensive task processing capabilities, including accurate cross-scenario traffic data analysis, autonomous execution of multi-objective tasks, and in-depth data mining. For example, in network security risk control scenarios, it can efficiently identify concealed network attack payloads, unconventional and abnormal access links, unauthorized access behaviors, and covert data theft operations; in intelligent anti-fraud scenarios, it can deeply mine abnormal fund transfer links, characteristics of bulk registration of fake accounts, clustered illegal bulk operation patterns, and gray industry linkage behaviors; in user behavior trend analysis scenarios, it can accurately dissect continuous user operation behavior sequences and quickly identify abnormal behavior patterns such as abnormal path jumps, low-frequency high-risk operations, and abnormal behavior during specific time periods; in comprehensive risk situation early warning scenarios, it can rely on the linkage analysis of historical behavior data and real-time traffic data to deduce and judge the overall security risk development trend and regional risk diffusion trend of the platform, and complete the prediction of potential security risks and proactive risk prevention in advance, comprehensively improving the platform's intelligent management, risk identification, and security protection effectiveness.

[0045] As described above, the traffic data processing method for a computer service platform provided in this embodiment obtains raw traffic data from the computer service platform, determines the field type of each field in the raw traffic data, obtains the field type of at least one field, then determines a target simplification strategy matching the field type from a preset set of field expression simplification strategies, and simplifies the expression form of the corresponding field in the raw traffic data based on the target simplification strategy to obtain target traffic data; finally, based on a trained platform task processing model, a target service task is executed according to the target traffic data to obtain the task processing result, that is, first based on the large language model's segmentation... The word encoding feature is designed with targeted strategies to simplify the expression of different types of fields. Combined with the actual field types of each field in the original traffic data, the expression form of each field is transformed at the field level. On the one hand, while retaining the core information of the task and without changing the data encoding method, the expression form of the fields is made closer to the word segmentation encoding standard of the large model. This reduces the number of invalid words generated during the word segmentation encoding process of the large model from the root, and avoids the total number of words exceeding the limit. On the other hand, the standardized and concise field expression form weakens the interference of redundant noise data on model judgment, strengthens the identification of key features of traffic data, and helps to improve the efficiency and accuracy of task processing.

[0046] Based on the methods described in the above embodiments, please refer to Figure 3 , Figure 3 This is a flowchart illustrating another traffic data processing method for a computer service platform provided as an exemplary embodiment of this application. The executing entity of this traffic data processing method for the computer service platform can be an electronic device on the computer service platform side. Specifically, the traffic data processing method for the computer service platform includes the following steps S302-S308, wherein: S302. Obtain raw traffic data from the computer service platform.

[0047] The steps S302 and S202 are the same as those described above, and will not be repeated here.

[0048] S304A. Input the raw traffic data into the trained field type detection model for processing to detect the field type of at least one field. The field type detection model is trained based on the first traffic data sample set, and each field in each first traffic data sample is labeled with a corresponding field type label.

[0049] The field type detection model is used to detect the field type of different fields and needs to be trained and deployed in advance. During the training phase of this model, multi-source and multi-type historical traffic data can be collected from a computer service platform and preprocessed, including data cleaning, deduplication, and noise removal. Then, the sample data is decomposed into fields, and each field is manually labeled according to the work specifications to generate training sample data with field type labels. Afterward, the training sample data can be divided into training, validation, and test sets according to a preset ratio. The training set is input into the field type detection model for iterative training. The model automatically learns the message format, character features, semantic distribution, and job association features corresponding to different fields. The validation set is used to fine-tune the model's hyperparameters and adjust the model weights. The test set is used to evaluate the model's generalization ability until the model meets the preset recognition accuracy conditions. Then, iteration stops, and the final model parameters are saved, completing the training of the field type detection model.

[0050] In some embodiments, in addition to determining the field type based on the model as shown in step S304 above, it can also be determined by mapping table matching. See also... Figure 4 , Figure 4 This is a flowchart illustrating another traffic data processing method for a computer service platform provided as an exemplary embodiment of this application. After step S302 above, the traffic data processing method for the computer service platform includes the following steps S3042B-S3046B, wherein: S3042B. Extract the field names and values ​​of each field from the raw traffic data; S3044B. Determine the job type that matches the field name from the preset job type mapping table, and determine the data format that matches the field value based on the preset data format determination rules; S3046B. Determine the field type of the corresponding field from the preset field type mapping table based on the matching job type and data format.

[0051] The field name represents the operational meaning and functional attributes of the data field, serving as an identifier to distinguish different data dimensions. The field value is the actual operational data content carried under the field name, reflecting specific data information. When the raw traffic data is structurally parsed and split through methods such as message parsing and protocol decomposition, the field name corresponding to each field and the actual storage content carried by the field can be separated. The operational type mapping table pre-stores the corresponding relationship between field names and their respective operational categories under various operational scenarios. Subsequently, the operational type to which a field belongs can be determined by searching for the actual field name. The data format determination rules can comprehensively determine the data form of the field value by combining multiple dimensions such as character composition, numerical range, encoding form, and length characteristics to determine the format category to which the field value belongs, such as text, number, time, encoding, identifier, and other data formats. The field type mapping table pre-establishes a binding correspondence between various operational types, data formats, and standard field types. By using the two types of information obtained from the above matching as joint search conditions, the unique corresponding standard field type can be quickly matched by looking up the table, thus completing the field type determination.

[0052] In this embodiment, various mapping tables are configured, and the field type identification is completed by mapping table matching. On the one hand, there is no need to rely on deep learning models to carry out complex feature calculations. The overall implementation logic is simple, the computing power is low, the processing response is fast, and the deployment difficulty is low. It is especially suitable for lightweight traffic data processing scenarios where the job fields are regular and uniform and the field types are relatively fixed. On the other hand, this method can be flexibly switched with the model recognition method to adapt to different computing power resources and recognition accuracy requirements. For example, in low-precision and fast processing scenarios such as preliminary sorting of regular batch traffic data and rapid classification of standardized fields, the mapping table matching method is preferred to improve processing efficiency. When faced with heterogeneous traffic data with messy fields, diverse formats, and complex job scenarios, the model recognition method can be switched to, relying on the powerful feature learning capabilities of the model to ensure the accuracy of field type identification.

[0053] S306. Determine the target simplification strategy that matches the field type from the preset set of field expression simplification strategies, and simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data.

[0054] Steps S306 and S206 are the same as those mentioned above, and will not be repeated here.

[0055] S308. Based on the trained platform task processing model, execute the target service task according to the target traffic data to obtain the task processing result.

[0056] The steps S308 and S208 are the same, and will not be repeated here.

[0057] Based on the methods described in the above embodiments, please refer to Figure 5 , Figure 5 This is a flowchart illustrating another traffic data processing method for a computer service platform provided as an exemplary embodiment of this application. The executing entity of this traffic data processing method for the computer service platform can be an electronic device on the computer service platform side. Specifically, the traffic data processing method for the computer service platform includes the following steps S402-S410, wherein: S402. Obtain raw traffic data from the computer service platform.

[0058] S404. Determine the field type of each field in the original traffic data to obtain the field type of at least one field.

[0059] The steps S402 and S202 are the same, and the steps S404 and S204 are the same, so they will not be repeated here.

[0060] S406. Determine the task type of the target service task, and determine the importance level of each field in the raw traffic data based on the task type.

[0061] Different downstream service tasks have significantly different requirements and focuses regarding the selection of data fields. The importance level of each field in the raw traffic data can be categorized based on the attributes of the target service task to be executed. For example, fields can be classified as important, ordinary, and redundant from highest to lowest importance. This provides a basis for subsequent targeted data filtering, simplification, compression, and extraction. Generally, the higher the importance level of a field, the higher its participation and weight in the data parsing, computational analysis, and result judgment of the corresponding service task, and the higher its priority for complete data retention. Low-intensity field simplification, such as abbreviation of field names, can be adopted. Conversely, the lower the importance level, the weaker its impact on the task processing results, and high-intensity field simplification, such as trimming and replacement, can be prioritized.

[0062] In some embodiments, step S406 specifically includes: Determine the target field level classification table corresponding to the task type from multiple preset field level classification tables; The importance of each field in the original traffic data is classified according to the target field classification table to obtain the importance level of each field.

[0063] The field level classification table can be pre-configured with priority and weight standards for traffic data fields under different task types. This clarifies the role and priority of different fields in their respective operational scenarios and clearly defines the criteria for determining key, ordinary, and redundant fields. During actual execution, the corresponding classification table can be retrieved based on the current task type to quickly and automatically classify all fields, eliminating the need for manual classification and effectively improving the efficiency and standardization of field level classification. Optionally, unknown new fields not recorded in the classification table can be temporarily categorized as ordinary fields. Information on these fields can be summarized and stored for future updates to the field level classification table based on actual operational needs, continuously expanding the rule coverage.

[0064] In other embodiments, step S406 specifically includes: Based on the trained field importance assessment model, the importance of each field in the original traffic data is classified according to the task type to obtain the importance level of each field. The field importance assessment model is trained on the second traffic data sample set corresponding to different task types, and each field in each second traffic data sample is labeled with the corresponding importance level label.

[0065] During the model training phase, the model can autonomously learn the correlation patterns, job dependencies, and data contribution characteristics between different service tasks and traffic fields, continuously fitting the logic for judging the importance of fields. In practical applications, only the task type and traffic field information need to be input, and the importance level of each field can be automatically output. Compared with the table lookup matching method, this method can adapt to scenarios with flexible job rules and complex and varied field combinations, and has stronger adaptive and generalization capabilities, accurately completing the importance level assessment of non-standardized fields.

[0066] S408. Determine the target simplification strategy that matches the field type and importance level from the preset set of field expression simplification strategies, and simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data.

[0067] The field simplification strategy is no longer selected based on a single dimension of field type, but rather by combining field type and field importance level for joint matching. It comprehensively considers the data attributes of the field itself and its participation value in the corresponding service task, making the selected simplification strategy more in line with the actual task requirements. It can efficiently complete the lightweight simplification of traffic data and reduce the total number of tokens, while ensuring the complete retention of core field information. It effectively prevents problems such as missing key data and distorted understanding and analysis caused by inappropriate simplification.

[0068] For example, the transaction address field has significantly different value in anti-fraud risk control tasks and regular traffic statistics tasks. In anti-fraud risk control tasks, this field is classified as an important field, while in regular traffic statistics tasks, it is classified as a common or even redundant field. The same field, depending on its level of importance in different task scenarios, may also employ significantly different simplification strategies. For instance, in anti-fraud risk control tasks, the transaction address field is typically simplified by retaining complete information and only using name abbreviations, while in regular traffic statistics tasks, it can be simplified by using high-intensity strategies such as prefix truncation. This allows for on-demand control of data simplification, balancing the need for data integrity and lightweight compression in different task scenarios.

[0069] S410. Based on the trained platform task processing model, execute the target service task according to the target traffic data to obtain the task processing result.

[0070] The steps S410 and S208 are the same as those described above, and will not be repeated here.

[0071] Furthermore, this embodiment also supports dynamically adjusting the simplification strategy based on the actual context window size of the model. That is, before the above step "simplifying the expression of corresponding fields in the original traffic data based on the target simplification strategy", the traffic data processing method of this computer service platform also includes: Determine the maximum number of input tokens for the platform's task processing model; Based on the target simplification strategy, estimate the total number of encoded tokens in the original traffic data after field simplification; If the estimated total number of encoded tokens is greater than the maximum number of input tokens, the target simplification strategy will be adjusted.

[0072] The maximum number of input tokens is the maximum number of tokens that the platform's task processing model context window can hold, used to constrain the encoding length and feature scale of traffic data. Specifically, based on the simplification strategies adapted to each field, the average encoding ratio corresponding to different field types and simplification methods can be statistically analyzed. Combined with the original field data length and the number of fields, the total number of encoded tokens (tokens) of all fields after simplification can be quickly estimated. When the estimated total number of encoded tokens exceeds the maximum number of tokens that the model can hold, the system can automatically increase the simplification intensity of non-critical fields, achieving early adaptation between the simplification strategy and model input constraints. For example, increasing the field pruning ratio and expanding the scope of placeholder usage can keep the overall number of tokens within the model's allowable range, avoiding abnormal issues such as model processing errors and inference lag caused by exceeding input data limits, and ensuring the stability and smoothness of traffic data parsing and service task execution.

[0073] Furthermore, all field simplification operations in this embodiment retain the ability to trace the original data, supporting the restoration of the original field content as needed. That is, after obtaining the target traffic data, the traffic data processing method of this computer service platform may further include: In response to a restoration instruction for a target field in the target traffic data, the metadata generated during the simplification of the representation is retrieved; Based on the metadata and the target simplification strategy corresponding to the target field, the expression of the target field is restored.

[0074] The field simplification process is reversible throughout, with various restoration metadata being retained and archived independently during the simplification operation. For example, field name mappings are saved synchronously with the simplified data, enumeration value lookup tables are version-managed, original hash values ​​are retained for data consistency verification, and various placeholders are labeled with their specific types to clarify the basis for restoration. Subsequently, the retained metadata is used to match the corresponding field simplification logic, and the original field content is accurately restored by reverse deduction according to the established logic. This design can meet the needs of daily lightweight data transmission and efficient model inference, and can also completely restore the original traffic data in scenarios such as data verification, problem tracing, and job review, effectively balancing data simplification and optimization efficiency with full-process data traceability capabilities.

[0075] For example, if the target field is a job enumeration value (job code) and the target simplification strategy is integer mapping of enumeration values, the system will store the corresponding enumeration mapping dictionary metadata when simplifying the field. When a restoration instruction for this field is received, the system can reverse the parsing based on the mapping relationship to restore the original field content, i.e., the original job code. If the target field is a long string of random request identifiers (hash value) and the target simplification strategy is placeholder replacement, when a restoration instruction is received, the system can restore the original field content, i.e., the hash value, by using the marked placeholder type and the original hash record.

[0076] Based on the computer service platform traffic data processing method described in the above embodiments, please refer to [link to relevant documentation]. Figure 6 , Figure 6 This application provides an exemplary embodiment of a traffic data processing apparatus for a computer service platform. This traffic data processing apparatus for the computer service platform can be implemented as an electronic device on the computer service platform side. Specifically, the traffic data processing apparatus 500 for the computer service platform includes an acquisition unit 501, a determination unit 502, a simplification unit 503, and an execution unit 504, wherein: Acquisition unit 501 is used to acquire raw traffic data from the computer service platform; The determining unit 502 is used to determine the field type of each field in the original traffic data, so as to obtain the field type of at least one field; The simplification unit 503 is used to determine the target simplification strategy that matches the field type from a preset set of field expression simplification strategies, and to simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data. The execution unit 504 is used to execute target service tasks based on the target traffic data according to the trained platform task processing model, so as to obtain task processing results.

[0077] In one possible implementation, the determining unit 502 is specifically used for: The raw traffic data is input into a trained field type detection model for processing to detect the field type of at least one field. This field type detection model is trained based on a first set of traffic data samples, and each field in each first set of traffic data samples is labeled with a corresponding field type label.

[0078] In one possible implementation, the determining unit 502 is specifically used for: Extract the field names and values ​​of each field from the raw traffic data; The job type matching the field name is determined from the preset job type mapping table, and the data format matching the field value is determined based on the preset data format determination rules; Based on the matching job type and data format, determine the field type of the corresponding field from the preset field type mapping table.

[0079] In one possible implementation, the simplification unit 503 is also used for: Determine the task type for the target service task; Determine the importance level of each field in the raw traffic data based on the task type; The above step "determine the target simplification strategy that matches the field type from the preset set of field expression simplification strategies" includes: determining the target simplification strategy that matches the field type and the importance level from the preset set of field expression simplification strategies.

[0080] In one possible implementation, the simplified unit 503 is specifically used for: Determine the target field level classification table corresponding to this task type from multiple preset field level classification tables; Based on the target field ranking table, the importance of each field in the original traffic data is ranked to obtain the importance level of each field.

[0081] In one possible implementation, the simplified unit 503 is specifically used for: Based on the trained field importance assessment model, the importance of each field in the original traffic data is classified according to the task type to obtain the importance level of each field. The field importance assessment model is trained on the second traffic data sample set corresponding to different task types, and each field in each second traffic data sample is labeled with the corresponding importance level label.

[0082] In one possible implementation, after obtaining the target traffic data, the simplification unit 503 is further used to: In response to a restoration instruction for a target field in the target traffic data, the metadata generated during the simplification of the representation is retrieved; Based on the metadata and the target simplification strategy corresponding to the target field, the expression of the target field is restored.

[0083] In one possible implementation, before simplifying the representation of the corresponding field in the original traffic data based on the target simplification strategy, the simplification unit 503 is further configured to: Determine the upper limit of the encoding feature input for the task processing model of this platform; Based on this simplification strategy, the number of encoded features of the original traffic data after the representation transformation is estimated; If the estimated number of encoded features is greater than the upper limit of the encoded feature input, the simplification strategy for the target will be adjusted.

[0084] As described above, the computer service platform traffic data processing device 500 provided in this application embodiment acquires raw traffic data from the computer service platform through an acquisition unit 501, determines the field type of each field in the raw traffic data through a determination unit 502, obtains the field type of at least one field, then a simplification unit 503 determines a target simplification strategy matching the field type from a preset set of field expression simplification strategies, and simplifies the expression form of the corresponding field in the raw traffic data based on the target simplification strategy to obtain target traffic data; finally, an execution unit 504 executes a target service task based on the target traffic data according to a trained platform task processing model to obtain task processing. As a result, based on the word segmentation and encoding characteristics of the large language model, a field expression simplification strategy was designed for different types of fields. Then, the actual field types of each field in the original traffic data were combined to transform the field-level expression form. On the one hand, while retaining the core information of the task and without changing the data encoding method, the expression form of the fields is made closer to the word segmentation and encoding specifications of the large model. This reduces the number of invalid tokens generated during the word segmentation and encoding process of the large model from the root, and avoids the total number of tokens exceeding the limit. On the other hand, through the standardized and concise field expression form, the interference of redundant noise data on model judgment is weakened, the identification of key features of traffic data is strengthened, and the efficiency and accuracy of task processing are improved.

[0085] It should be noted that the division of the module units in the traffic data processing device 500 of the computer service platform described above is only for illustrative purposes. In other embodiments, the traffic data processing device 500 of the computer service platform can be divided into different module units as needed to complete all or part of the functions of the traffic data processing device 500 of the computer service platform. The implementation of each module unit in the traffic data processing device 500 of the computer service platform provided in this application embodiment can be in the form of a computer program. The computer program can run on a terminal or server. The program modules constituted by the computer program can be stored in the memory of the terminal or server. When the computer program is executed by the processor, it implements all or part of the steps of the traffic data processing method of the computer service platform described in this application embodiment.

[0086] This application also provides an electronic device, which can be a server, and its internal structure diagram can be as follows: Figure 7As shown, this electronic device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. The processor executes computer programs to implement a traffic data processing method for a computer service platform.

[0087] Those skilled in the art will understand that Figure 7 The structures shown are merely block diagrams of some structures related to the embodiments of this application and do not constitute a limitation on the electronic devices to which the embodiments of this application are applied. Specific electronic devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0088] In one possible implementation, a computer storage medium is provided that stores multiple instructions, which, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium.

[0089] In one possible implementation, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0090] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted through the computer storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0091] It should be noted that the information (including but not limited to UI screenshots, requirement descriptions, image data, user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the modification record information involved in this specification was obtained under full authorization.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0093] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Various modifications and improvements made by those skilled in the art to the technical solutions of the present application without departing from the design spirit of the present application should fall within the protection scope defined by the claims.

[0094] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for processing traffic data on a computer service platform, characterized in that, The method includes: Obtain raw traffic data from the computer service platform; Determine the field type of each field in the original traffic data to obtain the field type of at least one field; A target simplification strategy matching the field type is determined from a preset set of field expression simplification strategies, and the expression of the corresponding field in the original traffic data is simplified based on the target simplification strategy to obtain the target traffic data. Based on the trained platform task processing model, target service tasks are executed according to the target traffic data to obtain task processing results.

2. The method according to claim 1, characterized in that, Determining the field type of each field in the original traffic data to obtain the field type of at least one field includes: The raw traffic data is input into a trained field type detection model for processing to detect the field type of at least one field. The field type detection model is trained based on a first set of traffic data samples, and each field in each first traffic data sample is labeled with a corresponding field type label.

3. The method according to claim 1, characterized in that, Determining the field type of each field in the original traffic data to obtain the field type of at least one field includes: Extract the field names and values ​​of each field from the original traffic data; The job type matching the field name is determined from the preset job type mapping table, and the data format matching the field value is determined based on the preset data format determination rules; Based on the matching job type and data format, the field type of the corresponding field is determined from a preset field type mapping table.

4. The method according to claim 1, characterized in that, The method further includes: Determine the task type of the target service task; Determine the importance level of each field in the raw traffic data based on the task type; The step of determining the target simplification strategy that matches the field type from the preset set of field expression simplification strategies includes: determining the target simplification strategy that matches the field type and the importance level from the preset set of field expression simplification strategies.

5. The method according to claim 4, characterized in that, The step of determining the importance level of each field in the raw traffic data according to the task type includes: Determine the target field level classification table corresponding to the task type from multiple preset field level classification tables; The importance of each field in the original traffic data is classified according to the target field classification table to obtain the importance level of each field.

6. The method according to claim 4, characterized in that, The step of determining the importance level of each field in the raw traffic data according to the task type includes: Based on the trained field importance assessment model, the importance of each field in the original traffic data is classified according to the task type to obtain the importance level of each field. The field importance assessment model is trained on the second traffic data sample set corresponding to different task types, and each field in each second traffic data sample is labeled with a corresponding importance level label.

7. The method according to any one of claims 1-6, characterized in that, After obtaining the target traffic data, the method further includes: In response to a restoration instruction for a target field in the target traffic data, metadata generated during the simplification of the representation is obtained; Based on the metadata and the target simplification strategy corresponding to the target field, the expression of the target field is restored.

8. The method according to any one of claims 1-6, characterized in that, Before simplifying the representation of the corresponding fields in the original traffic data based on the target simplification strategy, the method further includes: Determine the maximum number of input lexical units for the platform's task processing model; Based on the target simplification strategy, estimate the total number of encoded tokens in the original traffic data after field simplification; If the estimated total number of encoded tokens is greater than the maximum number of input tokens, the target simplification strategy will be adjusted.

9. A traffic data processing device for a computer service platform, characterized in that, include: The acquisition unit is used to acquire raw traffic data from the computer service platform. A determining unit is used to determine the field type of each field in the original traffic data, so as to obtain the field type of at least one field; A simplification unit is used to determine a target simplification strategy that matches the field type from a preset set of field expression simplification strategies, and to simplify the expression of the corresponding field in the original traffic data based on the target simplification strategy to obtain the target traffic data. The execution unit is used to execute target service tasks based on the target traffic data according to the trained platform task processing model, so as to obtain task processing results.

10. An electronic device, characterized in that, include: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-8.

11. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method as described in any one of claims 1-8.

12. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer or processor, it causes the computer or processor to perform the method as described in any one of claims 1-8.

Images