A method and system for selecting traffic labels based on state-bound evidence and numerical cross-validation
By using state-bound evidence objects and numerical cross-validation, the problem of unstable label value judgment caused by changes in network traffic distribution is solved, enabling continuous updating and efficient labeling of the network traffic classification model, and ensuring the accuracy and reliability of the classifier.
Patent Information
- Application Number
- CN202611577592.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-13
- Publication Date
- 2026-11-10
AI Technical Summary
Given the continuously changing distribution of network traffic and the limited budget for manual annotation, how can we transform the multi-source technical status of candidate traffic into labeled value evidence objects with stable length and field position, synchronized with the current classifier and the current unlabeled network traffic pool? This would prevent large language models from making sample value judgments based on invalid or fictitious fields, and, without allowing them to generate category labels and enter the training set, allow verified high-value traffic to enter manual annotation to form a closed loop for classifying model updates.
By calculating the multi-source numerical state of candidate network traffic, a state-bound slotted label value evidence object is generated, and large language model judgment, field reference verification and conflict routing are performed to form a high-value candidate pool, expert review path, evidence regeneration path or cache path. Finally, within the manual labeling budget, the final sample set is formed and the system state is updated.
It achieves stability and traceability of the input structure of large language models, avoids the mixing of old classification probabilities and coding space prototypes with new unlabeled pool neighborhood data, ensures the effectiveness of labeled value evidence objects and the reliability of classification models, and reduces input overhead and mislabel contamination.
Smart Images

Figure CN123074946A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence-assisted annotation technology, specifically to a method and system for selecting traffic annotations based on state-bound evidence and numerical cross-validation. Background Technology
[0002] With the deep integration of artificial intelligence and the Internet of Things (IoT), smart home terminals, industrial IoT devices, mobile applications, and cyberattack behaviors are constantly evolving, generating new traffic patterns in the network. Network traffic classification models typically rely on historical labeled samples for training. When device models, application versions, protocol implementations, or attack strategies change, a shift occurs between the historical training data and the current traffic distribution, leading to reduced classification accuracy. Continuously manually labeling new traffic and updating the classification model is crucial for maintaining network traffic identification capabilities.
[0003] Network traffic labeling requires analyzing packet length, arrival time interval, transmission direction, burst structure, session statistics, and protocol behavior, and combining this with knowledge of network security and device behavior to confirm the true category. The size of the unlabeled network traffic pool is usually far greater than the amount of manual labeling that can be undertaken; therefore, it is necessary to prioritize the selection of samples more valuable for classifier updates from this large pool of unlabeled traffic. Existing active learning methods mostly rely on prediction entropy, confidence interval, embedding space distance, cluster coverage, or sample density for selection. A single high uncertainty may simultaneously correspond to true boundary samples, unknown new patterns, isolated outliers, or noise points; simply relying on numerical ranking is insufficient to explain its specific labeling value.
[0004] Large language models possess the ability to make comprehensive judgments and generate explanations based on structured context. Existing solutions attempt to enable large language models to select samples, generate labels, or serialize structured fields into text. However, raw network traffic mainly consists of numerical sequences and statistical features. Directly writing long packet sequences and a large number of floating-point numbers into prompts increases input length, and the field length and position vary with samples, which is detrimental to stable inference. More importantly, the classification probability, class prototype distance, and local neighborhood relationship of network traffic samples depend on the current classifier, the current encoding space, and the current unlabeled network traffic pool, respectively. When the classifier or unlabeled network traffic pool has been updated but old evidence is still used, multiple fields in the same evidence may come from different system states, causing the large language model to make judgments based on invalid information.
[0005] Furthermore, large language models may output sample types inconsistent with numerical evidence, or cite fields that do not exist, are outdated, or are insufficient to support conclusions in their explanations. Simply restricting the output format or using multiple voting methods can only improve output parsingability or statistical stability; it cannot verify whether the large language model truly makes judgments based on the current state of network technology. Writing category labels directly generated by the large language model into the training set may also cause erroneous label contamination, impairing the accuracy and reliability of subsequent classification models.
[0006] Therefore, a sample selection technique for dynamic network traffic classification is needed to form fixed and traceable evidence objects by combining traffic behavior, classifier state, category prototype state, and unlabeled network traffic pool neighborhood state under the same system state; requiring a large language model to return field numbers supporting the judgment; and then using the original values before quantization retained before generating the evidence objects and the current version to perform deterministic verification of the judgment, thereby reliably controlling the path of candidate traffic to manual annotation, expert review, evidence regeneration, or caching. Summary of the Invention
[0007] The technical problems to be solved by this invention are: how to convert the multi-source technical state of candidate traffic into labeled value evidence objects with stable length and field position, synchronized with the current classifier and the current unlabeled network traffic pool, under the conditions of continuously changing network traffic distribution and limited manual labeling budget; how to prevent large language models from making sample value judgments based on invalid fields, fictitious fields or fields that do not meet numerical conditions; and how to hand over the verified high-value traffic to manual labeling and form a closed loop for classification model update without allowing the category labels generated by the large language model to enter the training set.
[0008] To address the aforementioned technical problems, this invention provides a method for selecting network traffic labeling samples based on state-bound evidence objects and numerical cross-validation, comprising the following steps.
[0009] Step S1: Calculate the multi-source numerical state of candidate network traffic. Obtain the labeled network traffic set and the unlabeled network traffic pool, and obtain candidate network traffic from the unlabeled network traffic pool; use the current traffic classifier to obtain the normalized embedding, class probability, and classification uncertainty of the candidate network traffic; based on the normalized embedding, obtain the prototype state of the candidate network traffic relative to the current class prototype set, and the neighborhood state of the candidate network traffic in the current unlabeled network traffic pool.
[0010] Step S2: Generate a state-bound, slot-labeled value evidence object. For continuous numerical fields in the current candidate network traffic set, based on the empirical distribution of the continuous numerical fields in the current candidate network traffic set, encode them into a preset number of quantile levels; generate labeled value evidence objects according to preset field numbers and fixed field slots, and write the current classifier version, the current category prototype version, and the current unlabeled network traffic pool snapshot version into the labeled value evidence object.
[0011] Step S3: Perform large language model judgment, field reference verification, and conflict routing. Input the labeled value evidence object into the large language model to obtain the labeled value, sample type, suggested action, and evidence field number set, which do not contain the true category label of network traffic. Determine the corresponding necessary field set and necessary numerical conditions based on the sample type; check whether the version of the labeled value evidence object is consistent with the current system state, whether the necessary field set is included in the evidence field number set, and use the original values before quantization retained before generating the labeled value evidence object to check the necessary numerical conditions and obtain the deterministic verification result; based on the deterministic verification result, route the candidate network traffic to the high-value candidate pool, expert review path, evidence regeneration path, cache path, or discard path.
[0012] Step S4: Form the final sample set within the manual annotation budget. Within the manual annotation budget, form a sample set to be manually annotated from the high-value candidate pool; if the number of verified high-value candidates exceeds the budget, first select the candidate network traffic with the highest comprehensive value, and then select the remaining candidate network traffic according to the maximum and minimum embedding distance, so as to reduce the repetitive patterns in the final annotation set.
[0013] Step S5: Manually label and synchronously update the system status. Obtain the real human labels for the sample set to be manually labeled, update the labeled network traffic set and traffic classifier, recalculate the category prototype using the updated traffic classifier, and update the classifier version, category prototype version, and unlabeled network traffic pool snapshot version, thus invalidating the old version of labeled valuable evidence objects.
[0014] Furthermore, the current traffic classifier includes a traffic encoder, and the normalized embedding representation of candidate network traffic is as follows:
[0015] ;
[0016] in, For the i-th candidate network traffic, The flow encoder used in round t, For candidate network traffic Normalized embedding; the current traffic classifier outputs class probabilities for C known classes. Candidate network traffic Normalized prediction entropy Represented as:
[0017] ;
[0018] Where c is a known category number, and ε is a positive number to prevent the logarithmic input from being zero.
[0019] Furthermore, for a known category c, the currently labeled network traffic is grouped into categories c. The normalized embeddings of each sample are averaged to obtain the class prototypes.
[0020] ;
[0021] in, Represents a known category Category prototype, Indicates a known category number. This indicates that the current network traffic concentration is of the true category. The number of samples, This indicates the index of the labeled network traffic samples. Indicates the first Human-generated real categories of labeled network traffic samples. Indicates the first The normalized embeddings of each labeled network traffic sample are obtained by the traffic encoder corresponding to the current classifier version; the above summation only applies to samples that satisfy... The labeled network traffic samples are executed, and each normalized embedding used to calculate the same class prototype is generated by the same traffic encoder corresponding to the current classifier version, so that each normalized embedding is in the same embedding space.
[0022] Candidate network traffic To category prototype The cosine distance is:
[0023] ;
[0024] Sort the distances from candidate network traffic to each category prototype in ascending order, and denote the first and second smallest distances as follows: and Novelty and prototype interval They are respectively:
[0025] ;
[0026] Among them, novelty The prototype interval is used to indicate the degree to which candidate network traffic deviates from the prototypes of all known categories. Used to indicate the degree to which candidate network traffic lies among multiple known class prototypes.
[0027] Furthermore, candidate network traffic is identified within the currently unlabeled network traffic pool. The set of k nearest neighbors (i) The median of the k-th nearest neighbor distance of each sample in the current unlabeled network traffic pool is used as the scale. Candidate network traffic Local representativeness Represented as:
[0028] ;
[0029] in, For the normalized embedding of the j-th nearest neighbor sample; when Greater than or equal to the preset novelty threshold and When the value is greater than or equal to a preset local representativeness threshold, the corresponding numerical relationship is used as a necessary numerical condition for the new pattern sample. Greater than or equal to the preset novelty threshold and When the value is less than the preset local representativeness threshold, the corresponding numerical relationship is used as a necessary numerical condition for the noise sample.
[0030] Furthermore, the current system state version tuple is represented as:
[0031] ;
[0032] in, , and Let be the classifier version, category prototype version, and unlabeled network traffic pool snapshot version for the t-th round, respectively; for the j-th continuous field in the current candidate network traffic set, define the empirical distribution function:
[0033] ;
[0034] Where n is the number of current candidate network traffic, Let I(·) be the original value of the j-th consecutive field of the r-th candidate network traffic, and let I(·) be the indicator function; encode the j-th consecutive field of the i-th candidate network traffic into one of L levels:
[0035] ;
[0036] The objects of the labeled value evidence are represented according to fixed slots as follows:
[0037] ;
[0038] in, As candidate network traffic identifiers, , , and These are fixed-length field vectors representing the traffic behavior block, classifier state block, prototype state block, and neighborhood state block, respectively.
[0039] Furthermore, the output of the large language model is represented as:
[0040] ;
[0041] in, To indicate value, For sample type, For suggested actions, The set of evidence field numbers referenced by the large language model; the output structure of the large language model does not have a network traffic true category label field.
[0042] Determine the criteria for valid evidence based on the version recorded in the object of the valued evidence:
[0043] ;
[0044] Among them, state( ) for the object of marked value evidence The version tuple read. This is the current system state version tuple.
[0045] Furthermore, uncertainty thresholds are determined for each candidate network traffic set. Prototype Interval Threshold Novelty threshold and local representative threshold :
[0046] ;
[0047] in, Represents the empirical β quantile; the numerical support indicators for boundary samples, novel pattern samples, representative samples, and noisy samples are as follows:
[0048] ;
[0049] ;
[0050] ;
[0051] ;
[0052] Based on the sample type output by the large language model ,from , , and Select the corresponding numerical support indicator. ; Sample type The corresponding set of necessary fields is denoted as The deterministic verification result is as follows:
[0053] ;
[0054] Among them, when When the value is equal to 1, it indicates that the versions are consistent, the original value meets the necessary numerical conditions of the sample type, and the large language model actually references the corresponding necessary fields.
[0055] Furthermore, when the evidence is valid... When the value is 0, candidate network traffic is routed to the evidence regeneration path; when... For 1 but When the value is 0, candidate network traffic is routed to the expert review path; when... 1. When the labeled value is high and the sample type is a boundary sample, a new pattern sample, or a representative sample, the candidate network traffic is added to the high-value candidate pool:
[0056] ;
[0057] When the number of candidate network traffic in the high-value candidate pool exceeds the manual annotation budget B, the comprehensive value is determined based on the quantitative levels of uncertainty, prototype interval, novelty, and local representativeness in the evidence object:
[0058] ;
[0059] in, , , and These represent the quantization levels for the corresponding fields; first, select the candidate network traffic with the highest overall value, and then select the candidate network traffic with the maximum and minimum embedding distances to the selected sample set according to the following formula:
[0060] ;
[0061] in, As a high-value candidate pool, For the currently selected sample set, until The number of samples reached the manual annotation budget B.
[0062] Furthermore, this invention also provides a network traffic labeling sample selection system based on state-bound evidence objects and numerical cross-validation, comprising: a candidate state calculation module for obtaining the traffic behavior field, classifier state field, category prototype state field, and neighborhood state field of candidate network traffic; an evidence object generation module for performing quantile-level encoding to form a labeled value evidence object carrying the classifier version, category prototype version, and unlabeled network traffic pool snapshot version according to fixed field numbers and fixed field slots; a large language model judgment module for outputting the labeled value, sample type, suggested action, and evidence field number set without real category labels; a deterministic verification and routing module for checking the large language model output based on the current version, necessary field set, and pre-quantization original values, and controlling the path of candidate network traffic entering the high-value candidate pool, expert review, evidence regeneration, caching, or discarding; and a budget selection and update module for forming a sample set to be manually labeled, and updating the traffic classifier, category prototype, and version using real human labels.
[0063] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the above-described method. The present invention also provides a computer-readable storage medium storing a computer program thereon, the computer program, when executed by a processor, implementing the above-described method.
[0064] The beneficial effects are as follows: Compared with the prior art, the present invention has the following significant advantages:
[0065] (1) Since the classifier version, category prototype version and unlabeled network traffic pool snapshot version are written into the labeled value evidence object, and the deterministic validator strictly compares the version tuple recorded in the evidence object with the current system state version tuple, when any of the classifier, category prototype or unlabeled network traffic pool is updated, the old version labeled value evidence object is identified as invalid and enters the evidence regeneration path, thereby avoiding the mixing of old classification probability, old coding space prototype and new unlabeled pool neighborhood data into the same evidence object, and avoiding the problem of large language models judging based on invalid information due to multi-source state asynchrony.
[0066] (2) Since the continuous numerical field is encoded into a preset number of quantile levels based on the empirical distribution of the current candidate network traffic set, and organized into four blocks according to fixed field number and fixed field slot, the traffic behavior, classifier state, prototype state and neighborhood state are different candidate traffic with the same block order and field position, thereby avoiding the problem of field length and position changing with the sample, reducing the input overhead caused by long floating-point numbers and repeated natural language field names, and making the input structure of the large language model stable.
[0067] (3) This invention binds the classifier state, category prototype state, and neighborhood state to a specific version, avoiding the mixing of old classification probabilities, old coding space prototypes, and new unlabeled pool neighborhoods into the same evidence object; it adopts fixed field numbers, fixed blocks, and fixed slots, and encodes continuous values as quantile levels within the candidate set, so that different candidate flows have a unified input structure, reducing input overhead caused by long floating-point numbers and repeated natural language field names; it requires the large language model to output the evidence field numbers it references, and the deterministic validator uses the original values before quantization to check the necessary conditions of the sample type, so that the large language model's judgment is transformed from only natural language reasons into traceable and verifiable network technology evidence; it does not allow the category labels output by the large language model to be written into the training set; the large language model only controls the labeling priority and flow path, and the final category is provided by human experts or credible real labels, thereby isolating the large language model's illusion from contaminating the training labels; it distinguishes version failure, numerical conflict, and missing necessary fields into different processing paths, and can perform evidence reconstruction, expert review, or caching for different anomalies, rather than simply accepting or discarding all conflicting samples. Attached Figure Description
[0068] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;
[0069] Figure 2 A schematic diagram of the structure of a slot-labeled value evidence object bound to a state;
[0070] Figure 3 A diagram illustrating deterministic verification and conflict routing based on field references, original values, and state versions;
[0071] Figure 4 A diagram illustrating the selection of high-value candidates within budget and the updating of manual annotations. Detailed Implementation
[0072] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. These embodiments are used to illustrate possible implementations of the present invention and are not intended to limit the scope of protection of the present invention.
[0073] Figures 1 to 4 Together, they demonstrate the complete closed loop of the network traffic labeling sample selection method based on state-bound evidence objects and numerical cross-validation: Figure 1 The overall process of the corresponding method is to calculate the normalized embedding, prediction entropy, category prototype state and neighborhood state of candidate traffic from the labeled traffic set and the unlabeled traffic pool. After state binding evidence object generation, large language model judgment, deterministic verification and conflict routing, and within-budget selection, the classifier, category prototype and version are finally updated by manual labeling of real labels. Figure 2The corresponding evidence object structure consists of a version tuple composed of classifier version, category prototype version, and unlabeled pool snapshot version, a candidate traffic identifier, and four fixed field slot blocks for traffic behavior, classifier status, prototype status, and neighborhood status. Continuous values are quantized into fixed quantile levels according to the empirical distribution of the candidate set, and the original values before quantization are retained for subsequent verification. Figure 3 For corresponding field reference verification and conflict routing, the deterministic verifier first checks the consistency between the evidence object version and the current system state, then determines the necessary field set and necessary numerical conditions based on the sample type, calculates the verification result using the original numerical values before quantization and the evidence field number set returned by the large language model, routes invalid version samples to the evidence regeneration path, routes valid version samples but numerical or reference conflict samples to the expert review path, and adds high-value samples that pass verification to the high-value candidate pool. Figure 4 For the selection of samples within the budget and model updates, when the high-value candidate pool exceeds the manual annotation budget, seed samples are first selected according to comprehensive value, and then the required number is supplemented according to the maximum and minimum embedding distance. After the manual annotation is completed, the labeled set, classifier and category prototype are updated, and the classifier version, category prototype version and unlabeled pool snapshot version are incremented, so that the old version evidence objects are automatically invalidated, thereby completing the continuous update closed loop of the dynamic network traffic classification model.
[0074] Example 1: This example provides a traffic labeling selection method based on state-bound evidence and numerical cross-validation.
[0075] I. Calculation of Candidate Flow Status:
[0076] 1. Traffic Input and Candidate Set. Network probes, gateways, or traffic analysis servers aggregate network packets according to bidirectional 5-tuples. For each traffic record, the number of packets, duration, total number of bytes, uplink / downlink byte ratio, average packet length, standard deviation of packet length, number of uplink bursts, and number of downlink bursts are extracted. The first few packets with directional packet lengths and arrival time intervals can also be input into a one-dimensional convolutional encoder. The system uses a labeled network traffic set. Train the current traffic classifier and retain the unlabeled network traffic pool. .
[0077] To limit the number of calls to the large language model, MiniBatch K-means clustering is performed on the traffic embeddings of the currently unlabeled network traffic pool. Let the manual annotation budget be B, and the number of clusters be K = min(2B, | Each cluster retains the sample closest to the cluster center and the sample with the highest predicted entropy, thus the number of candidates before deduplication does not exceed 4B. This pre-screening is only used to reduce computational overhead and does not change the subsequent core evidence verification process.
[0078] 2. Normalized embedding and uncertainty. For candidate traffic... Using the flow encoder of the t-th round Obtain normalized embeddings:
[0079]
[0080] The current classifier outputs the predicted probabilities of C known classes. The normalized prediction entropy is calculated according to the following formula:
[0081]
[0082] In this embodiment, ε= ,thus It is between 0 and 1. The larger the value, the less certain the current classifier is about the candidate traffic.
[0083] 3. Category Prototype, Novelty, and Prototype Spacing. For a known category c, the number of samples belonging to category c in the currently labeled network traffic set is: The category prototype is:
[0084]
[0085] in, Represents a known category Category prototype, Indicates a known category number. This indicates that the current network traffic concentration is of the true category. The number of samples, This indicates the index of the labeled network traffic samples. Indicates the first Human-generated real categories of labeled network traffic samples. Indicates the first The normalized embeddings of each labeled network traffic sample are obtained by the traffic encoder corresponding to the current classifier version; the above summation only applies to samples that satisfy... The labeled network traffic samples are executed, and each normalized embedding used to calculate the same class prototype is generated by the same traffic encoder corresponding to the current classifier version, so that each normalized embedding is in the same embedding space.
[0086] The cosine distance from the candidate traffic to the category prototype is:
[0087]
[0088] Sort the prototype distances of each category in ascending order to obtain the minimum distance. Second smallest distance And calculate:
[0089]
[0090] A large value indicates that the candidate traffic deviates from all known categories. The smaller value indicates that the candidate flow is close to both class prototypes.
[0091] 4. Local Representativeness. Find the k nearest neighbors of the candidate traffic in the current unlabeled pool; in this embodiment, k=10. Use the median distance of the kth nearest neighbor of each unlabeled sample as... The local representativeness is calculated according to the following formula:
[0092]
[0093] high and high This collectively indicates that multiple unlabeled samples form local structures at locations far from known categories; high But low This is more likely to correspond to isolated noise.
[0094] II. Slotted Evidence Objects Bound by State:
[0095] 1. Status Version. The current system status version tuple is:
[0096]
[0097] Classifier version Increments as classifier or encoder parameters are updated; category prototype version. Increment after recalculating all category prototypes using the current encoder; unlabeled pool snapshot version Increment when unlabeled pools are added or deleted, neighborhood indexes are rebuilt, or clustering results are updated. The classifier probability field must come from... The prototype distance field must come from The neighborhood field must come from .
[0098] 2. Quantile Rank Coding. For each continuous field in the current candidate set, first establish an empirical distribution using the following formula:
[0099]
[0100] The original numerical values are then encoded into L fixed levels:
[0101]
[0102] In this embodiment, L=10, and Q0 to Q9 represent the quantile levels from low to high. The original values before quantization are stored in the structured record of the deterministic validator and are not directly given to the large language model as the sole basis for judgment.
[0103] 3. Field Numbering and Fixed Slots. Evidence object settings are shown in Table 1 below;
[0104] Table 1: Evidence Object Setting Table
[0105]
[0106] The form of the evidence is:
[0107]
[0108] When no valid value is generated for a fixed slot, a uniform null code NA is written, and the field position is not deleted, thus ensuring that different candidate traffic has the same block order and field position. A specific evidence object can be represented as:
[0109] STATE|FV=017|PV=021|UV=008;
[0110] FLOW|ID=3A7F;
[0111] B|B01=Q2|B02=Q6|B03=Q8|B04=Q7|B05=Q8|B06=Q5|B07=Q2|B08=Q7;
[0112] M|M01=Q9|M02=Q2|M03=Q1;
[0113] P|P01=C04|P02=Q8|P03=Q0;
[0114] L|L01=Q7|L02=Q3|L03=K12;
[0115] Wherein, P01=C04 only indicates that the candidate traffic is closest to the known category prototype numbered C04, and does not represent the artificial real category of the candidate traffic.
[0116] III. Field Reference Validation and Conflict Routing:
[0117] 1. Large Language Model Output. The large language model is set up as a network traffic labeling value evaluator, and its output is:
[0118]
[0119] Choose high, medium, or low; Take the boundary, new pattern, representativeness, or noise; Manual annotation, expert review, caching, or discarding are all acceptable options. This is a set of evidence field numbers. The predicted_class, final_label, or attack_type fields are not set in the output structure.
[0120] 2. Evidence Version Check. First, check whether the evidence object belongs to the current system state using the following formula:
[0121]
[0122] like If the value is 0, the sample type in the old evidence will no longer be checked, and the evidence object will be regenerated directly based on the current classifier, the current category prototype, and the current unlabeled pool.
[0123] 3. Dynamic threshold and necessary numerical conditions. Set the following for the current candidate set:
[0124]
[0125] The numerical support indicators for boundary, new mode, representativeness, and noise are as follows:
[0126]
[0127]
[0128]
[0129]
[0130] The necessary field set for boundary samples is {M01, P03}; the necessary field sets for new patterns, representative samples, and noisy samples all include at least {P02, L01}. The system may also add protocol fields or burst fields in subordinate implementations, but must not remove the necessary numerical fields that directly correspond to the sample type.
[0131] 4. Deterministic verification. Based on... from , , and Selected from And verify according to the following formula:
[0132]
[0133] This formula requires that the evidence version be valid, the original numerical values meet the necessary conditions for the corresponding sample type, and that the large language model actually uses the necessary fields. The validator uses the data before quantization. , , and It not only checks the text format output by the large language model.
[0134] 5. Conflict routing. Specific execution rules are shown in Table 2 below:
[0135] Table 2: Execution Rules Table
[0136]
[0137] The validated positive high-value candidate pool is represented as follows:
[0138] .
[0139] IV. Budgetary Options and Manual Updates:
[0140] 1. Overall Value. The quantitative levels of M01, P03, P02, and L01 in the evidence are respectively denoted as follows: , , and Since a smaller prototype interval represents closer to the category boundary, the composite value is calculated using the following formula:
[0141]
[0142] 2. Diversity Selection. If the number of high-value candidates does not exceed the budget B, all of them will be added to the set awaiting manual annotation. Otherwise, select... The largest candidate traffic is used as the seed, and then the process is repeated:
[0143]
[0144] Until the selected set It contains B candidate traffic streams. This step ensures that the final set is not dominated by a large number of similar traffic streams.
[0145] 3. Manual annotation and training set updates. Human experts confirm the categories based on network traffic analysis rules, device logs, attack replay records, or trusted datasets with verified labels. Update the labeled network traffic set according to the following formula:
[0146]
[0147] Large language model output , and Not used as a category label .use After retraining the classifier, all labeled traffic embeddings and category prototypes are recalculated using the new encoder, and classifier versions, prototype versions, and unlabeled pool snapshot versions are added respectively.
[0148] V. Exemplary Execution Process:
[0149] The following values are for illustrative purposes only and do not represent actual experimental results. Let the current value of a candidate flow rate be... =0.86、 =0.03、 =0.74、 =0.79; the threshold for the current candidate set is =0.78、 =0.06、 =0.70、 =0.65. This candidate also satisfies the necessary conditions for the new mode. ≥ and ≥ .
[0150] The evidence object carries the state STATE|FV=017|PV=021|UV=008. The large language model outputs the sample type (novel), value (high), and action (manual_labeling), and references P02 and L01. If the current system state is still (017, 021, 008), then... =1; the required fields {P02,L01} are contained in And the new mode supports numerical values =1, therefore =1, the candidate enters the high-value candidate pool.
[0151] If the same candidate If the actual value is 0.31 but the large language model still outputs "novel", then the new mode's numerical support is... =0, =0, the system transfers it to expert review instead of directly labeling or discarding it. If the classifier has already been updated to FV=018, then =0, the system regenerates the evidence object and no longer uses the old conclusion.
[0152] Example 2: This example provides a network traffic annotation sample selection system based on state-bound evidence objects and numerical cross-validation, including a candidate state calculation module, an evidence object generation module, a large language model judgment module, a deterministic verification and routing module, and a budget selection and update module. The candidate state calculation module can be deployed on a network traffic analysis server or an edge gateway; the large language model can be deployed locally or called through a controlled interface. For fields containing sensitive network identifiers, hashing or partial anonymization can be performed before generating annotation value evidence objects.
[0153] The evidence object generation module stores the field dictionary and fixed slot templates; the deterministic verification and routing module stores the original values before quantization, the current version tuples, the necessary field sets for different sample types, and threshold calculation rules. The large language model only accesses the fixed-slot labeled value evidence objects and does not have permission to directly write to the training label storage area. Only after the manual labeling interface confirms the true labels can the budget selection and update module allow modification of the labeled network traffic set.
[0154] Example 3: This example provides an optional implementation method;
[0155] The flow encoder can employ a one-dimensional convolutional network, a multilayer perceptron, a Transformer encoder, or other models capable of generating fixed-dimensional embeddings; the category prototype can employ category means, robust truncated means, or multiple sub-prototypes. The large language model can be a local general model, a domain-enhanced model, or other generative model capable of outputting results according to a constrained structure.
[0156] The fixed field block can add fields such as TLS handshake statistics, QUIC connection identifier changes, TCP flag ratios, or cross-stream device behavior fields according to the application scenario. However, each newly added field should have a fixed field number, a clearly defined source version, and a corresponding original value record. The necessary field set and numerical conditions can be determined through the validation set, but cannot be modified solely by the large language model.
[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the principles and spirit of the present invention are included within the protection scope of the present invention.
Claims
1. A method for selecting traffic annotations based on state-bound evidence and numerical cross-validation, characterized in that, Includes the following steps: Obtain a set of labeled network traffic and a pool of unlabeled network traffic. Determine candidate network traffic from the pool of unlabeled network traffic. Use the current traffic classifier to obtain the normalized embedding and classification uncertainty of the candidate network traffic. Based on the normalized embedding, obtain the prototype state of the candidate network traffic relative to the current category prototype set and its neighborhood state in the current pool of unlabeled network traffic. Encode the continuous numerical fields in the current candidate network traffic set into quantile levels according to the empirical distribution, generate a labeled value evidence object according to the fixed field number and fixed field slot, and write the current classifier version, the current category prototype version and the current unlabeled network traffic pool snapshot version into the evidence object; Input the labeled value evidence object into the large language model to obtain a set of labeled value, sample type, suggested action and evidence field number without network traffic real category labels; Based on the sample type, determine the necessary field set and necessary numerical conditions, verify the version consistency, necessary field references and original numerical conditions before quantification of the labeled value evidence object, and obtain deterministic verification results; Based on the verification results, candidate network traffic is routed to the high-value candidate pool, expert review, evidence regeneration, caching, or discarding path; within the manual annotation budget, a set of samples to be manually annotated is formed, and after obtaining the real manual labels, the traffic classifier and category prototype are updated and the version is incremented, so that the old labeled value evidence objects become invalid.
2. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 1, characterized in that, The current traffic classifier includes a traffic encoder, and the normalized embedding representation of candidate network traffic is as follows: ; in, For the i-th candidate network traffic, The flow encoder used in round t, For candidate network traffic Normalized embedding; the current traffic classifier outputs class probabilities for C known classes. Candidate network traffic Normalized prediction entropy Represented as: ; Where c is a known category number, and ε is a positive number to prevent the logarithmic input from being zero.
3. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 2, characterized in that, Given a known category c, group the currently labeled network traffic belonging to category c. The normalized embeddings of each sample are averaged to obtain the class prototypes. ; in, Represents a known category The category prototype, Indicates a known category number. This indicates that the current network traffic concentration is of the true category. The number of samples, This indicates the index of the labeled network traffic samples. Indicates the first Human-generated real categories of labeled network traffic samples. Indicates the first The normalized embeddings of each labeled network traffic sample are obtained by the traffic encoder corresponding to the current classifier version; the above summation only applies to samples that satisfy... The labeled network traffic samples are executed, and each normalized embedding used to calculate the same class prototype is generated by the same traffic encoder corresponding to the current classifier version, so that each normalized embedding is in the same embedding space. Candidate network traffic To category prototype The cosine distance is: ; Sort the distances from candidate network traffic to each category prototype in ascending order, and denote the first and second smallest distances as follows: and Novelty and prototype interval They are respectively: ; Among them, novelty The prototype interval is used to indicate the degree to which candidate network traffic deviates from the prototypes of all known categories. Used to indicate the degree to which candidate network traffic lies among multiple known class prototypes.
4. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 3, characterized in that, Identify candidate network traffic from the currently unlabeled network traffic pool. The set of k nearest neighbors (i) The median of the k-th nearest neighbor distance of each sample in the current unlabeled network traffic pool is used as the scale. Candidate network traffic Local representativeness Represented as: ; in, For the normalized embedding of the j-th nearest neighbor sample; when Greater than or equal to the preset novelty threshold and When the value is greater than or equal to a preset local representativeness threshold, the corresponding numerical relationship is used as a necessary numerical condition for the new pattern sample. Greater than or equal to the preset novelty threshold and When the value is less than the preset local representativeness threshold, the corresponding numerical relationship is used as a necessary numerical condition for the noise sample.
5. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 4, characterized in that, The current system state version tuple is represented as: ; in, , and Let be the classifier version, category prototype version, and unlabeled network traffic pool snapshot version for the t-th round, respectively; for the j-th continuous field in the current candidate network traffic set, define the empirical distribution function: ; Where n is the number of current candidate network traffic, Let I(·) be the original value of the j-th consecutive field of the r-th candidate network traffic, and let I(·) be the indicator function; encode the j-th consecutive field of the i-th candidate network traffic into one of L levels: ; The objects representing the marked value evidence are represented according to fixed slots as follows: ; in, As candidate network traffic identifiers, , , and These are fixed-length field vectors representing the traffic behavior block, classifier state block, prototype state block, and neighborhood state block, respectively.
6. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 5, characterized in that, The output of the large language model is represented as: ; in, To indicate value, For sample type, For suggested actions, A set of evidence field numbers referenced by the large language model; the output structure of the large language model does not include a network traffic true category label field. Determine the criteria for valid evidence based on the version recorded in the object of the valued evidence: ; Among them, state( ) for the object of marked value evidence The version tuple read. This is the current system state version tuple.
7. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 6, characterized in that, Determine uncertainty thresholds for the current candidate network traffic set. Prototype Interval Threshold Novelty threshold and local representative threshold : ; in, Represents the empirical β quantile; the numerical support indicators for boundary samples, novel pattern samples, representative samples, and noisy samples are as follows: ; ; ; ; Based on the sample type output by the large language model ,from , , and Select the corresponding numerical support indicator. ; Sample type The corresponding set of necessary fields is denoted as The deterministic verification result is as follows: ; Among them, when When the value is equal to 1, it indicates that the versions are consistent, the original value meets the necessary numerical conditions of the sample type, and the large language model actually references the corresponding necessary fields.
8. The flow labeling selection method based on state-bound evidence and numerical cross-validation according to claim 7, characterized in that, When the evidence is valid When the value is 0, candidate network traffic is routed to the evidence regeneration path; when... For 1 but When the value is 0, candidate network traffic is routed to the expert review path; when...
1. When the labeled value is high and the sample type is a boundary sample, a new pattern sample, or a representative sample, the candidate network traffic is added to the high-value candidate pool: ; When the number of candidate network traffic in the high-value candidate pool exceeds the manual annotation budget B, the comprehensive value is determined based on the quantitative levels of uncertainty, prototype interval, novelty, and local representativeness in the evidence object: ; in, , , and These represent the quantization levels for the corresponding fields; first, select the candidate network traffic with the highest overall value, and then select the candidate network traffic with the maximum and minimum embedding distances to the selected sample set according to the following formula: ; in, As a high-value candidate pool, For the currently selected sample set, until... The number of samples reached the manual annotation budget B.
9. A network traffic labeling sample selection system based on state-bound evidence objects and numerical cross-validation, characterized in that, include: The candidate state calculation module is used to obtain the traffic behavior field, classifier state field, category prototype state field, and neighborhood state field of candidate network traffic. The evidence object generation module is used to perform quantile-level coding and form labeled value evidence objects carrying classifier version, category prototype version and unlabeled network traffic pool snapshot version according to fixed field numbers and fixed field slots; The large language model judgment module is used to output the annotation value, sample type, suggested action, and evidence field number set that do not contain the true category label; The deterministic verification and routing module is used to check the output of the large language model based on the current version, the necessary field set, and the original values before quantization, and to control the entry of candidate network traffic into the high-value candidate pool, expert review, evidence regeneration, caching, or discarding of paths; The budget selection and update module is used to generate a set of samples to be manually labeled, and to update the traffic classifier, category prototype, and version using real human labels.
10. An electronic device comprising a processor and a memory, characterized in that, The memory stores a computer program, which, when executed by the processor, implements the method described in any one of claims 1 to 8.
