A data classification and processing method and system for parallel computing on multi-core processors

Abstract

This invention discloses a data classification and processing method and system for parallel computing on multi-core processors, relating to the field of data classification and processing technology. This invention effectively addresses the challenge of abnormal disturbances in multi-core parallel data classification through a confidence-driven heterogeneous pipeline architecture; based on dynamic hierarchical processing of data block confidence, it ensures the complete processing chain for high-confidence data while implementing rapid classification and cache isolation for low-confidence data, preventing abnormal data from blocking the pipeline; an adaptive window segmentation mechanism links confidence fluctuations with computing resource allocation, mitigating inter-core load imbalance caused by sudden anomalies; and a dual closed-loop control strategy combines real-time queue status with historical load patterns, reducing the risk of cache pollution caused by thread migration.

Description

Technical Field

[0001] This invention relates to the field of data classification and processing technology, and in particular to a data classification and processing method and system for parallel computing on multi-core processors. Background Technology

[0002] Current multi-core processors, in real-time streaming data classification tasks, need to cope with the continuous inflow of data in high-throughput scenarios such as industrial IoT and financial transactions, while also dealing with abnormal data disturbances caused by factors such as sensor failures and network jitter. The mainstream solution adopts a three-stage pipeline architecture to achieve inter-stage parallelism by binding specific core groups with thread pools. However, as the frequency of data disturbances increases, such as sudden environmental noise in autonomous driving, traditional methods expose their adaptive defects under the impact of sudden abnormal data flow.

[0003] Newer solutions, such as the Dynamic Water Level Control (DWC) scheme, can adjust the processing rhythm according to the queue depth. However, when encountering consecutive abnormal data blocks, the global tick adjustment of DWC is prone to causing inter-stage resource mismatch. The preprocessing core idles while waiting for the downstream to release the buffer, and the classification decision core becomes idle due to insufficient input data. The Elastic Thread Migration (ETM) technology proposed in recent years alleviates this problem by dynamically rebinding threads to idle cores. However, the cache pollution caused by frequent migration increases the L2 miss rate, resulting in unacceptable quality fluctuations in low-tolerance scenarios such as real-time medical image classification.

[0004] The latest research direction is lightweight disturbance rejection mechanisms, such as embedding soft switches based on data confidence between pipeline stages. When the confidence of a data block is lower than a threshold, the subsequent feature calculation stage is automatically skipped, and a simplified classifier is directly called to output emergency results. However, the interference caused by abnormal data features sharing computing resources with normal data remains. Some solutions introduce the concept of computing path isolation, which preserves the complete pipeline for high-confidence data, while low-confidence data is transferred to a dedicated lightweight channel for processing. This design works well in video stream analysis, but the control of state synchronization overhead during channel switching is still not ideal and it is difficult to meet the needs of microsecond-level decision-making. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] This invention provides a data classification and processing method and system for parallel computing on multi-core processors to solve the problems of existing solutions relying on fixed pipelines and global load balancing, which are prone to inter-stage resource mismatch and cache efficiency degradation when facing sudden abnormal data, and lack of dynamic adjustment mechanism and deep coupling with data characteristics.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a data classification and processing method for parallel computing on a multi-core processor, comprising,

[0009] Step S1: Divide the input data stream into consecutive data blocks and add a timestamp and source identifier to each data block;

[0010] Step S2: Calculate the confidence index of each data block in real time. The confidence index is generated based on a composite function of the feature distribution dispersion of the current data block and the historical classification accuracy.

[0011] Step S3, dynamically allocate computing resources based on the confidence index, including:

[0012] For data blocks with a confidence level higher than a first threshold, they are assigned to a complete classification pipeline consisting of a main computing core group. The complete classification pipeline includes a series of processing steps for feature extraction, dimensionality reduction, and classification decision.

[0013] For data blocks with confidence levels below the first threshold but above the second threshold, they are assigned to a lightweight computing core group to perform a fast classification process, which skips the dimensionality reduction stage and uses a simplified classification model.

[0014] For data blocks with a confidence level below the second threshold, the exception handling channel is triggered to perform cache isolation and feature marking;

[0015] The cache hit rate and queue depth of each core group are collected periodically, and the core ratio between the main computing core group and the lightweight computing core group is dynamically adjusted.

[0016] As a preferred embodiment of the data classification and processing method for parallel computing on multi-core processors described in this invention, the data block segmentation in step S1 employs an adaptive window mechanism.

[0017] When the confidence standard deviation of N consecutive data blocks exceeds the preset threshold, the segmentation window size is automatically reduced.

[0018] When the queue depth is below the critical value and the duration reaches T seconds, the segmentation window size is gradually increased to the initial value.

[0019] As a preferred embodiment of the data classification processing method for parallel computing on multi-core processors described in this invention, in step S1, during the process of automatically reducing the segmentation window size, the confidence sequence of the most recent N data blocks is continuously tracked. When the fluctuations are severe, the process involves adaptively reducing the window size, including:

[0020] Calculate the standard deviation of the confidence sequence to quantify the degree of volatility. If the standard deviation exceeds a preset threshold, dynamically adjust the segmentation window size based on the volatility amplitude. The formula is as follows:

[0021]

[0022] Where N represents the number of consecutive data blocks used for statistics, and c i This represents the confidence index for the i-th data block.

[0023] σ represents the arithmetic mean of the most recent N confidence levels, t represents the time sequence number of the current data block, and σ represents the time sequence number of the current data block. C This represents the calculated confidence level standard deviation;

[0024] When σ C >T σ At that time, update the split window size for the next time step using the following formula:

[0025] α=exp(-γ(σ C -T σ )),

[0026]

[0027] Among them, T σ The default threshold for the confidence level standard deviation is represented by γ, the adaptive scaling rate coefficient is represented by α, and W is represented by the scaling factor. t W represents the current segmentation window size. t+1 W represents the updated split window size. min Indicates the minimum allowed size of the split window. This indicates the floor function.

[0028] As a preferred embodiment of the data classification and processing method for parallel computing on multi-core processors described in this invention, the calculation of the confidence index in step S2 includes:

[0029] Extract the feature distribution histogram from the head region of the data block and calculate its KL divergence with the historical normal data distribution;

[0030] Obtain the K most recent classification results for identifiers from the same source, and calculate the moving weighted average of their accuracy.

[0031] The KL divergence and accuracy are weighted and integrated, with the weight coefficients dynamically increasing with the freshness of the data block timestamps.

[0032] As a preferred embodiment of the data classification processing method for parallel computing on multi-core processors described in this invention, in step S2, a composite confidence score is calculated based on the feature distribution dispersion and historical classification accuracy, and a confidence index c is generated in real time for each data block. t The calculation and fusion are performed in three stages:

[0033] Calculate the feature distribution dispersion: Extract the feature distribution histogram from the head region of the t-th data block. and histogram of historical normal data distribution Calculate the KL divergence:

[0034]

[0035] Where M represents the total number of histogram buckets, H t (m) represents the frequency of the m-th feature bucket in the current data block, H hist (m) represents the frequency of the m-th feature bucket in historical normal data, D t H represents the dispersion of the characteristic distribution of the current data block. hist The `hist` in the middle represents the historical normal data distribution;

[0036] Calculate historical classification accuracy: Obtain the sequence of the K most recent classification results for the same source identifier {r}. t-K+1 ,…,r t}, where r j ∈{0,1}, and a moving weighted average is calculated using exponentially decaying weights ρ∈(0,1), as follows:

[0037]

[0038] Where K represents the number of historical classification results, ρ represents the moving weighted decay factor, and r t-K+j Let A represent the classification result of the (t-K+j)th classification. t Represents the moving weighted average of historical classification accuracy;

[0039] Dynamic fusion to generate confidence score: defining the freshness coefficient φ t With timestamp τ t Incrementing, represented as:

[0040] φ t =1-exp[-λ(τ) t -τ ref )],

[0041] Where, τ t τ represents the timestamp of the current data block. ref Indicates the reference start time, λ represents the freshness growth rate, and φ t This represents the freshness weighting coefficient;

[0042] The dispersion and accuracy are combined by weight, and the dispersion is normalized to obtain the maximum possible dispersion D. max The formula is:

[0043]

[0044] Among them, D max c represents the maximum value of the divergence normalization. t This indicates the generated confidence level index.

[0045] As a preferred embodiment of the data classification and processing method for parallel computing on multi-core processors described in this invention, the exception handling channel in step S3 includes:

[0046] An isolated storage area is allocated in the last-level cache to store low-confidence data blocks and their feature tags;

[0047] Start a separate monitoring thread to periodically scan the isolated storage area, and trigger a recalculation when one of the following conditions is met:

[0048] The confidence level of newly arriving data blocks with the same source identifier has recovered to above the first threshold.

[0049] The overall system load rate is lower than the set threshold and the duration exceeds the preset period.

[0050] As a preferred embodiment of the data classification and processing method for parallel computing of multi-core processors described in this invention, wherein the core ratio adjustment in step S3 adopts a dual closed-loop control strategy:

[0051] The inner loop adjusts the thread migration frequency between core groups based on the real-time queue depth, while the outer loop predicts the core group size requirements for future time windows based on historical load patterns.

[0052] When the average processing latency of the lightweight computing core group exceeds β times the latency of the main computing core group, a specified number of core resources are forcibly reclaimed and allocated to the main computing core group.

[0053] As a preferred embodiment of the data classification and processing method for parallel computing of multi-core processors described in this invention, wherein: in step S3, when the average processing latency L of the lightweight computing core group... t Exceeding the main computing core latency M t When the latency ratio is β times, the following resource reclamation and allocation mechanism is triggered: by calculating the latency ratio and comparing it with the threshold, the number of cores to be reclaimed is dynamically determined, and then, while ensuring the availability of the lightweight group, the resources are preferentially allocated to the main computing core group to restore overall performance.

[0054] If R t >1, then:

[0055]

[0056] N L,t+1 =N L,t -ΔN,N M,t+1 =N M,t +ΔN,

[0057] Among them, L t M represents the average processing latency of the lightweight computing kernel at time t. t R represents the average processing latency of the main computing kernel group at time t, β represents the latency multiplier threshold, and R t N represents the delay ratio. L,t N represents the current number of cores in the lightweight computing core group. M,t N represents the current number of cores in the main compute core group. L,min κ represents the minimum number of cores allowed in the lightweight group, L represents the lightweight core group, M represents the host core group, and ΔN represents the number of core resources recycled in this instance. The expression indicates rounding up, and min(·) indicates taking the minimum of the two values.

[0058] Secondly, the present invention provides a data classification and processing system for parallel computing of multi-core processors, including a data ingestion module configured to divide the input data stream into continuous data blocks and add a timestamp and source identifier to each data block;

[0059] The confidence prediction module is connected to the data ingestion module and configured to generate a confidence index for data blocks;

[0060] The elastic computing module includes a main computing core group, a lightweight computing core group, and an exception handling channel, and its input is connected to the confidence prediction module.

[0061] The dynamic scheduling module is configured to collect the operating indicators of each core group and adjust the core allocation ratio, and its control terminal is connected to the elastic computing module.

[0062] The main computing core group includes a cascaded feature extraction unit, a dimensionality reduction unit, and a classification decision unit, while the lightweight computing core group bypasses the dimensionality reduction unit and directly connects to the classification decision unit.

[0063] As a preferred embodiment of the data classification and processing system for parallel computing on multi-core processors described in this invention, the exception handling channel includes:

[0064] The cache isolation controller is configured to dynamically divide isolated storage regions in the last-level cache. The cache isolation controller divides the isolated regions by modifying the bitmask of the CPU cache allocation table CAT. The bitmask value increases in increments of 256KB based on the current number of isolated data blocks.

[0065] A recalculation trigger is connected to the cache isolation controller and the dynamic scheduling module and is configured to activate the data reprocessing process when preset conditions are met.

[0066] The beneficial effects of this invention are as follows: This invention effectively addresses the challenge of abnormal disturbances in multi-core parallel data classification through a confidence-driven heterogeneous pipeline architecture; based on dynamic hierarchical processing of data block confidence, it ensures the complete processing chain for high-confidence data while implementing rapid classification and cache isolation for low-confidence data, preventing abnormal data from blocking the pipeline; an adaptive window segmentation mechanism links confidence fluctuations with computing resource allocation, mitigating inter-core load imbalance caused by sudden anomalies; a dual closed-loop control strategy combines real-time queue status with historical load patterns, reducing the risk of cache pollution caused by thread migration; and the anomaly handling channel, through hardware-level cache isolation and condition-triggered recalculation, maintains the throughput of the mainstream pipeline while ensuring delayed repair of abnormal data. Attached Figure Description

[0067] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart illustrating the data classification and processing method for parallel computing on a multi-core processor in Example 1.

[0069] Figure 2 This is a schematic diagram of the data classification and processing system for parallel computing on a multi-core processor in Example 1. Detailed Implementation

[0070] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0071] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0073] Example 1, referring to Figure 1 and Figure 2This embodiment provides a data classification and processing method for parallel computing on multi-core processors, including the following steps:

[0074] Step S1: Divide the input data stream into consecutive data blocks and add a timestamp and source identifier to each data block;

[0075] The data block segmentation in step S1 uses an adaptive window mechanism:

[0076] When the confidence standard deviation of N consecutive data blocks exceeds the preset threshold, the segmentation window size is automatically reduced.

[0077] When the queue depth is below the critical value and the duration reaches T seconds, the segmentation window size is gradually increased to the initial value.

[0078] In step S1, during the process of automatically reducing the segmentation window size, the confidence sequence of the most recent N data blocks is continuously tracked. When the fluctuations are severe, the process involves adaptively reducing the window size, including:

[0079] Calculate the standard deviation of the confidence sequence to quantify the degree of volatility. If the standard deviation exceeds a preset threshold, dynamically adjust the segmentation window size based on the volatility amplitude. The formula is as follows:

[0080]

[0081] Where N represents the number of consecutive data blocks used for statistics, and c i This represents the confidence index for the i-th data block.

[0082] σ represents the arithmetic mean of the most recent N confidence levels, t represents the time sequence number of the current data block, and σ represents the time sequence number of the current data block. C This represents the calculated confidence level standard deviation;

[0083] When σ C >T σ At that time, update the split window size for the next time step using the following formula:

[0084] α=exp(-γ(σ C -T σ )),

[0085]

[0086] Among them, T σ The default threshold for the confidence level standard deviation is represented by γ, the adaptive scaling rate coefficient is represented by α, and W is represented by the scaling factor. t W represents the current segmentation window size. t+1 W represents the updated split window size. minIndicates the minimum allowed size of the split window. This indicates the floor function;

[0087] Specifically, in high-volatility scenarios, rapid response to changes in confidence levels is crucial. Narrowing the window improves sensitivity to sudden patterns and avoids information ambiguity caused by large windows. The dynamic factor α scales in a stepwise manner based on the volatility amplitude, ensuring the contraction is neither too aggressive nor too slow. The lower limit W... min The settings ensure that the window does not shrink indefinitely, thus maintaining system stability;

[0088] Step S2: Calculate the confidence index of each data block in real time. The confidence index is generated based on a composite function of the feature distribution dispersion of the current data block and the historical classification accuracy.

[0089] The calculation of the confidence index in step S2 includes:

[0090] Extract the feature distribution histogram from the head region of the data block and calculate its KL divergence with the historical normal data distribution;

[0091] Obtain the K most recent classification results for identifiers from the same source, and calculate the moving weighted average of their accuracy.

[0092] The KL divergence and accuracy are weighted and integrated, with the weight coefficients dynamically increasing with the freshness of the data block timestamps;

[0093] In step S2, a composite confidence score is calculated based on the feature distribution dispersion and historical classification accuracy, and a confidence index c is generated in real time for each data block. t The calculation and fusion are performed in three stages:

[0094] Calculate the feature distribution dispersion: Extract the feature distribution histogram from the head region of the t-th data block. and histogram of historical normal data distribution Calculate the KL divergence:

[0095]

[0096] Where M represents the total number of histogram buckets, H t (m) represents the frequency of the m-th feature bucket in the current data block, H hist (m) represents the frequency of the m-th feature bucket in historical normal data, D t H represents the dispersion of the characteristic distribution of the current data block. hist The `hist` in the middle represents the historical normal data distribution;

[0097] Calculate historical classification accuracy: Obtain the sequence of the K most recent classification results for the same source identifier {r}. t-K+1 ,…,r t}, where r j ∈{0,1}, and a moving weighted average is calculated using exponentially decaying weights ρ∈(0,1), as follows:

[0098]

[0099] Where K represents the number of historical classification results, ρ represents the moving weighted decay factor, and r t-K+j Let A represent the classification result of the (t-K+j)th classification. t Represents the moving weighted average of historical classification accuracy;

[0100] Dynamic fusion to generate confidence score: defining the freshness coefficient φ t With timestamp τ t Incrementing, represented as:

[0101] φ t =1-exp[-λ(τ) t -τ ref )],

[0102] Where, τ t τ represents the timestamp of the current data block. ref Indicates the reference start time, λ represents the freshness growth rate, and φ t This represents the freshness weighting coefficient;

[0103] The dispersion and accuracy are combined by weight, and the dispersion is normalized to obtain the maximum possible dispersion D. max The formula is:

[0104]

[0105] Among them, D max c represents the maximum value of the divergence normalization. t This indicates the generated confidence level index;

[0106] Specifically, this composite function takes into account both current features and historical performance, organically integrates distribution dispersion and classification accuracy, and dynamically adjusts the weight allocation through the freshness coefficient. This allows the confidence index to reflect sudden shifts in a timely manner while maintaining model stability. The distribution dispersion term detects the deviation of data block features from the normal distribution, while the accuracy term reflects the recent performance of the model. The two complement each other, and the freshness mechanism ensures greater sensitivity to the latest data and prevents judgment lag caused by outdated historical information. This method improves the robustness and response speed of confidence assessment and is suitable for real-time data classification in a multi-core parallel environment.

[0107] Step S3, dynamically allocate computing resources based on the confidence index, including:

[0108] For data blocks with a confidence level higher than the first threshold, they are assigned to a complete classification pipeline consisting of a main computing core group. The complete classification pipeline includes a series of processing steps for feature extraction, dimensionality reduction, and classification decision.

[0109] For data blocks with confidence levels below the first threshold but above the second threshold, they are assigned to lightweight computing cores to perform a fast classification process. The fast classification process skips the dimensionality reduction stage and uses a simplified classification model.

[0110] For data blocks with a confidence level below the second threshold, the exception handling channel is triggered to perform cache isolation and feature marking;

[0111] Periodically collect cache hit rate and queue depth metrics for each core group, and dynamically adjust the core ratio between the main computing core group and the lightweight computing core group;

[0112] The exception handling channel in step S3 includes:

[0113] An isolated storage area is allocated in the last-level cache to store low-confidence data blocks and their feature tags;

[0114] Start a separate monitoring thread to periodically scan the isolated storage area, and trigger a recalculation when one of the following conditions is met:

[0115] The confidence level of newly arriving data blocks with the same source identifier has recovered to above the first threshold.

[0116] The overall system load rate is lower than the set threshold and the duration exceeds the preset period;

[0117] The core ratio adjustment in step S3 adopts a dual closed-loop control strategy:

[0118] The inner loop adjusts the thread migration frequency between core groups based on the real-time queue depth, while the outer loop predicts the core group size requirements for future time windows based on historical load patterns.

[0119] When the average processing latency of the lightweight computing core group exceeds β times the latency of the main computing core group, a specified number of core resources are forcibly reclaimed and allocated to the main computing core group.

[0120] In step S3, when the average processing latency L of the lightweight computing core group t Exceeding the main computing core latency M t When the latency ratio is β times, the following resource reclamation and allocation mechanism is triggered: by calculating the latency ratio and comparing it with the threshold, the number of cores to be reclaimed is dynamically determined, and then, while ensuring the availability of the lightweight group, the resources are preferentially allocated to the main computing core group to restore overall performance.

[0121] If R t >1, then:

[0122]

[0123] N L,t+1 =N L,t -ΔN,N M,t+1 =N M,t +ΔN,

[0124] Among them, L t M represents the average processing latency of the lightweight computing kernel at time t. t R represents the average processing latency of the main computing kernel group at time t, β represents the latency multiplier threshold, and R t N represents the delay ratio. L,t N represents the current number of cores in the lightweight computing core group. M,t N represents the current number of cores in the main compute core group. L,min κ represents the minimum number of cores allowed in the lightweight group, L represents the lightweight core group, M represents the host core group, and ΔN represents the number of core resources recycled in this instance. This indicates the rounding operation, and min(·) indicates taking the minimum value between the two.

[0125] Specifically, this mechanism can quickly identify and proportionally reclaim kernel resources when the latency of the lightweight processing group abnormally increases, prioritizing the allocation of resources to the main computing kernel group to restore the performance of critical classification tasks. The reclamation coefficient κ can be tuned offline according to the system tolerance, ensuring that the kernel resource reclamation is neither too drastic nor too conservative, with a lower limit N. L,min Ensure that the lightweight group always has a sufficient number of cores to avoid complete incompetence.

[0126] This embodiment also provides a data classification and processing system for parallel computing on multi-core processors, including:

[0127] The data ingestion module is configured to divide the input data stream into continuous data blocks and add a timestamp and source identifier to each data block;

[0128] The confidence prediction module is connected to the data ingestion module and configured to generate confidence metrics for data blocks.

[0129] The elastic computing module includes a main computing core group, a lightweight computing core group, and an anomaly handling channel. Its input end is connected to the confidence prediction module.

[0130] The dynamic scheduling module is configured to collect the operating indicators of each core group and adjust the core allocation ratio. Its control terminal is connected to the elastic computing module.

[0131] The main computing core group includes cascaded feature extraction units, dimensionality reduction units, and classification decision units, while the lightweight computing core group's bypass dimensionality reduction units are directly connected to the classification decision units.

[0132] The exception handling channel includes:

[0133] The cache isolation controller is configured to dynamically divide isolated storage regions in the last level cache. The cache isolation controller divides the isolated regions by modifying the bitmask of the CPU cache allocation table (CAT). The bitmask value increases in increments of 256KB based on the current number of isolated data blocks.

[0134] The recalculation trigger is connected to the cache isolation controller and the dynamic scheduling module and is configured to activate the data reprocessing process when preset conditions are met.

[0135] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data classification and processing method for parallel computing on multi-core processors, characterized in that, include, Step S1: Divide the input data stream into consecutive data blocks and add a timestamp and source identifier to each data block; Step S2: Calculate the confidence index of each data block in real time. The confidence index is generated based on a composite function of the feature distribution dispersion of the current data block and the historical classification accuracy. Step S3, dynamically allocate computing resources based on the confidence index, including: For data blocks with a confidence level higher than a first threshold, they are assigned to a complete classification pipeline consisting of a main computing core group. The complete classification pipeline includes a series of processing steps for feature extraction, dimensionality reduction, and classification decision. For data blocks with confidence levels below the first threshold but above the second threshold, they are assigned to a lightweight computing core group to perform a fast classification process, which skips the dimensionality reduction stage and uses a simplified classification model. For data blocks with a confidence level below the second threshold, the exception handling channel is triggered to perform cache isolation and feature marking; The cache hit rate and queue depth of each core group are collected periodically, and the core ratio between the main computing core group and the lightweight computing core group is dynamically adjusted.

2. The data classification and processing method for parallel computing on a multi-core processor as described in claim 1, characterized in that, The data block segmentation in step S1 employs an adaptive window mechanism: When the confidence standard deviation of N consecutive data blocks exceeds the preset threshold, the segmentation window size is automatically reduced. When the queue depth is below the critical value and the duration reaches T seconds, the segmentation window size is gradually increased to the initial value.

3. The data classification and processing method for parallel computing on a multi-core processor as described in claim 2, characterized in that, In step S1, during the process of automatically reducing the segmentation window size, the confidence sequence of the most recent N data blocks is continuously tracked. When the fluctuations are severe, the process involves adaptively reducing the window size, including: Calculate the standard deviation of the confidence sequence to quantify the degree of volatility. If the standard deviation exceeds a preset threshold, dynamically adjust the segmentation window size based on the volatility amplitude. The formula is as follows: Where N represents the number of consecutive data blocks used for statistics, and c i This represents the confidence index for the i-th data block. σ represents the arithmetic mean of the most recent N confidence levels, t represents the time sequence number of the current data block, and σ represents the time sequence number of the current data block. C This represents the calculated confidence level standard deviation; When σ C >T σ At that time, update the split window size for the next time step using the following formula: α=exp(-γ(σ C -T σ )), Among them, T σ The default threshold for the confidence level standard deviation is represented by γ, the adaptive scaling rate coefficient is represented by α, and W is represented by the scaling factor. t W represents the current segmentation window size. t+1 W represents the updated split window size. min Indicates the minimum allowed size of the split window. This indicates the floor function.

4. The data classification and processing method for parallel computing on a multi-core processor as described in claim 1, characterized in that, The calculation of the confidence index in step S2 includes: Extract the feature distribution histogram from the head region of the data block and calculate its KL divergence with the historical normal data distribution; Obtain the K most recent classification results for identifiers from the same source, and calculate the moving weighted average of their accuracy. The KL divergence and accuracy are weighted and integrated, with the weight coefficients dynamically increasing with the freshness of the data block timestamps.

5. The data classification and processing method for parallel computing on a multi-core processor as described in claim 4, characterized in that, In step S2, a composite confidence score is calculated based on the feature distribution dispersion and historical classification accuracy, and a confidence index c is generated in real time for each data block. t The calculation and fusion are performed in three stages: Calculate the feature distribution dispersion: Extract the feature distribution histogram from the head region of the t-th data block. and histogram of historical normal data distribution Calculate the KL divergence: Where M represents the total number of histogram buckets, H t (m) represents the frequency of the m-th feature bucket in the current data block, H hist (m) represents the frequency of the m-th feature bucket in historical normal data, D t H represents the dispersion of the characteristic distribution of the current data block. hist The `hist` in the middle represents the historical normal data distribution; Calculate historical classification accuracy: Obtain the sequence of the K most recent classification results for the same source identifier {r}. t-K+1 ,…,r t }, where r j ∈{0,1}, and a moving weighted average is calculated using exponentially decaying weights ρ∈(0,1), as follows: Where K represents the number of historical classification results, ρ represents the moving weighted decay factor, and r t-K+j Let A represent the classification result of the (t-K+j)th classification. t Represents the moving weighted average of historical classification accuracy; Dynamic fusion to generate confidence score: defining the freshness coefficient φ t With timestamp τ t Incrementing, represented as: f t =1-exp[-λ(τ t -t ref )], Where, τ t τ represents the timestamp of the current data block. ref Indicates the reference start time, λ represents the freshness growth rate, and φ t This represents the freshness weighting coefficient; The dispersion and accuracy are combined by weight, and the dispersion is normalized to obtain the maximum possible dispersion D. max The formula is: Among them, D max c represents the maximum value of the divergence normalization. t This indicates the generated confidence level index.

6. The data classification and processing method for parallel computing on a multi-core processor as described in claim 1, characterized in that, The exception handling channel mentioned in step S3 includes: An isolated storage area is allocated in the last-level cache to store low-confidence data blocks and their feature tags; Start a separate monitoring thread to periodically scan the isolated storage area, and trigger a recalculation when one of the following conditions is met: The confidence level of newly arriving data blocks with the same source identifier has recovered to above the first threshold. The overall system load rate is lower than the set threshold and the duration exceeds the preset period.

7. The data classification and processing method for parallel computing on a multi-core processor as described in claim 6, characterized in that, The core ratio adjustment in step S3 adopts a dual closed-loop control strategy: The inner loop adjusts the thread migration frequency between core groups based on the real-time queue depth, while the outer loop predicts the core group size requirements for future time windows based on historical load patterns. When the average processing latency of the lightweight computing core group exceeds β times the latency of the main computing core group, a specified number of core resources are forcibly reclaimed and allocated to the main computing core group.

8. The data classification and processing method for parallel computing on a multi-core processor as described in claim 7, characterized in that, In step S3, when the average processing latency L of the lightweight computing core group t Exceeding the main computing core latency M t When the latency ratio is β times, the following resource reclamation and allocation mechanism is triggered: by calculating the latency ratio and comparing it with the threshold, the number of cores to be reclaimed is dynamically determined, and then, while ensuring the availability of the lightweight group, the resources are preferentially allocated to the main computing core group to restore overall performance. If R t >1, then: N L,t+1 =N L,t -ΔN,N M,t+1 =N M,t +ΔN, Among them, L t M represents the average processing latency of the lightweight computing kernel at time t. t R represents the average processing latency of the main computing kernel group at time t, β represents the latency multiplier threshold, and R t N represents the delay ratio. L,t N represents the current number of cores in the lightweight computing core group. M,t N represents the current number of cores in the main compute core group. L,min κ represents the minimum number of cores allowed in the lightweight group, L represents the lightweight core group, M represents the host core group, and ΔN represents the number of core resources recycled in this instance. The expression indicates rounding up, and min(·) indicates taking the minimum of the two values.

9. A data classification and processing system for parallel computing on multi-core processors, based on the data classification and processing method for parallel computing on multi-core processors as described in any one of claims 1 to 8, characterized in that, include: The data ingestion module is configured to divide the input data stream into continuous data blocks and add a timestamp and source identifier to each data block; The confidence prediction module is connected to the data ingestion module and configured to generate a confidence index for data blocks; The elastic computing module includes a main computing core group, a lightweight computing core group, and an exception handling channel, and its input is connected to the confidence prediction module. The dynamic scheduling module is configured to collect the operating indicators of each core group and adjust the core allocation ratio, and its control terminal is connected to the elastic computing module. The main computing core group includes a cascaded feature extraction unit, a dimensionality reduction unit, and a classification decision unit, while the lightweight computing core group bypasses the dimensionality reduction unit and directly connects to the classification decision unit.

10. A data classification and processing system for parallel computing on a multi-core processor as described in claim 9, characterized in that, The exception handling channel includes: The cache isolation controller is configured to dynamically divide isolated storage regions within the last-level cache. A recalculation trigger is connected to the cache isolation controller and the dynamic scheduling module and is configured to activate the data reprocessing process when preset conditions are met.

Images