Multi-channel parallel data acquisition and processing method and system

Abstract

The present invention relates to the field of data processing technology, specifically to a multi-channel parallel data acquisition and processing method and system, comprising the following steps: by performing data acquisition on each channel, recording the data type and quantity, using a timer to synchronize the data streams of multiple channels, and storing the data in a database, performing unified processing of timestamps, and summarizing and generating an original data set. In the present invention, data is collected in real time through channels to ensure data synchronization and time consistency of multiple signal sources, provide a reliable basis for subsequent analysis, refine hierarchical information, strengthen data relevance, improve processing depth, perform feature classification using a classifier, and significantly improve classification accuracy and real-time performance. In performance evaluation, the comparison between true values ​​and predicted values ​​can extract misclassified samples, provide a basis for subsequent adjustments, combine load monitoring with dynamic resource adjustment, optimize resource use, make allocation more flexible and adaptable, and effectively improve the responsiveness and stability of multi-channel processing.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a multi-channel parallel data acquisition and processing method and system. Background Art

[0002] The field of data processing technology involves technologies and methods for collecting, converting, analyzing and storing raw data. The field covers a variety of technologies, including data mining, signal processing and statistical analysis. The goal of data processing is to extract valuable information from data to support decision-making and optimize system performance. In multi-channel data processing scenarios, the key to the technology lies in how to efficiently process data from multiple signal sources simultaneously to improve the speed and accuracy of data acquisition, thereby achieving real-time analysis and feedback.

[0003] Among them, the multi-channel parallel data acquisition and processing method refers to the simultaneous collection and processing of data from differentiated sources through multiple channels. This method is widely used in medical monitoring, environmental monitoring, industrial control and other fields. Its uses include real-time monitoring of physiological signals, environmental changes and industrial equipment status, etc., and efficient data analysis and rapid response are achieved through parallel processing. This technology helps to improve the real-time and reliability of the system and enhance decision-making support capabilities.

[0004] Existing technologies have certain limitations in the data processing process, including the real-time and accuracy of multi-channel data. It is difficult to achieve efficient synchronization of data acquisition from multiple signal sources, resulting in inconsistent data timestamps, which affects the reliability of analysis results. Traditional methods often rely on fixed indicators for feature extraction, which are difficult to dynamically adapt to changing signal characteristics, resulting in insufficient analysis depth and inflexible resource allocation, leading to slow response during peak load periods, affecting overall performance. For example, in medical monitoring scenarios, delayed signal processing leads to misjudgment of patient conditions, affecting the effectiveness of timely intervention, limiting the application of existing technologies in multi-channel parallel data processing and failing to meet the actual needs of high real-time and high accuracy requirements. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a multi-channel parallel data acquisition and processing method and system.

[0006] In order to achieve the above object, the present invention adopts the following technical solution, a multi-channel parallel data acquisition and processing method, comprising the following steps:

[0007] S1: Collect data from each channel, record the data type and quantity, synchronize the data streams of multiple channels using a timer, store the data in a database, perform unified processing of timestamps, and aggregate to generate the original data set;

[0008] S2: extract key features based on the original data set, perform multi-channel feature value mapping, use weight adjustment mechanism to refine hierarchical information, analyze and optimize reconstruction errors, and generate deep feature representation;

[0009] S3: importing the deep feature representation into the classifier, performing feature classification, calculating the confidence of the classification result, using threshold judgment to perform category division, and generating a channel data classification decision result;

[0010] S4: Performing performance evaluation on the channel data classification decision result and recording it, comparing the true value with the predicted value, extracting the data of the misclassified samples, and obtaining the channel classification performance data;

[0011] S5: According to the channel classification performance data, monitor the load during the processing, record the use of multi-channel resources, calculate the processing time and occupancy ratio, dynamically adjust the resources, and obtain channel load monitoring data;

[0012] S6: Based on the channel load monitoring data, perform statistical analysis on resource usage, identify load change trends, dynamically adjust resource allocation, and generate a channel resource scheduling strategy.

[0013] As a further solution of the present invention, the original data set includes channel data type, channel data volume, and timestamp information; the deep feature representation includes hierarchical features, mapping features, and reconstruction errors; the channel data classification decision results include classification labels, confidence values, and categories; the channel classification performance data include true values, predicted values, and misclassified samples; the channel load monitoring data include processing load, resource usage, and processing time; and the channel resource scheduling strategy includes resource allocation schemes, scheduling rules, and load adjustments.

[0014] As a further solution of the present invention, by collecting data for each channel, recording the data type and quantity, synchronizing the data streams of multiple channels using a timer, storing the data in a database, and performing unified processing of the timestamp, the steps of summarizing and generating the original data set are specifically as follows:

[0015] S101: by collecting data for each channel, deploying a data collection unit on the data channel, each unit is equipped with a monitoring MEMS sensor, automatically recording the type and quantity of multiple types of data according to a set frequency, assigning an identification code to each data type, and accumulating and recording the data volume to obtain a data type recording result;

[0016] S102: according to the data type recording result, a timer is set to control the data collection frequency, and a time synchronization protocol is used to adjust the data collection time of multiple channels to generate a time-synchronized data stream record;

[0017] S103: Integrate the time-synchronized data stream records and store them in a database, perform time stamp processing on each piece of data, encode and classify the data using a unified time format, summarize and organize the data, and construct an original data set.

[0018] As a further solution of the present invention, the steps of extracting key features based on the original data set, performing multi-channel feature value mapping, using a weight adjustment mechanism to refine hierarchical information, analyzing and optimizing reconstruction errors, and generating deep feature representation are specifically as follows:

[0019] S201: screening the original data set, selecting data points associated with the target variable, creating a primary data model using the data points, screening characteristics associated with the target, and obtaining a preliminary key data set;

[0020] S202: applying the preliminary key data set to feature mapping, assigning an initial weight to each data channel, and dynamically adjusting the weight according to the data response of each channel, refining the feature hierarchy, and generating an adjusted feature mapping set;

[0021] S203: Using the adjusted feature mapping set, measuring the reconstruction error of each feature, adjusting mapping parameters, iteratively optimizing the error, and obtaining a deep feature representation.

[0022] As a further solution of the present invention, the steps of importing the deep feature representation into the classifier, performing feature classification, calculating the confidence of the classification result, and using threshold judgment to perform category division to generate the channel data classification decision result are specifically as follows:

[0023] S301: Input the deep feature representation for classification processing, match the feature with the preset classification standard according to the similarity, assign a preliminary category to each data point, use the similarity score to determine the category of each feature, and obtain a preliminary classification label set;

[0024] S302: Based on the preliminary classification label set, a confidence evaluation is performed on each category label, including a category contribution analysis of the feature and a consistency analysis of the data points within the category, a numerical confidence is calculated for each category label, and a confidence value set is generated;

[0025] S303: Using the confidence value set, a threshold is set for category determination. For each category, the confidence is compared with the threshold. Category labels exceeding the threshold are determined to be qualified. Category verification is performed through threshold comparison to obtain channel data classification decision results.

[0026] As a further solution of the present invention, the performance evaluation of the channel data classification decision result is performed and recorded, the true value is compared with the predicted value, the data of the misclassified samples is extracted, and the steps of obtaining the channel classification performance data are specifically as follows:

[0027] S401: According to the channel data classification decision result, a data comparison unit is configured, and parameters are set in the unit to extract the predicted value and the true value of each channel. The unit uses an internal script to compare the data of each channel, and records the matching and non-matching results to obtain a matching difference record;

[0028] S402: Analyze the matching difference records and use a data filtering mechanism to filter out misclassified samples, including detecting whether the predicted category of each sample is consistent with the real-time category, and recording information of inconsistent samples to generate a misclassified sample list;

[0029] S403: Using the misclassified sample list, applying the random forest algorithm, performing statistical analysis and evaluating the performance of each channel, calculating the classification accuracy of each channel, collating the data to form a performance overview of each channel, summarizing the data and performing performance evaluation, and obtaining channel classification performance data.

[0030] As a further solution of the present invention, the formula of the random forest algorithm is as follows:

[0031] ,

[0032] Calculate the classification performance and get the performance value of each channel, where represents the classification accuracy, Represents the total number of samples with correct channel classification, Represents the total number of samples processed by the channel, represents the number of misclassified samples of the channel, is the adjustment coefficient.

[0033] As a further solution of the present invention, according to the channel classification performance data, the load in the processing process is monitored, the use of multi-channel resources is recorded, the processing time and occupancy ratio are calculated, and resources are dynamically adjusted. The steps of obtaining channel load monitoring data are specifically as follows:

[0034] S501: Based on the channel classification performance data, monitor the load of each channel, record the resource usage of each channel during the processing, count the processing time and resource consumption of each channel, and integrate the data to obtain the channel load status;

[0035] S502: Based on the channel load situation, analyze the occupancy ratio of multiple channels, evaluate resource utilization efficiency, compare the load differences between multiple channels, and perform dynamic adjustments to generate resource utilization evaluation results;

[0036] S503: Based on the resource usage evaluation result, implement dynamic resource adjustment, optimize channel resource allocation, adjust processing priority, perform load balancing, and obtain channel load monitoring data.

[0037] As a further solution of the present invention, the steps of performing statistical analysis of resource usage based on the channel load monitoring data, identifying load change trends, dynamically adjusting resource allocation, and generating a channel resource scheduling strategy are specifically as follows:

[0038] S601: Based on the channel load monitoring data, statistics are collected on the usage of multiple channel resources, the average load, maximum load and minimum load of each channel are calculated, the data are integrated, and channel resource usage statistics are obtained;

[0039] S602: Based on the channel resource usage statistics, identify the load change trend, analyze the load fluctuations of multiple channels in differentiated time periods, compare the load data within the time period, and generate a load change trend analysis result;

[0040] S603: Based on the load change trend analysis result, dynamically adjust the resource allocation of the channel, optimize the resource configuration, implement the scheduling strategy, and generate the channel resource scheduling strategy.

[0041] A multi-channel parallel data acquisition and processing system, the multi-channel parallel data acquisition and processing system is used to execute the multi-channel parallel data acquisition and processing method, the system comprises:

[0042] The data acquisition module collects data from each channel, records multi-channel data, classifies data types and quantities, sets timers and synchronizes data streams, stores data in a database, processes timestamps and unifies formats, and generates raw data sets;

[0043] The feature extraction module extracts key features based on the original data set, performs feature value mapping and combines the weight adjustment mechanism to refine the hierarchical information, analyze the reconstruction error, and generate a deep feature representation;

[0044] The classification decision module inputs the data into the classifier based on the deep feature representation, performs feature classification, calculates the confidence of the classification result, uses the threshold to divide the category, and generates the channel data classification decision result;

[0045] The performance evaluation module evaluates the classification performance based on the channel data classification decision result, compares the true value with the predicted value, extracts the misclassified sample data, and generates channel classification performance data;

[0046] The resource scheduling module monitors the load situation, records the resource usage situation, calculates the processing time and occupancy ratio, dynamically adjusts the resource allocation, and generates a channel resource scheduling strategy based on the channel classification performance data.

[0047] Compared with the prior art, the advantages and positive effects of the present invention are:

[0048] In the present invention, data is collected through each channel, and the data type and quantity are recorded in real time to achieve efficient synchronization of data streams from multiple signal sources. This synchronization process ensures the consistency of data in time, provides a reliable basis for subsequent analysis, extracts key features from the original data set, and refines hierarchical information, so that the data correlation in the analysis process can be strengthened and the depth of data processing can be improved. The generation of deep feature representations, the execution of feature classification by a classifier, the calculation of the confidence of the results, and the use of threshold judgment for category division greatly improves the accuracy and real-time performance of classification. In performance evaluation, the comparison between the true value and the predicted value further extracts misclassified samples, providing a basis for subsequent adjustments. The combination of load monitoring and dynamic resource adjustment optimizes resource usage, making resource allocation more flexible and adaptable, thereby effectively responding to differentiated processing needs and improving the responsiveness and stability of multi-channel parallel processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0050] Figure 2 This is a detailed flow chart of S1 of the present invention;

[0051] Figure 3 This is a detailed flow chart of S2 of the present invention;

[0052] Figure 4 This is a detailed flow chart of S3 of the present invention;

[0053] Figure 5 This is a detailed flow chart of S4 of the present invention;

[0054] Figure 6 This is a detailed flow chart of S5 of the present invention;

[0055] Figure 7 This is a detailed flow chart of S6 of the present invention;

[0056] Figure 8 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] In the description of the present invention, it is necessary to interpret that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be interpreted as a limitation on the present invention. In addition, in the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0059] See also Figure 1 The present invention provides a technical solution, a multi-channel parallel data acquisition and processing method, comprising the following steps:

[0060] S1: Collect data from each channel, record the data type and quantity, synchronize the data streams of multiple channels using a timer, store the data in a database, perform unified processing of timestamps, and aggregate to generate the original data set;

[0061] S2: Extract key features based on the original data set, perform multi-channel feature value mapping, use the weight adjustment mechanism to refine the hierarchical information, analyze and optimize the reconstruction error, and generate deep feature representation;

[0062] S3: Import the deep feature representation into the classifier, perform feature classification, calculate the confidence of the classification result, use threshold judgment to divide the category, and generate the channel data classification decision result;

[0063] S4: Evaluate and record the performance of the channel data classification decision results, compare the true value with the predicted value, extract the data of the misclassified samples, and obtain the channel classification performance data;

[0064] S5: Based on the channel classification performance data, monitor the load during the processing, record the use of multi-channel resources, calculate the processing time and occupancy ratio, dynamically adjust resources, and obtain channel load monitoring data;

[0065] S6: Based on the channel load monitoring data, perform statistical analysis on resource usage, identify load change trends, dynamically adjust resource allocation, and generate a channel resource scheduling strategy.

[0066] The original data set includes channel data type, channel data volume, and timestamp information. The deep feature representation includes hierarchical features, mapping features, and reconstruction errors. The channel data classification decision results include classification labels, confidence values, and categories. The channel classification performance data includes true values, predicted values, and misclassified samples. The channel load monitoring data includes processing load, resource usage, and processing time. The channel resource scheduling strategy includes resource allocation plans, scheduling rules, and load adjustments.

[0067] Specifically, if Figure 2 As shown in the figure, by collecting data from each channel, recording the data type and quantity, using the timer to synchronize the data streams of multiple channels, storing the data in the database, and performing unified processing of the timestamp, the steps of summarizing and generating the original data set are as follows:

[0068] S101: by collecting data for each channel, deploying a data collection unit on the data channel, configuring a monitoring MEMS sensor for each unit, automatically recording the type and quantity of multiple types of data at a set frequency, assigning an identification code to each data type, and accumulating and recording the data volume, the execution process of obtaining the data type recording result is as follows;

[0069] By collecting data for each channel, the deployed data acquisition unit monitors the MEMS sensors on each channel to realize automatic recording of multiple types of data, including various sensor data such as temperature, pressure, acceleration, etc. The data acquisition unit uses the high sensitivity and fast response characteristics of the MEMS sensor to continuously record data according to the set acquisition frequency. Each data type is classified and stored according to the preset identification code. By parsing the identification code, the source and type of data can be accurately tracked and managed. At the same time, the system will automatically update the data storage status according to the increase in data volume to ensure the integrity and accuracy of the data, providing reliable original data input for subsequent data processing and analysis.

[0070] S102: According to the data type recording result, a timer is set to control the data collection frequency, and the time synchronization protocol is used to adjust the data collection time of multiple channels, and the execution process of generating a time-synchronized data stream record is as follows;

[0071] By using the time synchronization protocol to adjust the data acquisition time of multiple channels, according to the formula:

[0072] ,

[0073] Calculate the data collection time after time synchronization, where: Represents the time point after synchronization. Represents the time of the local data acquisition unit, Represents the deviation correction value from the master time;

[0074] In a multi-channel data acquisition system, the local time of each data acquisition unit Due to various reasons (such as clock deviation, operating system processing speed, etc.) and the main control system time, in order to synchronize the collection time of all data acquisition units, the time synchronization protocol is used to calculate the time deviation , the deviation is obtained through the Network Time Protocol (NTP) server;

[0075] For example, if the local time of a data acquisition unit is 15000 milliseconds, and the master control system detects through NTP that the unit is 200 milliseconds behind the master control time, then milliseconds, through Milliseconds are used to complete time synchronization, ensuring the accuracy of synchronous data collection and recording.

[0076] S103: Integrate the time-synchronized data stream records and store them in the database, perform timestamp processing on each piece of data, encode and classify the data using a unified time format, and summarize and organize the data. The execution process of constructing the original data set is as follows;

[0077] The time-synchronized data stream records are integrated and stored in the database, and each data is timestamped. The data is encoded and classified in a unified time format. The powerful functions of database management are used to achieve efficient storage and fast retrieval of data. The timestamp processing of data includes encoding the time synchronization data of each data acquisition unit in a unified time format. The process involves the formatting and standardization of time data to ensure the consistency of all data during storage and query. The classification process logically partitions the data according to the type and source of the data, which not only helps to organize and manage the data, but also improves the efficiency of data query. The original data set constructed through processing provides a structured and standardized data foundation for subsequent data analysis and application.

[0078] Specifically, if Figure 3 As shown in the figure, the steps of extracting key features based on the original data set, performing multi-channel eigenvalue mapping, using the weight adjustment mechanism to refine the hierarchical information, analyzing and optimizing the reconstruction error, and generating deep feature representation are as follows:

[0079] S201: Screening the original data set, selecting data points associated with the target variable, using the data points to create a primary data model, screening the characteristics associated with the target, and obtaining the preliminary key data set. The execution process is as follows;

[0080] The original data set is screened and data points that are highly correlated with the target variable are selected. In the process of building the primary data model, the main focus is on the correlation analysis between the data points and the target variable, which involves complex statistical methods and data processing techniques. To ensure the accuracy and effectiveness of the data model, the data must be quality controlled and preprocessed. The screened data points will serve as the basis for model training, and further extract the characteristics associated with the target variable to form a preliminary key data set, which will directly affect the performance and prediction accuracy of subsequent models.

[0081] S202: Apply the preliminary key data set to the feature map, assign an initial weight to each data channel, and dynamically adjust the weight according to the data response of each channel, refine the feature hierarchy, and generate the adjusted feature map set. The execution process is as follows;

[0082] Assign initial weights to each data channel, and dynamically adjust the weights based on the data response of each channel, according to the formula:

[0083] ,

[0084] Calculate the adjusted weights, where represents the adjusted weight, represents the original weight, represents the learning rate, T represents the target output, and Y represents the predicted output. Represents input features;

[0085] Referring to the requirements of feature mapping, the initial weights Randomly assign according to the characteristics of multiple channels, for example, the initial weight of a channel is set to 0.5, and the learning rate is set is 0.01, the target output T is 1 (the expected response result), the current predicted output Y is 0.8, and the input feature is a specific value of the data channel, such as 0.3. According to the weight adjustment formula, the new weight The calculation is as follows: ;

[0086] This computational process shows how weights can be dynamically adjusted based on data response to optimize feature mapping and feature representation.

[0087] S203: Using the adjusted feature mapping set, measuring the reconstruction error of each feature, adjusting the mapping parameters, iteratively optimizing the error, and obtaining the deep feature representation. The execution process is as follows;

[0088] Using the adjusted feature mapping set, the reconstruction error of each feature is measured. Through the iterative optimization process, the mapping parameters are adjusted to ensure that the data model can capture and represent deep features more accurately. The measurement of reconstruction error involves calculating the difference between the actual output and the expected output, and the mean square error or cross entropy loss function is used for evaluation. This step is a key link in the model training process. Through continuous iterative optimization, the model will gradually reduce the prediction error, improve its generalization ability and prediction accuracy, and the deep feature representation finally obtained can effectively reflect the complex structure and internal connection of the input data, which is crucial for subsequent data application and analysis.

[0089] Specifically, if Figure 4 As shown in the figure, the steps of importing the deep feature representation into the classifier, performing feature classification, calculating the confidence of the classification result, and using threshold judgment to divide the categories to generate the channel data classification decision result are as follows:

[0090] S301: Input deep feature representation for classification processing, match the feature with the preset classification standard according to its similarity, assign a preliminary category to each data point, use the similarity score to determine the category of each feature, and obtain the execution process of the preliminary classification label set as follows;

[0091] The deep feature representation is input for classification processing. In the process of matching the features according to the similarity with the preset classification standards, a variety of similarity measurement methods are used, such as cosine similarity or Euclidean distance, to ensure accurate identification and matching of the association between each data point and the known category. Each data point can be assigned a preliminary category that best suits its characteristics, and the similarity score is used to determine the category of each feature. Then, a preliminary classification label set is obtained. This label set is the basis for subsequent processing and provides an important basis for further analysis and decision-making, ensuring that the data points are correctly classified, thereby enhancing the application effect and practicality of the model.

[0092] S302: Based on the preliminary classification label set, a confidence evaluation is performed on each category label, including the category contribution of the feature and the consistency analysis of the data points within the category, and a numerical confidence is calculated for each category label. The execution process of generating a confidence value set is as follows;

[0093] Confidence evaluation is performed on each category label, including the category contribution of the feature and the consistency analysis of the data points within the category, according to the formula:

[0094] ,

[0095] Calculate the confidence, where C represents the confidence and n represents the total number of data points in the category. Represents the category contribution of a single data point, Represents the average category contribution of all data points in the category;

[0096] In the category confidence assessment, a category is assumed to include 10 data points, and the contribution of each point is 0.1, 0.2, 0.1, 0.3, 0.2, 0.1, 0.2, 0.1, 0.3, 0.2, and the average category contribution of the data points is calculated: ;

[0097] Then apply the formula to calculate the confidence level:

[0098] ,

[0099] This calculation shows the consistency of data points within a category, which in turn determines the credibility of the category label and is a key indicator for evaluating the accuracy of the classification model.

[0100] S303: Use the confidence value set to set a threshold for category determination. For each category, compare the confidence with the threshold. Category labels exceeding the threshold are determined to be qualified. Category verification is performed through threshold comparison. The execution process of obtaining the channel data classification decision result is as follows;

[0101] When using a confidence value set and setting a threshold for category judgment, it is necessary to determine a suitable threshold. This threshold is based on historical data analysis. The confidence of each category is compared with the threshold. Category labels that exceed the threshold are determined to be qualified. The judgment process is a key step to ensure the accuracy of classification results. Through threshold comparison, not only can the validity of the category be verified, but also the data category can be further subdivided, the performance of the classification model can be optimized, the accuracy and reliability of each category can be ensured, and a scientific basis can be provided for the final decision, thereby enhancing the application value and actual influence of the classification results.

[0102] Specifically, if Figure 5 As shown, the performance of the channel data classification decision results is evaluated and recorded, the true value is compared with the predicted value, the data of the misclassified samples is extracted, and the specific steps for obtaining the channel classification performance data are as follows:

[0103] S401: According to the channel data classification decision result, a data comparison unit is configured, and parameters are set in the unit to extract the predicted value and the true value of each channel. The unit uses an internal script to compare the data of each channel, and records the matching and non-matching results to obtain the matching difference record. The execution process is as follows;

[0104] According to the channel data classification decision results, the data comparison unit is configured, and parameters are set in the unit to extract the predicted value and true value of each channel. This process involves designing the structure of the data comparison unit to ensure that the input channel data can be effectively processed. The data comparison unit is first initialized and the necessary resources such as memory and processing power are configured to match the data processing requirements. For each channel, the system extracts key data parameters such as data length, data format and key timestamps. The parameters are used to calibrate the data processing process to ensure the accuracy and real-time nature of the data. The unit uses internal scripts to automatically extract the predicted value and true value of each channel. The data of each channel is processed separately according to the preset parameters to ensure the independence and comparability of the data. The internal script compares the predicted value with the true value according to the set logic, and any mined differences will be recorded. By looping through each channel, the system records all matching and unmatched results, and finally generates a detailed matching difference record. This record not only includes the matching results, but also lists the comparison of each channel in detail, providing a data basis for further analysis.

[0105] S402: Analyze the matching difference records and use the data filtering mechanism to filter out the samples with incorrect classification, including detecting whether the predicted category of each sample is consistent with the real-time category, and recording the information of the inconsistent samples. The execution process of generating the list of misclassified samples is as follows;

[0106] Analyze the matching difference records and use the data filtering mechanism to filter out the misclassified samples. First, import the matching difference records into the analysis module, which is equipped with an efficient data processing algorithm that can quickly identify the key information in the records. By setting specific filtering conditions, such as the minimum probability threshold for category mismatch and the prediction error limit, the system filters out samples whose prediction results are inconsistent with the real-time category. For each marked sample, its identification information, category, predicted probability value and its position in the data set are recorded in detail. The detailed records will be used to generate a multidimensional list of misclassified samples. This list not only provides detailed data of misclassified samples, but also provides a basis for correcting classification models and improving algorithms. In this way, key misclassification information can be effectively filtered out from a large amount of data, assisting researchers and engineers in optimizing data processing processes and improving the overall performance of decision-making systems.

[0107] S403: Using the misclassified sample list, applying the random forest algorithm, performing statistical analysis and evaluating the performance of each channel, calculating the classification accuracy of each channel, collating the data to form a performance overview of each channel, summarizing the data and performing performance evaluation, the execution process of obtaining channel classification performance data is as follows;

[0108] The formula for the random forest algorithm is as follows:

[0109] ,

[0110] Calculate the classification performance and get the performance value of each channel, where represents the classification accuracy, Represents the total number of samples with correct channel classification, Represents the total number of samples processed by the channel, represents the number of misclassified samples of the channel, is the adjustment coefficient.

[0111] Detailed explanation of the formula and the process of formula calculation and derivation:

[0112] This formula is used to calculate the classification accuracy of a channel, where It indicates the number of samples correctly classified in this channel, which is obtained by comparing the labels of the peasant painting sample set with the classification results output by the algorithm. is the total number of samples processed by the channel, which is obtained by recording the samples input by each channel. The number of samples misclassified for this channel, which is also the result obtained through label comparison is an adjustment coefficient used to balance the impact of misclassification. This value is determined based on comparative analysis of previous test data to provide the most optimized accuracy adjustment;

[0113] Suppose that in one evaluation cycle, a channel processes 1000 samples, of which 950 samples are correctly classified, and 50 samples are misclassified. According to the actual monitoring data, you can set , , , referring to the channel performance stability and misclassification sensitivity, setting ;

[0114] Substitute the specific numbers into the formula for calculation:

[0115] ,

[0116] The results show that after referring to the adjustment coefficient of misclassification, the accuracy of the channel is 93.882%. This numerical result reflects the performance of the channel when processing a specific sample set under standard conditions, and significantly reflects the impact of misclassification and the importance of the adjustment coefficient. This performance evaluation result can be used for further channel optimization and parameter adjustment strategy formulation.

[0117] Specifically, if Figure 6 As shown, according to the channel classification performance data, the load in the processing process is monitored, the use of multi-channel resources is recorded, the processing time and occupancy ratio are calculated, and resources are dynamically adjusted. The specific steps for obtaining channel load monitoring data are as follows:

[0118] S501: Based on the channel classification performance data, monitor the load of each channel, record the resource usage of each channel during the processing, count the processing time and resource consumption of each channel, and integrate the data. The execution process of obtaining the channel load is as follows;

[0119] Based on channel classification performance data, the process of monitoring the load of each channel involves detailed records of the resource usage of the channel during the processing process, including CPU occupancy, memory usage, and the frequency of input and output operations. The statistics of processing time and resource consumption of each channel are automatically completed through integrated monitoring tools. The data is integrated into channel load status. The data is crucial for analyzing the performance status of each channel and provides basic data for optimizing resource allocation and performance adjustment. Administrators can clearly see which channels have become bottlenecks and which have excess resources, so as to make corresponding resource reconfiguration and optimization.

[0120] S502: Based on the channel load situation, analyze the occupancy ratio of multiple channels, evaluate the resource utilization efficiency, compare the load differences between multiple channels, and make dynamic adjustments to generate the resource utilization evaluation result. The execution process is as follows;

[0121] Analyze the occupancy rate of multiple channels and evaluate resource utilization efficiency according to the formula:

[0122] ,

[0123] Calculate the average resource utilization efficiency, where E represents resource utilization efficiency and m represents the total number of channels. Represents the resource occupancy ratio of the zth channel;

[0124] Assume that there are 5 channels, and the resource usage of each channel is 20%, 30%, 50%, 40% and 60% respectively. First calculate the sum of these values: ;

[0125] Then calculate the average resource utilization efficiency according to the formula: ;

[0126] This shows that under the current system configuration, the average resource occupancy rate of each channel is 40%. Through calculation, we can further compare the load differences between multiple channels and guide the dynamic adjustment of resources, thereby optimizing the overall resource utilization efficiency.

[0127] S503: Based on the resource usage evaluation result, implement dynamic resource adjustment, optimize channel resource allocation, adjust processing priority, and perform load balancing. The execution process of obtaining channel load monitoring data is as follows;

[0128] Based on the resource usage evaluation results, the process of implementing dynamic resource adjustment includes detailed operations of optimizing channel resource allocation and adjusting processing priorities. By setting a reasonable load balancing strategy, the system can dynamically adjust resources according to real-time monitoring data, such as increasing resource allocation for channels with weaker processing capabilities and reducing resource waste in idle channels. This dynamic adjustment helps the system maintain efficient operation, reduces performance bottlenecks caused by uneven resource allocation, ensures balanced operation of multiple channels, and improves overall processing efficiency and response speed, thereby effectively improving the overall performance of the system and the user's service experience.

[0129] Specifically, if Figure 7 As shown in the figure, based on the channel load monitoring data, statistical analysis of resource usage is performed, load change trends are identified, resource allocation is dynamically adjusted, and the steps of generating a channel resource scheduling strategy are as follows:

[0130] S601: Based on the channel load monitoring data, statistics are collected on the usage of multiple channel resources, the average load, maximum load and minimum load of each channel are calculated, the data are integrated, and the execution process of obtaining channel resource usage statistics is as follows;

[0131] Based on the channel load monitoring data, statistics on multi-channel resource usage are performed, which involves calculating the average load, maximum load and minimum load of each channel. Integrating data is the key to obtaining channel resource usage statistics. The methods used include data collection, processing and analysis, and the use of automated tools to continuously monitor the resource usage of each channel. Through data analysis, the peak and trough of resource usage of each channel can be identified, which provides a basis for further resource optimization. Statistical data is crucial for analyzing the overall operating efficiency of the equipment and the performance of each channel.

[0132] S602: Based on the channel resource usage statistics, identify the load change trend, analyze the load fluctuation of multiple channels in differentiated time periods, compare the load data within the time period, and generate the load change trend analysis result. The execution process is as follows;

[0133] Analyze the load fluctuation of multiple channels in different time periods according to the formula:

[0134] ,

[0135] Calculate the average load change trend, where: Represents the average load change trend, Represents the maximum load in a certain period of time. represents the minimum load, represents the number of time periods measured;

[0136] If there are data from three time periods, and the maximum and minimum load values ​​of each period are (60%, 30%), (70%, 40%), and (80%, 50%), calculate the load change in each time period: (60-30)=30%, (70-40)=30%, (80-50)=30%;

[0137] Average load change trend: ,

[0138] This formula assists system administrators in analyzing the volatility of resource demand within differentiated time periods, and further guides the rational allocation and optimization of resources.

[0139] S603: Based on the load change trend analysis results, dynamically adjust the channel resource allocation, optimize resource configuration, implement the scheduling strategy, and generate the execution process of the channel resource scheduling strategy as follows;

[0140] Based on the results of load change trend analysis, dynamic adjustment of channel resource allocation includes optimizing resource configuration and implementing scheduling strategies. Resources are reallocated according to the analysis results to ensure that high-load channels can obtain more resource support, while low-load channels have less resource allocation. This dynamic adjustment helps to maintain the balanced operation of the system and avoid performance problems caused by excessive or insufficient resource allocation. Through reasonable scheduling strategies, the system can respond more flexibly to differentiated operating conditions and improve response speed and service quality.

[0141] like Figure 8 As shown, a multi-channel parallel data acquisition and processing system includes:

[0142] The data acquisition module collects data from each channel, records multi-channel data, classifies data types and quantities, sets timers and synchronizes data streams, stores data in a database, processes timestamps and unifies formats, and generates raw data sets;

[0143] The feature extraction module extracts key features based on the original data set, performs feature value mapping and combines it with the weight adjustment mechanism to refine the hierarchical information, analyze the reconstruction error, and generate deep feature representation;

[0144] The classification decision module is based on deep feature representation, inputs data into the classifier, performs feature classification, calculates the confidence of the classification result, uses the threshold to divide the category, and generates the channel data classification decision result;

[0145] The performance evaluation module evaluates the classification performance based on the channel data classification decision results, compares the true value with the predicted value, extracts the misclassified sample data, and generates channel classification performance data;

[0146] The resource scheduling module monitors the load, records resource usage, calculates processing time and occupancy ratio, dynamically adjusts resource allocation, and generates channel resource scheduling strategies based on channel classification performance data.

[0147] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A multi-channel parallel data acquisition and processing method, characterized in that: The following steps are involved: By collecting data from each channel, recording the data type and quantity, using timers to synchronize the data streams of multiple channels, storing the data in a database, and performing unified processing of timestamps, the original data set is generated in aggregate; Extract key features based on the original data set, perform multi-channel feature value mapping, use a weight adjustment mechanism to refine hierarchical information, analyze and optimize reconstruction errors, and generate deep feature representation; The steps of extracting key features based on the original data set, performing multi-channel feature value mapping, using the weight adjustment mechanism to refine the hierarchical information, analyzing and optimizing the reconstruction error, and generating deep feature representation are as follows: Screening the original data set, selecting data points associated with the target variable, creating a primary data model using the data points, screening the characteristics associated with the target, and obtaining a preliminary key data set; Applying the preliminary key data set to feature mapping, assigning initial weights to each data channel, and dynamically adjusting the weights based on the data response of each channel, refining the feature hierarchy, and generating an adjusted feature mapping set; Using the adjusted feature mapping set, measuring the reconstruction error of each feature, adjusting mapping parameters, iteratively optimizing the error, and obtaining a deep feature representation; Importing the deep feature representation into the classifier, performing feature classification, calculating the confidence of the classification result, using threshold judgment to perform category division, and generating channel data classification decision results; The steps of importing the deep feature representation into the classifier, performing feature classification, calculating the confidence of the classification result, and using threshold judgment to perform category division to generate the channel data classification decision result are as follows: Input the deep feature representation for classification processing, match the feature with the preset classification standard according to the similarity, assign a preliminary category to each data point, use the similarity score to determine the category of each feature, and obtain a preliminary classification label set; Based on the preliminary classification label set, a confidence evaluation is performed on each category label, including a category contribution analysis of the feature and a consistency analysis of the data points within the category, a numerical confidence is calculated for each category label, and a confidence value set is generated; Using the confidence value set, a threshold is set for category determination. For each category, the confidence is compared with the threshold. Category labels exceeding the threshold are determined to be qualified. Category verification is performed through threshold comparison to obtain channel data classification decision results. Performing performance evaluation on the channel data classification decision results and recording them, comparing the true value with the predicted value, extracting the data of the misclassified samples, and obtaining channel classification performance data; The steps of performing performance evaluation on the channel data classification decision result and recording it, comparing the true value with the predicted value, extracting the data of the misclassified samples, and obtaining the channel classification performance data are as follows: According to the channel data classification decision result, a data comparison unit is configured, and parameters are set in the unit to extract the predicted value and the true value of each channel. The unit uses an internal script to compare the data of each channel, and records the matching and non-matching results to obtain a matching difference record; Analyze the matching difference records, use a data filtering mechanism to filter out misclassified samples, including detecting whether the predicted category of each sample is consistent with the real-time category, and record information of inconsistent samples to generate a list of misclassified samples; Using the misclassified sample list, applying the random forest algorithm, performing statistical analysis and evaluating the performance of each channel, calculating the classification accuracy of each channel, collating the data to form a performance overview of each channel, aggregating the data and performing performance evaluation, and obtaining channel classification performance data; According to the channel classification performance data, monitor the load during the processing, record the use of multi-channel resources, calculate the processing time and occupancy ratio, dynamically adjust resources, and obtain channel load monitoring data; According to the channel classification performance data, the load in the processing process is monitored, the use of multi-channel resources is recorded, the processing time and occupancy ratio are calculated, and resources are dynamically adjusted. The steps of obtaining channel load monitoring data are specifically as follows: Based on the channel classification performance data, monitor the load of each channel, record the resource usage of each channel during the processing, count the processing time and resource consumption of each channel, and integrate the data to obtain the channel load status; Based on the channel load conditions, analyze the occupancy ratios of multiple channels, evaluate resource utilization efficiency, compare load differences between multiple channels, and make dynamic adjustments to generate resource utilization evaluation results; Based on the resource usage evaluation results, implement dynamic resource adjustment, optimize channel resource allocation, adjust processing priority, perform load balancing, and obtain channel load monitoring data; Based on the channel load monitoring data, statistical analysis of resource usage is performed to identify load change trends, dynamically adjust resource allocation, and generate a channel resource scheduling strategy; The steps of performing statistical analysis of resource usage based on the channel load monitoring data, identifying load change trends, dynamically adjusting resource allocation, and generating a channel resource scheduling strategy are as follows: Based on the channel load monitoring data, statistics on the use of multiple channel resources are performed, the average load, maximum load and minimum load of each channel are calculated, the data is integrated, and channel resource usage statistics are obtained; Based on the channel resource usage statistics, identify the load change trend, analyze the load fluctuations of multiple channels in differentiated time periods, compare the load data within the time period, and generate a load change trend analysis result; Based on the load change trend analysis results, the resource allocation of the channel is dynamically adjusted, the resource configuration is optimized, the scheduling strategy is implemented, and the channel resource scheduling strategy is generated.

2. The multi-channel parallel data acquisition and processing method according to claim 1, characterized in that: The original data set includes channel data type, channel data volume, and timestamp information; the deep feature representation includes hierarchical features, mapping features, and reconstruction errors; the channel data classification decision results include classification labels, confidence values, and categories; the channel classification performance data include true values, predicted values, and misclassified samples; the channel load monitoring data includes processing load, resource usage, and processing time; and the channel resource scheduling strategy includes resource allocation plans, scheduling rules, and load adjustments.

3. The multi-channel parallel data acquisition and processing method according to claim 1, characterized in that: By collecting data from each channel, recording the data type and quantity, using timers to synchronize the data streams of multiple channels, storing the data in a database, and performing unified timestamp processing, the steps for summarizing and generating the original data set are as follows: By collecting data for each channel, deploying data collection units on the data channels, each unit is equipped with a monitoring MEMS sensor, automatically recording the types and quantities of multiple types of data at a set frequency, assigning an identification code to each data type, and accumulating and recording the data volume to obtain the data type recording results; According to the data type recording result, a timer is set to control the data collection frequency, and a time synchronization protocol is used to adjust the data collection time of multiple channels to generate a time-synchronized data stream record; The time-synchronized data stream records are integrated and stored in a database, each piece of data is timestamped, the data is encoded and classified using a unified time format, and the data is summarized and organized to construct an original data set.

4. The multi-channel parallel data acquisition and processing method according to claim 1, characterized in that: The formula of the random forest algorithm is as follows: ， Calculate the classification performance and get the performance value of each channel, where represents the classification accuracy, Represents the total number of samples with correct channel classification, Represents the total number of samples processed by the channel, represents the number of misclassified samples of the channel, is the adjustment coefficient.

5. Multi-channel parallel data acquisition and processing system, characterized in that: According to the multi-channel parallel data acquisition and processing method according to any one of claims 1 to 4, the system comprises: The data acquisition module collects data from each channel, records multi-channel data, classifies data types and quantities, sets timers and synchronizes data streams, stores data in a database, processes timestamps and unifies formats, and generates raw data sets; The feature extraction module extracts key features based on the original data set, performs feature value mapping and combines the weight adjustment mechanism to refine the hierarchical information, analyze the reconstruction error, and generate a deep feature representation; The classification decision module inputs the data into the classifier based on the deep feature representation, performs feature classification, calculates the confidence of the classification result, uses the threshold to divide the category, and generates the channel data classification decision result; The performance evaluation module evaluates the classification performance based on the channel data classification decision result, compares the true value with the predicted value, extracts the misclassified sample data, and generates channel classification performance data; The resource scheduling module monitors the load situation, records the resource usage situation, calculates the processing time and occupancy ratio, dynamically adjusts the resource allocation, and generates a channel resource scheduling strategy based on the channel classification performance data.

Images