A method and system for computing resource-aware hierarchical feature identification of heterogeneous data streams

Abstract

The application provides a kind of computing resource perception heterogeneous data stream hierarchical feature identification method and system, method includes: based on monitoring parameter set obtains comprehensive resource stress index;Through nonlinear mapping function and comprehensive resource stress index obtain feature extraction depth coefficient, and then obtain initial feature extraction dimension number;Through minimum feature dimension threshold and the guarantee dimension number corresponding to task type, initial feature extraction dimension number is converted into final feature extraction dimension number;Corresponding to the feature priority queue of task type is obtained, and the target feature is selected from the feature priority queue based on final feature extraction dimension number, to obtain feature vector.By the above-mentioned mode, with resource monitoring state dynamic driving feature classification, it provides the continuously adjustable feature classification strategy, and balances the computing power overhead and precision.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for hierarchical feature recognition of heterogeneous data streams with computing resource awareness. Background Technology

[0002] In a cloud-edge-device collaborative architecture, edge computing nodes are responsible for real-time processing of heterogeneous data streams generated by connected end-side devices (such as cameras, sensors, and user terminals). The typical process is as follows: end-side devices transmit collected video, audio, and sensor readings to the edge node. After preprocessing such as decoding, alignment, and fusion, feature vectors are extracted according to a preset strategy. Then, AI inference filters for high-value information before reporting to the cloud. However, such systems generally face a structural contradiction between "limited computing power" and "unlimited data." Taking video surveillance security scenarios as an example, performing full-frame, full-feature deep learning analysis on all video streams would exhaust limited computing resources; simply reducing the analysis frequency or resolution might miss key anomalies.

[0003] To address the aforementioned issues, existing technologies primarily follow two approaches. The first is computational resource scheduling optimization, which predicts task demand by establishing a resource consumption statistical model and then queues tasks or elastically scales resources accordingly. Its shortcomings are: resource scheduling and feature extraction are independent; the scheduling module is indifferent to data content, and when resources are scarce, it can only indiscriminately degrade all tasks, failing to guarantee the accuracy of high-value data analysis. The second approach is data tiering and classification, with preset high, medium, and low tiers, each using different processing strategies. Its shortcomings are: the tiering strategy is statically preset, unable to dynamically adjust the boundaries and processing depth of each tier based on real-time resource status; it wastes computing power when resources are plentiful, and continues to operate with aggressive strategies when resources are scarce, leading to task backlog. More importantly, these two approaches lack a linkage mechanism: the resource scheduling system is indifferent to the quality of feature extraction, and the feature tiering system is indifferent to resource sufficiency; they make independent decisions. When resource status changes, the feature tiering strategy cannot respond immediately; when data features fluctuate, the resource scheduling strategy is unaware. This open-loop architecture lacks status feedback capabilities and cannot autonomously adjust system behavior according to environmental changes. While some systems support coarse-grained policy switching based on fixed thresholds (such as dropping some tasks when CPU utilization exceeds 80%), the triggering of these policies is discrete and preset. The feature extraction granularity can only jump between a limited number of levels, and there is no automatic reversal after switching based on resource recovery. Essentially, it is still an open-loop control mode. In addition, existing solutions mostly rely on large-model AI solutions such as graph neural networks and deep reinforcement learning. The models are huge and difficult to meet the requirements of edge devices for lightweight and real-time performance.

[0004] In summary, existing technologies suffer from the following fundamental limitations: policy triggering is mostly open-loop, lacking an immediate causal relationship between resource monitoring and feature classification; policy granularity is mostly discrete and jump-like, failing to achieve continuous and smooth adjustment of feature dimensions; degradation policies are mostly indiscriminate, unable to prioritize the recognition accuracy of high-value data under resource constraints; and existing solutions largely rely on large-model AI solutions, making it difficult to meet the lightweight deployment requirements of edge scenarios. These limitations leave the system in a dilemma of "either wasting computing power or having insufficient accuracy" under resource fluctuations. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for hierarchical feature recognition of heterogeneous data streams with computing resource awareness. This invention addresses the technical problems in existing technologies, such as the lack of real-time causal drive between resource monitoring and feature hierarchical recognition due to the use of an open-loop mode, the inability to continuously and smoothly adjust feature granularity due to the use of a discrete jump mode, the inability to guarantee the accuracy of high-value data under resource constraints due to the use of an indiscriminate degradation mode, and the difficulty in meeting the requirements of lightweight edge deployment due to reliance on large-model AI solutions. These problems result in the system being in a dilemma of wasted computing power and insufficient accuracy under resource fluctuation environments.

[0006] To achieve the above objectives, in a first aspect, embodiments of this application provide a computing resource-aware heterogeneous data stream hierarchical feature identification method, applied to an edge computing node, wherein the edge computing node is connected to at least one end-side device, and the computing resource-aware heterogeneous data stream hierarchical feature identification method includes the following steps: A set of monitoring parameters is obtained based on a fixed sampling period, and a comprehensive resource pressure index is obtained based on the set of monitoring parameters. The feature extraction depth coefficient is obtained by using a nonlinear mapping function and a comprehensive resource pressure index, and the initial feature extraction dimension is obtained based on the feature extraction depth coefficient. The initial feature extraction dimension number is converted into the stage feature extraction dimension number by using the minimum feature dimension threshold, the guaranteed dimension number corresponding to the task type is obtained, and the stage feature extraction dimension number is converted into the final feature extraction dimension number by using the guaranteed dimension number. Obtain a feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. Select the top few task features from the feature priority queue that correspond to the final feature extraction dimension as target features to obtain feature vectors.

[0007] Furthermore, the monitoring parameter set includes several monitoring parameters, and the step of obtaining the comprehensive resource pressure index based on the monitoring parameter set includes: Several monitoring parameters were normalized to obtain several resource indicator values; A comprehensive resource pressure index is obtained based on several resource indicator values.

[0008] Furthermore, the formula for obtaining the comprehensive resource pressure index is: , in, Indicates the first The overall resource pressure index at any given moment. Indicates the first Resource weight coefficients for each resource indicator value Indicates the first The first moment Each resource indicator value, This represents the total number of resource indicator values.

[0009] Furthermore, after the steps of obtaining a monitoring parameter set based on a fixed sampling period and obtaining a comprehensive resource pressure index based on the monitoring parameter set, the method further includes: The fixed sampling period is adjusted based on the comprehensive resource pressure index.

[0010] Furthermore, the formula for obtaining the feature extraction depth coefficient is as follows: , in, Indicates the first Feature extraction depth coefficients at time step The representation is defined as follows: Represents an exponential function. Indicates the preset steepness coefficient. Indicates the first The overall resource pressure index at any given moment. This represents the preset decision median.

[0011] Furthermore, the formula for obtaining the initial feature extraction dimension number is: , in, Indicates the first The initial feature extraction dimension at time step [time]. The representation is defined as follows: Indicates the first Feature extraction depth coefficients at time step This represents the maximum allowed feature dimension threshold. This indicates rounding down to the nearest integer.

[0012] Furthermore, the step of converting the initial feature extraction dimension number into the stage feature extraction dimension number through the minimum feature dimension threshold includes: The initial number of feature extraction dimensions is compared with the minimum feature dimension threshold; If the initial feature extraction dimension is less than or equal to the minimum feature dimension threshold, then the minimum feature dimension threshold is selected as the stage feature extraction dimension. If the initial feature extraction dimension is greater than the minimum feature dimension threshold, then the initial feature extraction dimension is selected as the stage feature extraction dimension.

[0013] Furthermore, the step of converting the stage feature extraction dimension number into the final feature extraction dimension number by ensuring the dimension number includes: Compare the number of dimensions extracted for stage features with the number of dimensions for guarantee; If the number of feature extraction dimensions in a stage is less than the guaranteed number of dimensions, then the guaranteed number of dimensions will be selected as the final number of feature extraction dimensions. If the number of feature extraction dimensions in a stage is greater than or equal to the guaranteed number of dimensions, then the number of feature extraction dimensions in a stage will be selected as the final number of feature extraction dimensions.

[0014] Furthermore, after the step of obtaining a feature priority queue corresponding to the task type, wherein the feature priority queue includes several task features arranged in descending order of importance, and selecting the top several task features corresponding to the final feature extraction dimension from the feature priority queue as target features to obtain the feature vector, the method further includes: The index difference between two adjacent comprehensive resource pressure indices is obtained, and the index difference is compared with the change tolerance threshold for closed-loop control.

[0015] Secondly, embodiments of this application provide a computing resource-aware heterogeneous data stream hierarchical feature recognition system, applied to the computing resource-aware heterogeneous data stream hierarchical feature recognition method as described in the first aspect above, the system comprising: The acquisition module is used to acquire a set of monitoring parameters based on a fixed sampling period, and to acquire a comprehensive resource pressure index based on the set of monitoring parameters. The association module is used to obtain the feature extraction depth coefficient through a nonlinear mapping function and a comprehensive resource pressure index, and to obtain the initial feature extraction dimension based on the feature extraction depth coefficient. The analysis module is used to convert the initial feature extraction dimension number into the stage feature extraction dimension number by using the minimum feature dimension threshold, obtain the guaranteed dimension number corresponding to the task type, and convert the stage feature extraction dimension number into the final feature extraction dimension number by using the guaranteed dimension number. The execution module is used to obtain a feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. The first few task features corresponding to the final feature extraction dimension number are selected from the feature priority queue as target features to obtain feature vectors.

[0016] Thirdly, embodiments of this application provide a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the computing resource-aware heterogeneous data stream hierarchical feature recognition method as described in the first aspect above.

[0017] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the computing resource-aware heterogeneous data stream hierarchical feature recognition method as described in the first aspect above.

[0018] Compared with existing technologies, the advantages of this invention are as follows: By obtaining a comprehensive resource pressure index and then obtaining a feature extraction depth coefficient through a nonlinear mapping function, the initial feature extraction dimension is obtained, establishing a direct causal relationship between resource status and feature extraction. This enables resource status changes to drive strategy adjustments in real time, coupling two independent technical routes (resource scheduling and feature grading) into a unified closed-loop system and establishing a causal functional dependency between them. Furthermore, by obtaining the feature extraction depth coefficient through a nonlinear function mapping, its continuous S-shaped mapping allows the feature extraction depth coefficient to take any continuous value within the numerical range, and the initial feature extraction dimension can be continuously adjusted, realizing a control paradigm shift from shifting gears to focusing. This approach overcomes the dilemma of resource waste or insufficient accuracy caused by discrete jump modes. It transforms the initial feature extraction dimension number by using a minimum feature dimension threshold, and further transforms the stage feature extraction dimension number by combining it with a guaranteed dimension number. The final feature extraction dimension number is used as the standard for obtaining feature vectors. When resources are scarce, it can forcibly increase the dimension number according to the task type and prioritize the extraction of high-importance features, thus achieving differentiated accuracy assurance for key data and overcoming the defect of indiscriminate degradation that leads to the loss of accuracy in identifying high-value data. The recognition process is implemented based on lightweight computational logic, without relying on large AI models, which significantly reduces computational complexity and storage requirements, meeting the deployment requirements of edge computing nodes for lightweight and real-time performance. Attached Figure Description

[0019] Figure 1 This is a flowchart of the heterogeneous data stream hierarchical feature recognition method for computing resource awareness in the first embodiment of the present invention; Figure 2 This is a structural block diagram of the heterogeneous data stream hierarchical feature recognition system with computing resource awareness in the second embodiment of the present invention; The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0020] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0021] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] Please see Figure 1 The computing resource-aware heterogeneous data stream hierarchical feature recognition method provided in the first embodiment of the present invention is applied to an edge computing node. The edge computing node is connected to at least one end-side device. Understandably, the edge computing node is connected to the cloud. The computing resource-aware heterogeneous data stream hierarchical feature recognition method includes the following steps: S10: Obtain a set of monitoring parameters based on a fixed sampling period, and obtain a comprehensive resource pressure index based on the set of monitoring parameters; Understandably, the edge computing node uses a fixed sampling period (i.e., the time interval between each sampling is the same, ranging from 100ms to 1000ms). The monitoring parameter set includes several monitoring parameters. In this embodiment, the monitoring parameter set includes three monitoring parameters, which are the current processor busy time, the currently allocated physical memory, and the current network transmission rate.

[0024] Step S10 includes: S110: Normalize several monitoring parameters to obtain several resource indicator values; Specifically, the normalized processor utilization is obtained by dividing the current processor busy time by the total sampling period time; the normalized memory utilization is obtained by dividing the currently allocated physical memory by the total physical memory of the system; and the normalized bandwidth utilization is obtained by dividing the current network transmission rate by the maximum available bandwidth of the system. In essence, these three monitoring parameters correspond to three resource indicator values: normalized processor utilization, normalized memory utilization, and normalized bandwidth utilization. It should be noted that the resource indicator values ​​range from [0, 1].

[0025] S120: A comprehensive resource pressure index is obtained based on several resource indicator values; The formula for obtaining the comprehensive resource pressure index is as follows: , in, Indicates the first The overall resource pressure index at any given moment. Indicates the first Resource weight coefficients for each resource indicator value Indicates the first The first moment Each resource indicator value, This represents the total number of resource indicator values.

[0026] The closer the comprehensive resource pressure index is to 1, the greater the overall resource pressure of the system and the less available capacity; conversely, the closer it is to 1, the less the overall resource pressure and the more abundant the available capacity. The nonlinear fusion of multi-dimensional resources is achieved by subtracting the product of the idle rates of each dimension from 1, which is a nonlinear weighted fusion mechanism. When any dimension experiences resource stress, the comprehensive pressure index will move closer to 1, meaning the overall system pressure is dominated by the most stressed resource dimension, rather than a simple average. When multiple dimensions are simultaneously under moderate pressure, the product operation produces a superlinear amplification effect, reflecting the accelerated decline in system processing capacity when multiple dimensions are under simultaneous pressure. By adjusting the resource weight coefficients, the system can adapt to the resource sensitivity characteristics of different scenarios.

[0027] Preferably, the method further includes: S50: Adjust the fixed sampling period based on the comprehensive resource pressure index; Specifically, the comprehensive resource pressure index is compared with the first pressure threshold and the second pressure threshold respectively. If the first pressure threshold is less than the second pressure threshold, the fixed sampling period is extended if the comprehensive resource pressure index is less than the first pressure threshold, and the fixed sampling period is shortened if the comprehensive pressure index is greater than the second pressure threshold.

[0028] In this embodiment, the first pressure threshold is 0.3 and the second pressure threshold is 0.8. When the comprehensive resource pressure index is less than the first pressure threshold, the fixed sampling period is extended to reduce resource consumption. When the comprehensive pressure index is greater than the second pressure threshold, the fixed sampling period is shortened to enhance response sensitivity.

[0029] S20: Obtain the feature extraction depth coefficient through the nonlinear mapping function and the comprehensive resource pressure index, and obtain the initial feature extraction dimension based on the feature extraction depth coefficient; The formula for obtaining the feature extraction depth coefficient is as follows: , in, Indicates the first Feature extraction depth coefficients at time step The representation is defined as follows: Represents an exponential function. This indicates a preset steepness coefficient, preferably. The value ranges from 4 to 12 (the larger the value, the more sensitive the system is to resource changes, and the smaller the value, the smoother the response). This represents the preset decision median, preferably... The value ranges from 0.3 to 0.7. This corresponds to the decision-making threshold of the system when resource pressure is at a moderate level. It should be noted that the domain is [0, 1], and the range is [0, 1]. The value of is [0, 1].

[0030] In the above manner, the comprehensive resource pressure index is continuously and smoothly mapped to the feature extraction depth coefficient through an S-shaped function. Specifically, the mapping is a decreasing S-shaped nonlinear continuous mapping with saturation characteristics, ensuring that the system will not produce an overreaction under extreme resource conditions.

[0031] The acquisition of feature extraction depth coefficients has the following characteristics: 1. Continuity: Follow The changes are continuous, with no abrupt transitions, thus avoiding policy oscillations at threshold boundaries; 2. Monotonically decreasing: Follow The feature extraction depth coefficient decreases monotonically as resource pressure increases; 3. S-shaped saturation characteristic: when Significantly lower than When resources are plentiful. Approaching the upper limit of 1, when Significantly higher than At times (resource shortages), Approaching the lower limit value of 0, when exist When nearby, It exhibits near-linear sensitivity, and this saturation at both ends ensures that the system will not overreact under extreme resource conditions; 4. Adjustability: By adjusting the preset steepness coefficient, the sensitivity of the system to resource changes can be controlled. The larger the preset steepness coefficient, the steeper the middle section of the S-curve, and the more sensitive the system's response is near the preset decision median. The smaller the preset steepness coefficient, the smoother the curve transition, and the gentler the system's adjustment. The introduction of the nonlinear mapping function realizes a nonlinear transformation from physical resource states to strategy parameters, which is different from existing linear or piecewise mappings.

[0032] The formula for obtaining the initial feature extraction dimension is: , in, Indicates the first The initial feature extraction dimension at time step [time]. This represents the maximum allowed feature dimension threshold. This indicates rounding down. By mapping the feature extraction depth coefficients to discrete initial feature extraction dimensions with a step size of 1 dimension, the feature granularity is continuously adjustable.

[0033] S30: Convert the initial feature extraction dimension number into the stage feature extraction dimension number by using the minimum feature dimension threshold, obtain the guaranteed dimension number corresponding to the task type, and convert the stage feature extraction dimension number into the final feature extraction dimension number using the guaranteed dimension number; Step S30 includes: S310: Compare the initial number of feature extraction dimensions with the minimum feature dimension threshold; Preferably, the minimum feature dimension threshold is 3 to 5.

[0034] S320: If the initial feature extraction dimension is less than or equal to the minimum feature dimension threshold, then the minimum feature dimension threshold is selected as the stage feature extraction dimension; if the initial feature extraction dimension is greater than the minimum feature dimension threshold, then the initial feature extraction dimension is selected as the stage feature extraction dimension. S330: Compare the number of dimensions extracted for stage features with the number of guaranteed dimensions; S340: If the number of feature extraction dimensions in a stage is less than the guaranteed number of dimensions, then the guaranteed number of dimensions will be selected as the final number of feature extraction dimensions. If the number of feature extraction dimensions in a stage is greater than or equal to the guaranteed number of dimensions, then the number of feature extraction dimensions in a stage will be selected as the final number of feature extraction dimensions. Understandably, in security scenarios, high-risk areas at high-risk times, or in e-commerce scenarios, high-value business requests, require more refined identification. Therefore, it is necessary to ensure sufficient feature extraction dimensions. Preferably, in security scenarios, the number of dimensions for task types in high-risk times and high-risk areas is 0.9. In e-commerce scenarios, the maximum allowed feature dimension threshold for high-value business request task types is 0.8. Maximum allowed feature dimension threshold.

[0035] When the final feature extraction dimension number equals the maximum allowed feature dimension threshold, the system performs fine-grained feature recognition, extracting all usable features. When the final feature extraction dimension number equals the minimum allowed feature dimension threshold, the system performs coarse-grained feature recognition, extracting only the most core basic features. When the final feature extraction dimension number is between the minimum and maximum allowed feature dimension thresholds, the system performs medium-grained feature recognition, extracting features corresponding to the final feature extraction dimension number. This dimension enhancement mechanism ensures accuracy even in high-risk areas of security scenarios and high-value business requests in e-commerce scenarios when resources are limited, achieving precise delivery of limited computing power.

[0036] S40: Obtain the feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. Select the top several task features from the feature priority queue that correspond to the final feature extraction dimension as target features to obtain feature vectors. For any task type, the system maintains a feature priority queue. If it contains 50 task features and the final feature extraction dimension is 43, the first 43 task features are extracted from the feature priority queue as target features. All target features are concatenated into an active feature subset, and feature vectors are extracted by combining the heterogeneous data stream. The importance of task features in the feature priority queue can be determined based on the contribution of each task feature to the target recognition task in historical data (e.g., calculated through information gain, Gini impurity reduction, or SHAP contribution in cooperative game theory). The formula for obtaining the feature vector is: , in, Indicates the first The feature vector at time step, Indicates the first Heterogeneous data streams corresponding to time and task type Indicates the first A subset of active features at any given moment This represents the feature extraction operator.

[0037] Feature extraction operators, depending on the heterogeneous data stream, include, but are not limited to, the following specific implementation methods: For structured data (such as e-commerce user behavior logs): algebraic operations based on field projection are used to extract corresponding attribute values ​​from the corresponding columns of the heterogeneous data stream based on the field names specified in the active feature subset; For unstructured data (such as surveillance video frames): an early termination strategy of deep learning neural networks is adopted, terminating forward propagation only at the deepest index corresponding to the active feature to avoid full network computational overhead; For semi-structured data (such as JSON-formatted sensor data streams): extraction operations based on key path matching are used to extract corresponding values ​​from the heterogeneous data stream based on the key names or key paths specified in the active feature subset. Furthermore, after obtaining the feature vector, the feature vector, several resource indicator values, and the current timestamp are encapsulated into a structured output result according to a preset data interface format.

[0038] Preferably, the method further includes: S60: Obtain the index difference between two adjacent comprehensive resource pressure indices, compare the index difference with the change tolerance threshold, and perform closed-loop control. In some embodiments, the absolute difference between two adjacent comprehensive resource pressure indices is selected as the index difference. The value range of the change tolerance threshold is 0.01 to 0.1. Preferably, the change tolerance threshold is 0.05. If the index difference is greater than the change tolerance threshold, step S20 is re-executed to obtain the initial feature extraction dimension number again.

[0039] In this embodiment, the change tolerance threshold includes a first threshold and a second threshold. The first threshold is greater than the second threshold, and the first threshold is 0.07, while the second threshold is 0.03. The previous comprehensive resource pressure index (at the previous moment) is subtracted from the current comprehensive resource pressure index to obtain the index difference. If the index difference is greater than 0 (indicating a system transition from low to high pressure), the index difference is compared with the first threshold. If the index difference is greater than the first threshold, step S20 is re-executed to re-obtain the initial feature extraction dimension (i.e., the first threshold is triggered when resource pressure increases; a larger value is used for conservative downgrading to avoid unnecessary accuracy loss due to transient fluctuations). If the index difference is less than 0 (indicating a system transition from high to low pressure), the absolute value of the index difference is compared with the second threshold. If the absolute value of the index difference is greater than the second threshold, step S20 is re-executed to re-obtain the initial feature extraction dimension (i.e., the second threshold is triggered when resource pressure decreases; a smaller value is used for aggressive upgrading to ensure rapid accuracy improvement once resources recover). The setting of the first and second thresholds avoids unnecessary strategy switching due to transient fluctuations. By implementing closed-loop control, the system can continuously track changes in resource status and automatically adjust to the optimal operating point, thus forming a negative feedback control system with asymptotic stability in control theory.

[0040] This embodiment verifies effectiveness through simulation experiments. Specifically, it simulates an edge computing node of a smart park security system. This node simultaneously connects to 16 high-definition cameras (the simulated video streams are surveillance video clips extracted from the publicly available UCF-Crime dataset, with a resolution of 1920×1080 and a frame rate of 25fps), and is deployed on an edge server with limited GPU resources (NVIDIA Tesla T4, 16GB of VRAM). The experimental objective is to identify abnormal events (intrusion, loitering, fighting, etc.) in real time from multiple video streams under GPU computing power constraints, prioritizing the identification accuracy of high-risk areas (such as warehouses and vaults).

[0041] Heterogeneous data streams are analog surveillance video data streams, which include video frame data (unstructured image data, simulated from public datasets), audio data (time-series signal data, simulated from public voice datasets), and sensor data (structured event data, including simulated door magnetic status, infrared beam trigger signals, etc.). The three have different formats, forming a typical heterogeneous data stream.

[0042] The response results of the comparison index are obtained using the method in this embodiment and compared with those of conventional fixed-strategy edge devices (which always extract features of all dimensions and passively drop frames when resources are scarce). The results are shown in Table 1 below: Table 1 , Specifically, this is demonstrated in the following ways: Resource Adaptability Verification: When GPU utilization increased from 45% to 85%, the system automatically reduced the video analysis dimension in non-high-risk areas from 43 dimensions to 8 dimensions, reducing the GPU computational overhead per video stream by approximately 72%. This proves that the system can dynamically adjust its strategy based on resource status. In contrast, edge devices using conventional fixed strategies experience frame drops under the same resource pressure and cannot adjust autonomously. Key Data Accuracy Guarantee Verification: Under high-pressure conditions with 85% GPU utilization, the video stream in the high-risk area (east gate of the factory warehouse) maintains 45-dimensional (guaranteed dimension) fine-grained recognition in the simulated test set. The false negative rate for abnormal events in high-risk areas is 0%, proving that the accuracy of critical data identification can be prioritized even under resource constraints. System capacity improvement verification: Under the same GPU resource constraints, this embodiment increases the number of video channels simultaneously supported by edge nodes from 16 (fixed full-frame analysis) to 42 (experimental simulation data), an increase of approximately 162%, demonstrating a significant improvement in system processing capacity under resource constraints. Real-time performance verification: The measured response time from resource status change to policy adjustment completion is approximately 180ms, and the alarm response time remains within 3 seconds, meeting the real-time requirements of security scenarios. These experimental results demonstrate the effectiveness of the method in this embodiment in security scenarios.

[0043] Furthermore, this embodiment simulates the recommendation system service node of an e-commerce platform during a major promotional event. The data comes from a publicly available dataset (user behavior records extracted from the Amazon Reviews Dataset, including event types such as clicks, browsing, adding to cart, and payments). The simulation time window is 2 hours, and the simulated request volume is approximately 1.2 million times. The experimental objective is to identify high-value users (users who are making payments or adding to cart) in real time from massive amounts of user behavior data under CPU and memory resource constraints, ensuring the recommendation accuracy of the core transaction chain.

[0044] Heterogeneous data streams are simulated e-commerce user behavior data streams, including user click stream data (semi-structured JSON format, simulated from public datasets), transaction record data (relational table format, simulated from public datasets), user profile data (key-value pair format, simulated), and product attribute data (structured table format, simulated from public datasets). These four data streams have different formats and constitute a typical heterogeneous data stream.

[0045] The response results of the comparison index are obtained using the method in this embodiment and compared with those of conventional fixed-strategy edge devices (which always extract all 100-dimensional features, and request queuing or timeout when resources are scarce). The results are shown in Table 2 below: Table 2 , Specifically, this manifests as: resource adaptive capability verification: when CPU utilization increases from 75% to 88% ( When the efficiency was increased from 0.534 to 0.72, the system automatically reduced the analysis dimensions of ordinary browsing requests from 43 to 15, releasing approximately 65% ​​of the feature extraction computing power. This demonstrates that the system can dynamically adjust its strategy based on resource status. In contrast, edge devices using conventional fixed strategies experience request queuing and timeouts under the same resource pressure and cannot adjust autonomously. Key data accuracy assurance verification: Under high-pressure conditions with 88% CPU utilization, payment requests maintained 80-dimensional (guaranteed dimension) fine-grained recognition. On the simulated test set, the recommendation accuracy for payment scenarios maintained a fine-grained level of over 92%, demonstrating that the system can prioritize the recognition accuracy of high-value services when resources are scarce. System throughput improvement verification: Under the same resource configuration, the method in this embodiment increased system throughput by approximately 35% (simulated experimental data), and reduced critical service response latency by approximately 42%. Resource-accuracy trade-off optimization verification: Compared to a fixed 3-level hierarchical strategy (feature dimensions can only jump between 100, 50, and 10 dimensions), the continuous adjustment strategy in this embodiment increased the area under the resource-accuracy trade-off curve by approximately 28%, demonstrating that the continuous adjustment strategy is superior to the discrete jump strategy. Experimental results based on publicly available datasets and simulated environments demonstrate the effectiveness of the method described in this embodiment in e-commerce scenarios.

[0046] By obtaining the comprehensive resource pressure index and then acquiring the feature extraction depth coefficient through a nonlinear mapping function, the initial feature extraction dimension is determined. This establishes a direct causal relationship between resource status and feature extraction, enabling resource status changes to drive strategy adjustments in real time. This couples two independent technical approaches (resource scheduling and feature grading) into a unified closed-loop system, establishing a causal functional dependency between them. The feature extraction depth coefficient is obtained through a nonlinear function mapping; its continuous S-shaped mapping allows the feature extraction depth coefficient to take any continuous value within the numerical range, and the initial feature extraction dimension can be continuously adjusted. This achieves a shift in control paradigm from shifting gears to focusing, overcoming the limitations of discrete jump modes. This addresses the dilemma of resource waste or insufficient accuracy caused by traditional methods. By transforming the initial feature extraction dimension number through a minimum feature dimension threshold and further transforming the stage feature extraction dimension number in conjunction with a guaranteed dimension number, the final feature extraction dimension number serves as the standard for obtaining feature vectors. Under resource constraints, this method can forcibly increase the dimension number based on task type and prioritize the extraction of highly important features, achieving differentiated accuracy assurance for critical data and overcoming the defect of indiscriminate degradation leading to impaired accuracy in identifying high-value data. The recognition process is based on lightweight computational logic, eliminating the need to rely on large AI models, significantly reducing computational complexity and storage requirements, and meeting the deployment requirements of edge computing nodes for lightweight and real-time performance.

[0047] Please see Figure 2 The second embodiment of the present invention provides a computing resource-aware heterogeneous data stream hierarchical feature recognition system. This system is applied to the computing resource-aware heterogeneous data stream hierarchical feature recognition method described in the above embodiments, and will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0048] The system includes: The acquisition module 10 is used to acquire a set of monitoring parameters based on a fixed sampling period, and to acquire a comprehensive resource pressure index based on the set of monitoring parameters. The acquisition module 10 includes: The first unit is used to normalize several monitoring parameters to obtain several resource indicator values. The second unit is used to obtain a comprehensive resource pressure index based on several resource indicator values; Preferably, the system further includes: The first adjustment module 50 is used to adjust the fixed sampling period based on the comprehensive resource pressure index; The association module 20 is used to obtain the feature extraction depth coefficient through the nonlinear mapping function and the comprehensive resource pressure index, and to obtain the initial feature extraction dimension based on the feature extraction depth coefficient; Analysis module 30 is used to convert the initial feature extraction dimension number into the stage feature extraction dimension number through the minimum feature dimension threshold, obtain the guaranteed dimension number corresponding to the task type, and convert the stage feature extraction dimension number into the final feature extraction dimension number through the guaranteed dimension number. The analysis module 30 includes: The third unit is used to compare the initial feature extraction dimension number with the minimum feature dimension threshold; The fourth unit is used to select the minimum feature dimension threshold as the stage feature extraction dimension if the initial feature extraction dimension is less than or equal to the minimum feature dimension threshold, and to select the initial feature extraction dimension as the stage feature extraction dimension if the initial feature extraction dimension is greater than the minimum feature dimension threshold. The fifth unit is used to compare the number of dimensions for extracting stage features with the number of guaranteed dimensions; The sixth unit is used to select the guaranteed dimension as the final feature extraction dimension if the number of feature extraction dimensions in the stage is less than the guaranteed dimension, and to select the number of feature extraction dimensions in the stage as the final feature extraction dimension if the number of feature extraction dimensions in the stage is greater than or equal to the guaranteed dimension. The execution module 40 is used to obtain a feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. The first few task features corresponding to the final feature extraction dimension number are selected from the feature priority queue as target features to obtain feature vectors. Preferably, the system further includes: The second adjustment module 60 is used to obtain the index difference between two adjacent comprehensive resource pressure indices, and compare the index difference with the change tolerance threshold to perform closed-loop control.

[0049] The present invention also provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the computing resource-aware heterogeneous data stream hierarchical feature recognition method as described in the above technical solutions.

[0050] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the heterogeneous data stream hierarchical feature recognition method with computing resource awareness as described in the above technical solutions.

[0051] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0052] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness, applied to edge computing nodes, wherein the edge computing node is connected to at least one end-side device, characterized in that, The computational resource-aware heterogeneous data stream hierarchical feature recognition method includes the following steps: A set of monitoring parameters is obtained based on a fixed sampling period, and a comprehensive resource pressure index is obtained based on the set of monitoring parameters. The feature extraction depth coefficient is obtained by using a nonlinear mapping function and a comprehensive resource pressure index, and the initial feature extraction dimension is obtained based on the feature extraction depth coefficient. The initial feature extraction dimension number is converted into the stage feature extraction dimension number by using the minimum feature dimension threshold, the guaranteed dimension number corresponding to the task type is obtained, and the stage feature extraction dimension number is converted into the final feature extraction dimension number by using the guaranteed dimension number. Obtain a feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. Select the top few task features from the feature priority queue that correspond to the final feature extraction dimension as target features to obtain feature vectors.

2. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, The monitoring parameter set includes several monitoring parameters, and the step of obtaining the comprehensive resource pressure index based on the monitoring parameter set includes: Several monitoring parameters were normalized to obtain several resource indicator values; A comprehensive resource pressure index is obtained based on several resource indicator values.

3. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 2, characterized in that, The formula for obtaining the comprehensive resource pressure index is as follows: ， in, Indicates the first The overall resource pressure index at any given moment. Indicates the first Resource weight coefficients for each resource indicator value Indicates the first The first moment Each resource indicator value, This represents the total number of resource indicator values.

4. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, After the steps of obtaining the monitoring parameter set based on a fixed sampling period and obtaining the comprehensive resource pressure index based on the monitoring parameter set, the method further includes: The fixed sampling period is adjusted based on the comprehensive resource pressure index.

5. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, The formula for obtaining the feature extraction depth coefficient is as follows: ， in, Indicates the first Feature extraction depth coefficients at time step The representation is defined as follows: Represents an exponential function. Indicates the preset steepness coefficient. Indicates the first The overall resource pressure index at any given moment. This represents the preset decision median.

6. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, The formula for obtaining the initial feature extraction dimension is: ， in, Indicates the first The initial feature extraction dimension at time step [time]. The representation is defined as follows: Indicates the first Feature extraction depth coefficients at time step This represents the maximum allowed feature dimension threshold. This indicates rounding down to the nearest integer.

7. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, The step of converting the initial feature extraction dimension number into the stage feature extraction dimension number by using the minimum feature dimension threshold includes: The initial number of feature extraction dimensions is compared with the minimum feature dimension threshold; If the initial feature extraction dimension is less than or equal to the minimum feature dimension threshold, then the minimum feature dimension threshold is selected as the stage feature extraction dimension. If the initial feature extraction dimension is greater than the minimum feature dimension threshold, then the initial feature extraction dimension is selected as the stage feature extraction dimension.

8. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, The step of converting the stage feature extraction dimension number into the final feature extraction dimension number by ensuring the dimension number includes: Compare the number of dimensions extracted for stage features with the number of dimensions for guarantee; If the number of feature extraction dimensions in a stage is less than the guaranteed number of dimensions, then the guaranteed number of dimensions will be selected as the final number of feature extraction dimensions. If the number of feature extraction dimensions in a stage is greater than or equal to the guaranteed number of dimensions, then the number of feature extraction dimensions in a stage will be selected as the final number of feature extraction dimensions.

9. The method for hierarchical feature recognition of heterogeneous data streams with computing resource awareness according to claim 1, characterized in that, After obtaining the feature priority queue corresponding to the task type, wherein the feature priority queue includes several task features arranged in descending order of importance, and selecting the top few task features corresponding to the final feature extraction dimension from the feature priority queue as target features to obtain the feature vector, the method further includes: The index difference between two adjacent comprehensive resource pressure indices is obtained, and the index difference is compared with the change tolerance threshold for closed-loop control.

10. A computational resource-aware heterogeneous data stream hierarchical feature recognition system, applied to the computational resource-aware heterogeneous data stream hierarchical feature recognition method as described in any one of claims 1 to 9, characterized in that, The system includes: The acquisition module is used to acquire a set of monitoring parameters based on a fixed sampling period, and to acquire a comprehensive resource pressure index based on the set of monitoring parameters. The association module is used to obtain the feature extraction depth coefficient through a nonlinear mapping function and a comprehensive resource pressure index, and to obtain the initial feature extraction dimension based on the feature extraction depth coefficient. The analysis module is used to convert the initial feature extraction dimension number into the stage feature extraction dimension number by using the minimum feature dimension threshold, obtain the guaranteed dimension number corresponding to the task type, and convert the stage feature extraction dimension number into the final feature extraction dimension number by using the guaranteed dimension number. The execution module is used to obtain a feature priority queue corresponding to the task type. The feature priority queue includes several task features arranged in descending order of importance. The first few task features corresponding to the final feature extraction dimension number are selected from the feature priority queue as target features to obtain feature vectors.

Images