Data resource allocation method and system, electronic equipment and storage medium

Abstract

The invention discloses a data resource allocation method and system, electronic equipment and a storage medium. The method comprises the steps of obtaining operation data of an Internet of Things system; inputting the operation data into a prediction model to obtain resource demand data; generating a prediction curve according to the resource demand data, and performing curve analysis on the prediction curve to obtain a resource demand point or a resource demand change condition; the prediction curve is a resource demand changing along with time, and the resource demand point is a prediction moment when resources need to be adjusted; and according to the resource demand point or the resource demand change condition, a resource scheduling strategy is set, and resources are dynamically allocated. The resource utilization rate is improved, and the system efficiency is improved. The method can be widely applied to the technical field of Internet of Things.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to a data resource allocation method, system, electronic device and storage medium. Background Art

[0002] The rapid development of Internet of Things (IoT) technology has led to a surge in the number of devices, accelerated data generation, and increasingly complex resource management. In order to ensure the stable operation and efficient service of the IoT system, resource management has become a key link. The application of artificial intelligence (AI) technology in the field of prediction provides a new perspective and method for IoT resource management. In the related technologies, in the field of IoT, it is impossible to obtain resource demand in real time, resulting in the resource scheduling process lagging behind the system, which in turn affects resource utilization. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to provide an efficient data resource allocation method, system, electronic device and storage medium.

[0004] To achieve the above-mentioned purpose, one aspect of an embodiment of the present application proposes a data resource allocation method, the method comprising: obtaining the operating data of the Internet of Things system; inputting the operating data into a prediction model to obtain resource demand data; generating a prediction curve based on the resource demand data, and performing curve analysis on the prediction curve to obtain resource demand points or changes in resource demand; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted moment when resources need to be adjusted; according to the resource demand point or the change in resource demand, a resource scheduling strategy is set to dynamically allocate resources. The embodiment of the present application can pay attention to the resource demand situation in real time by obtaining the operating data of the Internet of Things system in real time and predicting the resource demand data through a prediction model; at the same time, the embodiment of the present application adjusts the resource scheduling strategy according to the prediction curve to achieve dynamic allocation of resources. The present application is conducive to improving resource utilization and improving system efficiency.

[0005] In some embodiments, the method provided in the embodiments of the present application, wherein the curve analysis is performed on the prediction curve to obtain the resource demand point, including:

[0006] Determining a moving average of the prediction curve;

[0007] Determining the resource demand point according to the change of the moving average;

[0008] Alternatively, extracting a number of curve data points of the prediction curve;

[0009] Clustering is performed on a number of the curve data points to obtain the resource demand points.

[0010] In some embodiments, the method provided in the embodiments of the present application, setting a resource scheduling strategy according to the resource demand point or the resource demand change, and dynamically allocating resources includes:

[0011] According to the changes in resource demand, load balancing is performed to dynamically allocate resources;

[0012] According to the resource demand points, resource reservation is performed and resources are dynamically allocated;

[0013] According to the resource demand points or the resource demand changes, priority setting is performed to dynamically allocate resources.

[0014] In some embodiments, the method provided in the embodiments of the present application, performing load balancing and dynamically allocating resources according to the change in resource demand, includes:

[0015] If the resource demand change is an increase in demand, determine the load information of each resource;

[0016] The resources are sorted according to the order of the load information from large to small or from small to large, and the new task is allocated to the resource ranked first.

[0017] In some embodiments, in the method provided by the embodiments of the present application, the prediction model includes a random forest model, and the prediction model is trained by the following steps:

[0018] Acquire an original sample set, and perform feature extraction on sample data in the original sample set to obtain a feature set;

[0019] Extracting a random sample from the original sample set and selecting a feature subset from the feature set;

[0020] Recursively constructing a decision tree through the random sample and the feature subset until an end condition is met;

[0021] The decision tree is split and hyperparameters are tuned to obtain the trained prediction model.

[0022] In some embodiments, the method provided in the embodiments of the present application, wherein the obtaining of the operation data of the Internet of Things system includes:

[0023] Identify key data sources that need to be monitored in the IoT system;

[0024] The monitoring tool of the Internet of Things platform is used to obtain the operating data of the Internet of Things system; the operating data corresponds to the key data source.

[0025] In some embodiments, the method provided by the embodiments of the present application further includes:

[0026] Obtain the accumulated duration; the accumulated duration is the duration that has passed since the last update;

[0027] If the accumulated duration is greater than or equal to the preset duration, the resource scheduling strategy and the prediction model are updated, and the process returns to the step of obtaining the accumulated duration; the preset duration is used to characterize the frequency of updates.

[0028] To achieve the above object, another aspect of the embodiment of the present application provides a data resource allocation system, the system comprising:

[0029] The first module is used to obtain the operation data of the Internet of Things system;

[0030] The second module is used to input the operation data into a prediction model to obtain resource demand data;

[0031] The third module is used to generate a prediction curve according to the resource demand data, and perform curve analysis on the prediction curve to obtain a resource demand point or a change in resource demand; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted time when the resource needs to be adjusted;

[0032] The fourth module is used to set a resource scheduling strategy and dynamically allocate resources according to the resource demand points or the resource demand changes.

[0033] To achieve the above objective, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0034] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0035] The embodiments of the present application include at least the following beneficial effects: The method provided by the embodiments of the present application includes: obtaining the operating data of the Internet of Things system; inputting the operating data into the prediction model to obtain resource demand data; generating a prediction curve based on the resource demand data, and performing curve analysis on the prediction curve to obtain resource demand points or changes in resource demand; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted moment when the resource needs to be adjusted; according to the resource demand point or the change in resource demand, a resource scheduling strategy is set to dynamically allocate resources. The embodiments of the present application can pay attention to the resource demand situation in real time by obtaining the operating data of the Internet of Things system in real time and predicting the resource demand data through the prediction model; at the same time, the embodiments of the present application adjust the resource scheduling strategy according to the prediction curve to realize dynamic allocation of resources. The present application is conducive to improving resource utilization and improving system efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of an embodiment of the data resource allocation method provided by the present application;

[0037] Figure 2 is a flow chart of another embodiment of the data resource allocation method provided by the present application;

[0038] Figure 3 It is a flowchart of an embodiment of a process for determining resource demand points provided by the present application;

[0039] Figure 4 is a flow chart of an embodiment of the dynamic allocation process provided by the present application;

[0040] Figure 5 is a flow chart of another embodiment of the dynamic allocation process provided by the present application;

[0041] Figure 6 is a flowchart of an embodiment of the training process of the prediction model provided by the present application;

[0042] Figure 7 is a flowchart of another embodiment of a process for obtaining operation data provided by the present application;

[0043] Figure 8 is a flowchart of an embodiment of a process for updating models and measurements provided by the present application;

[0044] Fig. 9 It is a structural diagram of a data resource allocation system provided in an embodiment of the present application;

[0045] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0047] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0048] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0050] The rapid development of Internet of Things (IoT) technology has led to a surge in the number of devices, accelerated data generation, and increasingly complex resource management. In order to ensure the stable operation and efficient service of the IoT system, resource management has become a key link. The application of artificial intelligence (AI) technology in the field of prediction provides a new perspective and method for IoT resource management. This application proposes an IoT resource management method based on AI prediction, which aims to optimize resource allocation and improve system efficiency. This application belongs to data resource optimization technology, and particularly to the technical field of IoT data resource optimization.

[0051] The data resource allocation method provided in the embodiment of the present application relates to the field of Internet of Things technology. The data resource allocation method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application that implements the data resource allocation method, etc., but is not limited to the above forms.

[0052] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0053] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0054] Figure 1 This is an optional flow chart of the data resource allocation method provided in the embodiment of the present application; Figure 1The method may include but is not limited to steps S100 to S400.

[0055] Step S100, obtaining operation data of the Internet of Things system;

[0056] Step S200, inputting the operation data into the prediction model to obtain resource demand data;

[0057] Step S300, generating a prediction curve based on the resource demand data, and performing curve analysis on the prediction curve to obtain resource demand points or resource demand changes; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted time when the resource needs to be adjusted;

[0058] Step S400, setting a resource scheduling strategy according to resource demand points or resource demand changes, and dynamically allocating resources.

[0059] The operation data in the embodiment of the present application can be any data related to the operation of the Internet of Things system. The prediction model in the present application is used to predict future resource demand. The prediction curve is the resource demand based on time changes. The resource demand point can be a peak, an underestimate, or any point of concern. Figure 2 As shown, the present application provides an IoT data resource optimization method based on AI prediction, which aims to solve the challenges faced by resource management in existing IoT systems. The embodiments of the present application are conducive to improving resource utilization efficiency: through AI prediction technology, accurate prediction of IoT device resource requirements is achieved, thereby optimizing resource allocation, reducing resource idleness and waste, and improving overall resource utilization efficiency. The embodiments of the present application guarantee service quality: by dynamically adjusting resource allocation, it ensures that the IoT system can maintain stable operation during peak hours and emergencies, and improves service reliability and user experience. Reduce unnecessary resource consumption and operation and maintenance costs to maximize cost-effectiveness. The embodiments of the present application enhance system scalability: enable the IoT system to adaptively adjust resources and maintain system scalability as the number of devices increases and the amount of data grows. The embodiments of the present application promote technological innovation: combining AI technology with IoT resource management, promoting technological innovation in the field of IoT, and providing technical support for future intelligent systems and intelligent services.

[0060] In some embodiments, reference Figure 3 As shown, the method provided in the embodiment of the present application performs curve analysis on the prediction curve to obtain resource demand points, including:

[0061] Step S310, determining a moving average line of the prediction curve;

[0062] Determine resource demand points based on changes in the moving average;

[0063] Alternatively, step S320, extracting a number of curve data points of the prediction curve;

[0064] Clustering is performed on several curve data points to obtain resource demand points.

[0065] In some possible implementations, the resource demand points can be first determined by moving average, and then the obtained resource demand points can be verified by clustering; if the similarity of the resource demand points obtained by the two methods meets the preset similarity threshold, the resource demand points obtained by moving average are determined to be correct resource demand points. It can be understood that the moving average is a statistical analysis tool, which can help smooth data fluctuations and present the trend of resource demand more clearly when used for resource demand analysis.

[0066] The moving average of resource demand is calculated by averaging the resource demand within a specific period (such as the past 5 days, 10 days, etc.). For example, the 5-day moving average is the sum of the resource demand data of the past 5 days and then divided by 5. As time goes by, the data of the new day will be added every day, and the data of the earliest day will be removed to recalculate the average. Its main functions include predicting the trend of resource demand and providing a reference for resource procurement, configuration and other decisions; it can also filter out short-term fluctuations and allow users to focus on long-term demand changes. To calculate the moving average of a curve, first determine the curve data points of the curve, and then determine the moving average by averaging; it can also be determined by weighted averaging; it can also be determined by a smoothing coefficient, an exponential moving average. The present application does not limit the specific method for determining the moving average.

[0067] In some other embodiments, the peak of the curve may be identified by artificial intelligence methods, specifically:

[0068] Machine learning methods include supervised learning algorithms: collect a curve data set with peak marks, for example, mark the highest point that has been determined in the predicted curve. Then use feature extraction methods, such as extracting the slope, curvature, local maximum point, etc. of the curve as features. Select a suitable machine learning model, such as decision tree, support vector machine, etc. Taking decision tree as an example, it can build decision rules based on the extracted features and predict the peak position in the new curve by learning the feature differences between peak and non-peak data. Similarly, the detection process of underestimation is the same as the detection process of peak position mentioned above. Unsupervised learning algorithm: Clustering algorithm can cluster curve data points according to similarity. For example, using K-Means clustering, curve data points are clustered into different categories. The characteristics of the peak point are usually different from the surrounding data points, such as its value is higher than the adjacent points. These data point groups with peak characteristics can be separated by clustering. Of course, peaks can also be identified through high-density areas.

[0069] Deep learning methods can also be used, including: Convolutional Neural Network (CNN): treat curve data as a one-dimensional signal (if it is a two-dimensional curve, a two-dimensional CNN can be considered). For time series curves (such as resource demand over time curves), the data at each time point can be used as an input unit. CNN automatically extracts features from the curve through the convolution layer, such as local extreme points, slope changes, and other patterns. The pooling layer can compress and abstract the features, and finally the fully connected layer is used to determine which positions are peaks. Recurrent Neural Network (RNN) and its variants (LSTM, GRU): suitable for processing sequence data, such as curve data. Taking the long short-term memory network (LSTM) as an example, it can learn the time dependency of curve data. With the input of the data sequence, LSTM can remember the previous data features. When encountering a data point that may be a peak, it determines whether the point is a peak based on the previously learned peak pattern (such as the slope change around the previous peak point, etc.).

[0070] In some embodiments, reference Figure 4 As shown, the method provided in the embodiment of the present application sets a resource scheduling strategy according to resource demand points or resource demand changes, and dynamically allocates resources, including:

[0071] Step S410, performing load balancing and dynamically allocating resources according to changes in resource demand;

[0072] Step S420, executing resource reservation according to resource demand points and dynamically allocating resources;

[0073] Step S430, according to resource demand points or resource demand changes, priority setting is performed to dynamically allocate resources.

[0074] In some possible implementations, the resource demand point includes a maximum resource demand value, and the maximum resource demand value is used for overall resource expansion. The specific method of resource expansion is not specifically limited in this application.

[0075] In some embodiments, reference Figure 5 As shown, the method provided in the embodiment of the present application performs load balancing and dynamically allocates resources according to changes in resource requirements, including:

[0076] Step S411, if the resource demand change is an increase in demand, determine the load information of each resource;

[0077] Step S412: sort the resources in order of load information from large to small or from small to large, and assign the new task to the resource ranked first.

[0078] In some possible implementations, if the resource demand change is a decrease in demand, some loads of resources ranked higher (overloaded) are transferred to resources ranked lower. More specifically, if the resource utilization rate is lower than a preset utilization rate, some resources are recycled.

[0079] In some other embodiments, if the load of the first resource is greater than a load threshold, a resource instance is added to transfer the load of the first resource. The load threshold is adjusted according to historical data and prediction results.

[0080] In some embodiments, reference Figure 6 As shown, in the method provided in the embodiment of the present application, the prediction model includes a random forest model, and the prediction model is trained by the following steps:

[0081] Step S210, obtaining an original sample set, performing feature extraction on sample data in the original sample set, and obtaining a feature set;

[0082] Step S220, extracting a random sample from the original sample set, and selecting a feature subset from the feature set;

[0083] Step S230, recursively constructing a decision tree through random samples and feature subsets until an end condition is met;

[0084] Step S240, splitting the decision tree and optimizing hyperparameters to obtain a trained prediction model.

[0085] Of course, it is understandable that the prediction model in the embodiments of the present application may also be other artificial intelligence models, and the present application does not limit the specific type of the model.

[0086] In some embodiments, reference Figure 7 As shown, the method provided in the embodiment of the present application obtains the operation data of the Internet of Things system, including:

[0087] Step S110, determining the key data sources that need to be monitored in the Internet of Things system;

[0088] Step S120, using the monitoring tool of the Internet of Things platform to obtain the operating data of the Internet of Things system; the operating data corresponds to the key data source.

[0089] In some embodiments, reference Figure 8 As shown, the method provided in the embodiment of the present application further includes:

[0090] Step S500, obtaining the accumulated duration; the accumulated duration is the duration that has passed since the last update;

[0091] Step S600, if the accumulated duration is greater than or equal to the preset duration, update the resource scheduling strategy and prediction model, and return to the step of obtaining the accumulated duration; the preset duration is used to characterize the frequency of updates.

[0092] The following is a detailed description and explanation of the solution of the embodiment of the present invention in conjunction with a specific application example:

[0093] The solution mainly includes data collection and preprocessing, AI prediction model construction, prediction result analysis, and dynamic resource scheduling mechanism. The specific steps include the following:

[0094] 1. Data collection and preprocessing.

[0095] Data source identification: Determine the key data sources that need to be monitored in the IoT system, including device status, network traffic, storage usage, energy consumption data, etc. Use data collection methods such as the IoT platform's network monitoring tools, host monitoring tools, and Pulsar and other middleware throughput monitoring to collect real-time operating data of the IoT system. Data cleaning: Remove duplicate values, outliers, and irrelevant information from the collected data to ensure data quality. Feature extraction: Extract key features from the raw data that help predict resource requirements, and average the time series features such as the number of real-time online devices, network traffic, storage usage, and energy consumption data by time period. variance Skewness

[0096] Kurtosis Statistical feature extraction.

[0097] 2. Construction of AI prediction model.

[0098] The present invention adopts the prediction algorithm of random forest to construct the model.

[0099] 21. By pulling the time series features such as the number of real-time online devices, network traffic, storage usage, energy consumption data, etc. of the IoT platform in the past five years, the mean, variance, skewness, kurtosis and other features are calculated according to the time dimension and a certain interval time. 70% of the data is used for the training set, and the remaining 30% of the data is used for the test set. Random forests are constructed by aggregating multiple decision trees, each of which is trained on a random subset.

[0100] 22. Key steps in the training process: a. Sample sampling: For each tree, perform random sampling with replacement (Bootstrap sampling) from the original data set to obtain a sub-data set. b. Feature selection: At the node of each decision tree, randomly select a feature subset from all features (for example, sqrt(m) features, where m is the total number of features) for splitting. c. Decision tree construction: Recursively build a decision tree using the selected features and sample subsets until the stopping condition (such as node purity, depth limit, etc.) is met. The construction of a decision tree can be expressed as the following formula: Gini(D) = 1-sum(p^2for p in P) where Gini(D) is the impurity of node D, and P is the relative frequency of each category in node D. d. Tree splitting: Select features and splitting points that minimize the impurity of child nodes. The formula is as follows: Gini_index(D,A)=(|D1| / |D|)*Gini(D1)+(|D2| / |D|)*Gini(D2), where Gini_index(D,A) is the impurity of node D after splitting according to feature A, and D1 and D2 are the two child nodes after the split.

[0101] 23. Hyperparameter tuning: The hyperparameters of random forest include:

[0102] n_estimators: the number of decision trees. max_features: the number of features considered by each decision tree. max_depth: the maximum depth of a decision tree. min_samples_split: the minimum number of samples required for internal node repartitioning. min_samples_leaf: the minimum number of samples for leaf nodes.

[0103] The process of hyperparameter tuning can use the following formula to evaluate the performance of different hyperparameter combinations: MSE = 1 / n*sum((y_true-y_pred)^2), where MSE is the mean square error, y_true is the true value, and y_pred is the predicted value.

[0104] 24. Model evaluation.

[0105] The performance of the model was evaluated on the test set using the following metrics: Accuracy, Mean Square Error (MSE), Decision Coefficient (R 2 ):

[0106] Accuracy=(TP+TN) / (TP+TN+FP+FN);

[0107] R 2 =1-(SS_res / SS_tot);

[0108] Among them, TP is a true positive, TN is a true negative, FP is a false positive, FN is a false negative, SS_res is the residual sum of squares, and SS_tot is the total sum of squares. Through these steps, the random forest model can play an important role in the optimization of IoT data resources, providing accurate predictions and effective resource management suggestions.

[0109] 3. Analysis of prediction results.

[0110] 31. Forecast curve generation: Generate subsequent forecast curves based on the results of the AI ​​forecast model and dynamically update them over time. By observing and identifying the forecast curve, identify the rising and falling areas of resource demand, and use the forecast curve to estimate the maximum value of resource demand for overall resource expansion.

[0111] 32. Identification scheme for peaks and troughs.

[0112] The moving average + machine learning method is used as a hybrid: the moving average of resource requirements is calculated, and the peaks and valleys are identified through the changes in the moving average. The time series data points are clustered using an unsupervised learning algorithm (K-means), and high-density areas may be peaks, and low-density areas may be valleys for secondary proofreading.

[0113] 4. Dynamic resource scheduling mechanism.

[0114] 41. Scheduling strategy design: Design scheduling strategies based on resource requirements predicted by AI models, including resource reservation, load balancing, and dynamic expansion. Priority rules: Establish priority rules to ensure that key businesses are allocated first when resources are tight.

[0115] 42. Scheduling algorithm implementation: When the resource prediction curve rises, enable the minimum load priority scheme and calculate the load: For each resource, including the host, calculate its current load, which can be CPU usage, memory usage, or middleware usage indicators such as redis, pulsar, and flink. Sort the resources according to the calculated load. Prioritize assigning new tasks to the first (least loaded) resource after sorting. At the same time, set thresholds for existing resources with excessive loads, dynamically adjust thresholds based on historical data and prediction results, and automatically expand on demand through trigger time, automatically increasing resource instances such as virtual machines, containers, or servers. At the same time, when the prediction curve drops, transfer the load from overloaded resources to idle or low-load resources. When resource utilization is below a certain level, recycle excess resources to reduce operating costs.

[0116] In addition to real-time dynamic adjustments, the system's basic resource configuration will also be expanded from the overall resource level based on business development and system bottlenecks observed in the prediction curve.

[0117] 43. Scheduling strategy update: Update the scheduling rules based on real-time data and feedback. And regularly retrain the prediction model with the latest data to maintain prediction accuracy.

[0118] Exemplarily, the telecom IoT platform is a large IoT service provider that connects millions of devices and provides device management, data collection, analysis and application services. As the number of devices increases, the platform faces the challenge of optimizing resource management. Therefore, the data resource allocation method provided by this application is used for resource management. Specifically, 1. Data collection and preprocessing: On the telecom IoT platform, we identified the following key data sources: the number of real-time online devices, network traffic, storage usage and energy consumption data. The above data is collected in real time using the API and data acquisition tools provided by the platform. Using the data preprocessing module, the data is cleaned to remove outliers and noise to ensure data quality. The collected time series data is statistically extracted by time period, including mean, variance, skewness, kurtosis, etc. 2. AI prediction model construction: Extract relevant data from the platform historical database for the past five years, calculate statistical features by time interval, and divide the data into 70% training set and 30% test set. The random forest algorithm is used, and the training set data is used to build a prediction model. During the training process, sample sampling, feature selection, decision tree construction and hyperparameter tuning are performed. Hyperparameter tuning: Determine the optimal hyperparameter combination, number of decision trees, number of features, tree depth, etc. through cross-validation and mean square error (MSE) evaluation. 3. Forecast result analysis: Generate prediction curve: Generate prediction curve on the test set using the trained model and update it in real time to reflect the latest resource demand trend. Analyze the prediction curve through moving average and unsupervised learning algorithms to identify the peaks and valleys of resource demand. 4. Dynamic resource scheduling mechanism: Based on the prediction results, design resource reservation, load balancing and dynamic expansion strategies to reserve additional computing and storage resources for peak hours. By implementing the minimum load priority algorithm, calculate the current load of each resource and assign tasks according to the load situation. When an increase in resource demand is predicted, the resource expansion process is automatically triggered; when demand decreases, resources are automatically recovered. Based on actual operation data and user feedback, the scheduling strategy is regularly updated and the prediction model is retrained to maintain the optimal operation state of the system.

[0119] By implementing the present invention on the telecommunications Internet of Things platform, the platform can more accurately predict resource requirements and achieve dynamic scheduling and optimized management of resources, which not only improves the operating efficiency of the platform, but also reduces operating costs and improves user satisfaction.

[0120] This application uses the random forest algorithm to build a prediction model for resource prediction based on the time series characteristics of the number of real-time online users, network traffic, storage usage, energy consumption data, etc. in the IoT system. This application designs a dynamic resource scheduling mechanism based on the prediction results, and achieves optimal resource allocation through dynamic resource adjustment, load balancing, and resource reservation, and dynamically self-updates the idea.

[0121] Possible application scenarios of this application include: data resource operation and maintenance of the Internet of Things platform system; and data resource optimization solutions for other similar platform systems.

[0122] The embodiments of the present application can enhance the stability of the Internet of Things platform and increase customer stickiness by 100%. It combines AI technology with Internet of Things resource management, promotes technological innovation in the field of Internet of Things, and provides technical support for future intelligent systems and intelligent services.

[0123] See also Fig. 9 The embodiment of the present application further provides a data resource allocation system, which can implement the above data resource allocation method, and the system includes:

[0124] The first module 810 is used to obtain the operation data of the Internet of Things system;

[0125] The second module 820 is used to input the operation data into the prediction model to obtain resource demand data;

[0126] The third module 830 is used to generate a prediction curve according to the resource demand data, and perform curve analysis on the prediction curve to obtain resource demand points or resource demand changes; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted time when the resource needs to be adjusted;

[0127] The fourth module 840 is used to set a resource scheduling strategy and dynamically allocate resources according to resource demand points or changes in resource demand.

[0128] In some embodiments, the system provided by the embodiments of the present application further includes a sixth module for:

[0129] Get the cumulative duration; the cumulative duration is the duration that has passed since the last update;

[0130] If the accumulated duration is greater than or equal to the preset duration, the resource scheduling strategy and prediction model are updated, and the step of obtaining the accumulated duration is returned; the preset duration is used to characterize the frequency of updates.

[0131] In some embodiments, the system provided by the embodiments of the present application further includes a seventh module for:

[0132] Obtain an original sample set, perform feature extraction on sample data in the original sample set, and obtain a feature set;

[0133] Draw a random sample from the original sample set and select a feature subset from the feature set;

[0134] Recursively build a decision tree through random samples and feature subsets until the end condition is met;

[0135] The decision tree is split and the trained prediction model is obtained through hyperparameter tuning.

[0136] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0137] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above data resource allocation method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0138] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0139] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0140] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0141] The memory 902 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 may store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the data resource allocation method of the embodiment of this application;

[0142] Input / output interface 903, used to implement information input and output;

[0143] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0144] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0145] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0146] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned data resource allocation method is implemented.

[0147] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0148] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0150] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0151] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0153] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0154] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0156] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0159] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A data resource allocation method, characterized in that: The method comprises: Obtain the operating data of the IoT system; Inputting the operation data into a prediction model to obtain resource demand data; Generate a prediction curve based on the resource demand data, and perform curve analysis on the prediction curve to obtain resource demand points or resource demand changes; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted time when the resource needs to be adjusted; According to the resource demand points or the resource demand changes, a resource scheduling strategy is set to dynamically allocate resources.

2. The method according to claim 1, characterized in that The performing curve analysis on the prediction curve to obtain resource demand points includes: Determining a moving average of the prediction curve; Determining the resource demand point according to the change of the moving average; Alternatively, extracting a number of curve data points of the prediction curve; Clustering is performed on a number of the curve data points to obtain the resource demand points.

3. The method according to claim 1, characterized in that: The step of setting a resource scheduling strategy according to the resource demand point or the resource demand change to dynamically allocate resources includes: According to the changes in resource demand, load balancing is performed to dynamically allocate resources; According to the resource demand points, resource reservation is performed and resources are dynamically allocated; According to the resource demand points or the resource demand changes, priority setting is performed to dynamically allocate resources.

4. The method according to claim 3, characterized in that The performing load balancing and dynamically allocating resources according to the change in resource demand includes: If the resource demand change is an increase in demand, determine the load information of each resource; The resources are sorted according to the order of the load information from large to small or from small to large, and the new task is allocated to the resource ranked first.

5. The method according to claim 1, characterized in that: The prediction model includes a random forest model, and the prediction model is trained by the following steps: Acquire an original sample set, and perform feature extraction on sample data in the original sample set to obtain a feature set; Extracting a random sample from the original sample set and selecting a feature subset from the feature set; Recursively constructing a decision tree through the random sample and the feature subset until an end condition is met; The decision tree is split and hyperparameters are tuned to obtain the trained prediction model.

6. The method according to claim 1, characterized in that The obtaining of the operation data of the Internet of Things system includes: Identify key data sources that need to be monitored in the IoT system; The monitoring tool of the Internet of Things platform is used to obtain the operating data of the Internet of Things system; the operating data corresponds to the key data source.

7. The method according to claim 1, characterized in that The method further comprises: Obtain the accumulated duration; the accumulated duration is the duration that has passed since the last update; If the accumulated duration is greater than or equal to the preset duration, the resource scheduling strategy and the prediction model are updated, and the process returns to the step of obtaining the accumulated duration; the preset duration is used to characterize the frequency of updates.

8. A data resource allocation system, characterized in that: The system comprises: The first module is used to obtain the operation data of the Internet of Things system; The second module is used to input the operation data into a prediction model to obtain resource demand data; The third module is used to generate a prediction curve according to the resource demand data, and perform curve analysis on the prediction curve to obtain a resource demand point or a change in resource demand; the prediction curve is the resource demand that changes over time, and the resource demand point is the predicted time when the resource needs to be adjusted; The fourth module is used to set a resource scheduling strategy and dynamically allocate resources according to the resource demand points or the resource demand changes.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images