Distributed Task Processing Method, Device and Storage Medium Based on Spatiotemporal Data Complexity

Abstract

The present application discloses a distributed task processing method, device, and storage medium based on the complexity of spatio-temporal data, relating to the technical field of data processing, including: extracting a feature vector of the spatio-temporal data of a digital twin model and inputting the feature vector into a trained random forest model; through the trained random forest model, classifying and predicting the spatio-temporal data according to the feature vector, and outputting a class label corresponding to the feature vector, where the class label represents the complexity of the spatio-temporal data; mapping the class label of the spatio-temporal data into a weight coefficient, and formulating a distributed task queue based on the weight coefficient, so that the difference in the total weights of the task queues of each processing node is minimized to ensure load balancing between the distributed task queues; and distributing the distributed task queue to each cluster node for distributed processing. The present application achieves the technical effect of improving the efficiency of distributed task processing.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a distributed task processing method, device, and storage medium based on the complexity of spatio-temporal data. Background Art

[0002] Currently, in the construction of a digital twin city system platform, a large amount of urban spatio-temporal data needs to be processed. To achieve efficient data processing, a distributed task processing method is adopted. However, the current distributed computing system usually adopts a random task allocation mechanism, which does not consider the complexity of the data, resulting in unbalanced loads between different cluster nodes. The cluster nodes assigned simple tasks become idle after completing the tasks in advance, while the cluster nodes assigned complex tasks need to perform load operations for a long time, thereby reducing the overall data processing efficiency.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a distributed task processing method, device, and storage medium based on the complexity of spatio-temporal data, aiming to solve the technical problem of how to improve the efficiency of distributed task processing.

[0005] To achieve the above purpose, this application proposes a distributed task processing method based on the complexity of spatio-temporal data. The distributed task processing method based on the complexity of spatio-temporal data includes:

[0006] Extract the feature vector of the spatio-temporal data of the digital twin model, and input the feature vector into the trained random forest model;

[0007] Through the trained random forest model, classify and predict the spatio-temporal data according to the feature vector, and output the class label corresponding to the feature vector. The class label represents the complexity of the spatio-temporal data;

[0008] Map the class label of the spatio-temporal data into a weight coefficient, and formulate a distributed task queue based on the weight coefficient, so that the difference in the total weights of the task queues of each processing node is minimized to ensure load balancing between the distributed task queues;

[0009] Distribute the distributed task queue to each cluster node for distributed processing.

[0010] In one embodiment, the step of classifying and predicting the spatio-temporal data according to the feature vector by the trained random forest model and outputting the class label corresponding to the feature vector, where the class label represents the complexity of the spatio-temporal data, includes:

[0011] Input the input feature vector into each decision tree in the random forest model respectively;

[0012] Through each decision tree, classify and predict the input feature vector, and output the prediction results of each decision tree;

[0013] Statistically analyze the prediction results of each decision tree, and use the voting method to obtain the final classification prediction result, and output the corresponding class label.

[0014] In one embodiment, the step of mapping the class label of the spatio-temporal data into a weight coefficient and formulating a distributed task queue based on the weight coefficient to minimize the difference in the total weight of the task queues of each processing node to ensure load balancing among the distributed task queues includes:

[0015] According to the mapping rule between the class label and the weight coefficient, determine the weight coefficient corresponding to the spatio-temporal data;

[0016] Obtain the distributed task queues corresponding to each cluster node, calculate the total weight coefficient of each distributed task queue, and use the queue with the smallest difference in the total weight coefficient as the target task queue;

[0017] According to the task allocation strategy, allocate the spatio-temporal data to the target task queue.

[0018] In one embodiment, the step of determining the weight coefficient corresponding to the spatio-temporal data according to the mapping rule between the class label and the weight coefficient, the mapping rule includes:

[0019] Arrange the class labels in descending order of complexity, and establish a linear mapping relationship function; or

[0020] Allocate weight coefficients to each class label based on the complexity, and create a mapping table of the class label and the corresponding weight coefficient.

[0021] In one embodiment, before the step of extracting the feature vector of the spatio-temporal data of the digital twin model and inputting the feature vector into the trained random forest model, it includes:

[0022] Extract a preset number of sample data from the spatio-temporal dataset, and calculate the feature values of each sample data;

[0023] According to the range of the eigenvalue, corresponding class labels are assigned to each of the sample data, and all the labeled sample data are combined to form a labeled data set;

[0024] Based on the labeled data set, iterative training is performed on the initial random forest model to obtain a trained random forest model.

[0025] In one embodiment, the step of performing iterative training on the initial random forest model based on the labeled data set to obtain a trained random forest model includes:

[0026] Preprocess the labeled data set, and divide the preprocessed labeled data set into a training data set and a test data set;

[0027] Use the training data set to perform iterative training on the initial random forest model a predetermined number of times to obtain a well-fitted random forest model;

[0028] Perform cross-validation on the well-fitted random forest model and perform testing according to the test data set to obtain the trained random forest model.

[0029] In one embodiment, the step of preprocessing the labeled data set and dividing the preprocessed labeled data set into a training data set and a test data set includes:

[0030] Standardize the eigenvalue of the sample data in the labeled data set, and form a feature matrix according to the order of the sample data;

[0031] Organize the class labels of the sample data in the labeled data set into a label matrix, which is matched with the order of the sample data in the feature matrix;

[0032] Divide the preprocessed labeled data set into the training data set and the test data set according to a preset ratio.

[0033] In one embodiment, the step of performing cross-validation on the well-fitted random forest model and performing testing according to the test data set to obtain the trained random forest model includes:

[0034] Perform cross-validation on the well-fitted random forest model a predetermined number of times, and record the average accuracy rate of each cross-validation;

[0035] If the average accuracy rate reaches the accuracy threshold, use the test data set to perform prediction evaluation on the initial random forest model and calculate the test accuracy rate;

[0036] If the test accuracy rate reaches the accuracy rate threshold, determine the trained random forest model.

[0037] In addition, to achieve the above object, the present application also provides a distributed task processing device based on spatio-temporal data complexity. The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. The computer program is configured to implement the steps of the distributed task processing method based on spatio-temporal data complexity as described above.

[0038] In addition, to achieve the above object, the present application also provides a storage medium. The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the distributed task processing method based on spatio-temporal data complexity as described above.

[0039] The present application provides a distributed task processing method based on spatio-temporal data complexity. The present application extracts the feature vectors of the spatio-temporal data of the digital twin model and inputs the feature vectors into the trained random forest model; through the trained random forest model, classifies and predicts the spatio-temporal data according to the feature vectors, and outputs the class label corresponding to the feature vectors. The class label represents the complexity of the spatio-temporal data; maps the class label of the spatio-temporal data into a weight coefficient, and formulates a distributed task queue based on the weight coefficient to ensure load balancing between the distributed task queues; distributes the distributed task queue to each cluster node for distributed processing. The present application extracts feature vectors, inputs the feature vectors affecting data complexity into the random forest model, and improves the classification accuracy by constructing multiple decision trees and synthesizing their prediction results. By mapping the class label into a weight coefficient, a quantitative standard for task allocation is provided. By ensuring that the sum of the weight coefficients of each queue is similar, the workload of each cluster node is balanced, and the task processing efficiency is improved. The present application achieves the technical effect of improving the distributed task processing efficiency. Description of the Drawings

[0040] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0041] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1Schematic flowchart provided for the first embodiment of the distributed task processing method based on spatio-temporal data complexity in this application;

[0043] Figure 2 Schematic flowchart provided for the second embodiment of the distributed task processing method based on spatio-temporal data complexity in this application;

[0044] Figure 3 Schematic flowchart provided for the third embodiment of the distributed task processing method based on spatio-temporal data complexity in this application;

[0045] Figure 4 Schematic flowchart provided for the fourth embodiment of the distributed task processing method based on spatio-temporal data complexity in this application;

[0046] Figure 5 Overall schematic flowchart provided for the distributed task processing method based on spatio-temporal data complexity in this application;

[0047] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the distributed task processing method based on spatio-temporal data complexity in the embodiments of this application.

[0048] The realization of the purpose, functional features, and advantages of this application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0050] To better understand the technical solutions of this application, the following will be described in detail with reference to the accompanying drawings of the specification and the specific implementation manners.

[0051] The main solution of the embodiments of this application is:

[0052] Currently, the distributed computing system of digital twin cities usually adopts a task allocation mechanism of random allocation. However, this mechanism does not consider the complexity of data, resulting in unbalanced loads among different cluster nodes allocated. The cluster nodes allocated with simple tasks become idle after completing the tasks in advance, while the cluster nodes allocated with complex tasks need to perform load operations for a long time, thereby leading to low efficiency of overall data processing.

[0053] This application extracts feature vectors, inputs the feature vectors affecting data complexity into a random forest model, and improves the accuracy of classification by constructing multiple decision trees and synthesizing their prediction results. By mapping the class labels to weight coefficients, a quantitative standard for task allocation is provided. By ensuring that the sum of the weight coefficients of each queue is similar, the workloads of each cluster node are balanced, and the efficiency of task processing is improved.

[0054] It should be noted that the execution subject of this embodiment can be a distributed task processing system based on the spatio-temporal data complexity, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a control device of a distributed task processing system based on the spatio-temporal data complexity that can implement the above functions. This embodiment does not make specific limitations in this regard. Hereinafter, taking the distributed task processing system based on the spatio-temporal data complexity as the execution subject as an example, this embodiment and the following embodiments will be described.

[0055] Embodiment 1

[0056] Based on this, the present application proposes a distributed task processing method based on spatio-temporal data complexity in the first embodiment. Please refer to Figure 1 , the distributed task processing method based on spatio-temporal data complexity includes:

[0057] Step S10, extract the feature vectors of the spatio-temporal data of the digital twin model, and input the feature vectors into the trained random forest model.

[0058] By extracting the feature vectors, complex spatio-temporal data can be simplified into a series of key indicators reflecting the essential characteristics of the data, and the trained random forest model is used to classify these feature vectors to identify the complexity of the spatio-temporal data.

[0059] In this embodiment, the spatio-temporal data is data used to describe the time and space information of the digital twin model in the digital twin city system platform. The feature vectors are a set of numerical values extracted from the spatio-temporal data, used to evaluate the complexity of the spatio-temporal data of the digital twin model, including the number of triangular patches, curvature, surface area to volume ratio, texture complexity, etc. The number of triangular patches is an index used to describe the surface fineness of buildings or other objects in 3D modeling. The more the number of triangular patches, the finer the model and the higher the data complexity. Curvature is a numerical index describing the degree of bending of the surface of an object in the model, used to evaluate the geometric shape complexity of a building. The surface area to volume ratio is the ratio of the surface area of an object to its volume in the model, used to evaluate the compactness of the object. Texture complexity is used to describe the richness of texture details on the surface of an object.

[0060] In this embodiment, the random forest model is an ensemble learning algorithm that improves the accuracy and stability of classification by constructing multiple decision trees. Each decision tree is constructed based on part of the training data and features, and finally the prediction results of all decision trees are integrated by voting or averaging, etc.

[0061] As an alternative implementation, spatio-temporal data is collected from the digital twin city system platform, and the number of triangular patches, curvature, surface area to volume ratio, and texture complexity of the spatio-temporal data are extracted to form a feature vector.

[0062] Exemplarily, the number of triangular patches in the data is counted through geometric calculations or data structure analysis; the curvature of the data is calculated using geometric algorithms, and the Gaussian curvature or mean curvature is used to characterize the degree of bending of the data; the surface area of the data is obtained through geometric calculations, the volume of the data is calculated through integral or volume formulas, and the ratio of the surface area to the volume is calculated; image processing techniques are used to extract the texture features of the data and calculate the texture complexity.

[0063] In step S20, through the trained random forest model, the spatio-temporal data is classified and predicted according to the feature vector, and the class label corresponding to the feature vector is output, where the class label represents the complexity of the spatio-temporal data.

[0064] Using the already trained random forest model, according to the feature vector extracted from the spatio-temporal data, the spatio-temporal data is classified and predicted. The complexity of the spatio-temporal data represented by the feature vector is identified through the model, and the corresponding class label is output. The class label is a quantitative representation of the complexity of the spatio-temporal data, which helps to optimize the digital twin model through the spatio-temporal data subsequently.

[0065] In this embodiment, the random forest model is an ensemble learning method that improves the accuracy of classification by constructing multiple decision trees and integrating their results. The class label is the result output after classifying and predicting the spatio-temporal data, representing the complexity category of the spatio-temporal data, which can include three categories: "simple", "general", and "complex".

[0066] As an alternative implementation, the random forest model classifies and predicts according to the value of the feature vector through the branch structure of the decision tree, and outputs the class label corresponding to each feature vector, representing the complexity category of the spatio-temporal data.

[0067] Exemplarily, the random forest model comprehensively predicts through multiple decision trees and outputs the class label corresponding to each feature vector through methods such as voting or averaging.

[0068] Optionally, the random forest model is a machine learning model that uses the random forest algorithm as the core algorithm, and other machine learning algorithms such as support vector machines and gradient boosting trees can also be used.

[0069] In step S30, the class label of the spatio-temporal data is mapped into a weight coefficient, and a distributed task queue is formulated based on the weight coefficient to minimize the difference in the total weight of the task queues of each processing node, so as to ensure the load balance between the distributed task queues.

[0070] By mapping category labels to weight coefficients and allocating tasks based on these weights, the use of distributed computing resources can be optimized, computing efficiency can be improved, the situation where some nodes are overloaded while others are idle can be avoided, thereby enhancing the performance and stability of the entire distributed system.

[0071] In this embodiment, the weight coefficient is a numerical value assigned according to the category label, used to quantify the computing resource requirements of spatio-temporal data. The higher the weight coefficient, the higher the complexity of the data and the more computing resources are required. The distributed task queue is a queue for distributing tasks to different computing nodes, and each task queue corresponds to a computing node.

[0072] Optionally, set the weight coefficient according to the complexity of the category label.

[0073] Exemplarily, weight coefficients of "1", "2", and "4" are respectively assigned to the three categories of "simple", "general", and "complex".

[0074] As an alternative implementation, real-time monitor the load conditions of each cluster node, calculate the total sum of the weight coefficients of each distributed task queue, use a predefined mapping rule to convert the category label of the spatio-temporal data into a weight coefficient, and allocate the spatio-temporal data to the task queue with the smallest difference in the total sum of the weight coefficients.

[0075] As another alternative implementation, count the spatio-temporal data of the digital twin model that needs to be distributed for processing within a period of time, convert the category label of each spatio-temporal data into its corresponding weight coefficient, initialize a certain number of distributed task queues according to the number and computing power of the cluster nodes, start allocating from the spatio-temporal data with a larger weight coefficient, preferentially place it in the task queue with a smaller current total sum of the weight coefficients, real-time monitor the total sum of the weight coefficients of the task queue, and dynamically adjust the task allocation to ensure that the total sum of the weight coefficients of each task queue is equal.

[0076] Step S40, distribute the distributed task queue to each cluster node for distributed processing.

[0077] By distributing the task queue to multiple cluster nodes for parallel processing, the computing resources of the cluster can be fully utilized, the processing time can be reduced, and the response speed of the overall system can be improved.

[0078] In this embodiment, the task queue is a set containing multiple tasks, and each task is a spatio-temporal data item to be processed. The task queue is the basic unit for task allocation and scheduling. A cluster node is a computing unit in a distributed system, responsible for processing the tasks in the task queue assigned to it. A cluster node can be a physical server, a virtual machine, a container, etc. Distributed processing is a computing mode that splits large computing tasks into multiple smaller subtasks and distributes these subtasks to multiple computing nodes for parallel execution. Distributed processing can significantly improve computing efficiency.

[0079] As an alternative implementation, the task queue is sent to each cluster node through a communication protocol. After each node receives the task queue, it sequentially executes the tasks according to the order of the tasks in the task queue.

[0080] Optionally, the CPU usage rate, memory usage rate, disk I / O and other load metrics of each node are monitored in real time. If it is found that the load of a certain node is too high, some tasks can be removed from the task queue of this node and distributed to the task queue of a node with a lower load.

[0081] This embodiment provides a distributed task processing method based on spatio-temporal data complexity. In this embodiment, first, by extracting feature vectors, the feature vectors affecting data complexity are input into a random forest model. By constructing multiple decision trees and synthesizing their prediction results, the accuracy of classification is improved. By mapping the class labels to weight coefficients, a quantitative standard for task allocation is provided. By ensuring that the sum of the weight coefficients of each queue is similar, the workload of each cluster node is balanced and the efficiency of task processing is improved.

[0082] Based on Embodiment 1, Embodiment 2 of this application proposes a distributed task processing method based on spatio-temporal data complexity. Referring to Figure 2 , step S20 includes:

[0083] Step S21, input the input feature vectors into each decision tree in the random forest model respectively.

[0084] The input feature vectors are respectively passed to each decision tree in the random forest model so that each decision tree can independently perform classification prediction.

[0085] It should be noted that a decision tree is a single classifier in a random forest model. It splits through feature values, constructs a decision path, and finally outputs a class label.

[0086] Exemplarily, the feature vectors are respectively input into each decision tree in the random forest model.

[0087] Step S22: Through each of the decision trees, classify and predict the input feature vector, and output the prediction results of each decision tree.

[0088] It should be noted that the prediction result is the class label output after each decision tree classifies and predicts the input feature vector.

[0089] Exemplarily, each decision tree classifies and predicts the input feature vector according to its internal decision rules. In each decision tree, the feature vector traverses along the nodes of the tree until it reaches a leaf node. The leaf node represents a class label, that is, the prediction result of the decision tree for the feature vector.

[0090] Step S23: Statistically analyze the prediction results of each decision tree, and use the voting method to obtain the final classification prediction result, and output the corresponding class label.

[0091] By statistically analyzing the prediction results of all decision trees and using the voting method to synthesize the prediction results of all decision trees, the final classification prediction result is obtained.

[0092] It should be noted that the voting method is to statistically analyze the prediction results of all decision trees and select the class label with the most occurrences as the final prediction result.

[0093] Exemplarily, statistically analyze the prediction results of all decision trees, calculate the number of occurrences of each class label, and select the class label with the most occurrences as the final prediction result.

[0094] This embodiment provides a distributed task processing method based on spatio-temporal data complexity. In this embodiment, first, the feature vector is input into each decision tree in the random forest. Each decision tree independently classifies and predicts the input feature vector. Finally, the prediction results of all decision trees are synthesized by the voting method to obtain the final classification prediction result, which improves the reliability and accuracy of the prediction.

[0095] Based on Embodiment 1, Embodiment 3 of this application proposes a distributed task processing method based on spatio-temporal data complexity. Referring to Figure 3 , step S30 includes:

[0096] Step S31: Determine the weight coefficient corresponding to the spatio-temporal data according to the mapping rule between the class label and the weight coefficient.

[0097] According to the mapping rule between the class label and the weight coefficient, a weight coefficient is assigned to each spatio-temporal data, so as to consider the complexity of the data during task allocation and ensure load balancing.

[0098] It should be noted that the weight coefficient is a value assigned according to the category label and is used to quantify the computing resource requirements of each spatio-temporal data. The higher the weight coefficient, the higher the complexity of the data and the more computing resources are required. The mapping rule is the rule for converting the category label into the weight coefficient.

[0099] Exemplarily, according to the complexity of the category label, the corresponding weight coefficient is determined through the mapping rule, and the category label of each spatio-temporal data is converted into the corresponding weight coefficient. According to the mapping rule, the weight coefficients of 1, 2, and 4 are respectively assigned to the three categories of "simple", "general", and "complex".

[0100] Optionally, according to the business requirements and data characteristics, a set of category labels and corresponding weight coefficients are defined. The data is divided into categories such as "simple", "general", and "complex", and a weight coefficient is assigned to each category. According to the category label and the weight coefficient, a mapping rule is established.

[0101] Optionally, the mapping rule includes:

[0102] Step B10, arrange the category labels in descending order of complexity, and establish a linear mapping relationship function.

[0103] It should be noted that the mapping relationship function is a function used to map the complexity of the category label to the weight coefficient and ensure that the higher the complexity, the larger the weight coefficient.

[0104] Exemplarily, according to the business requirements or data characteristics, determine the complexity order of each category label, and define a linear function to map the complexity to the weight coefficient.

[0105] Step B20, assign the weight coefficient to each category label based on the complexity, and create a mapping table of the category label and the corresponding weight coefficient.

[0106] It should be noted that the mapping table is a table that records the mapping relationship between the category label and the corresponding weight coefficient, which is convenient for quickly looking up and updating the category label and its weight coefficient.

[0107] Exemplarily, set a weight assignment rule that the higher the complexity score of the category label, the larger the weight coefficient assigned, and store each category label and its corresponding weight coefficient in a mapping table.

[0108] Step S32, obtain the distributed task queues corresponding to each cluster node, calculate the sum of the weight coefficients of each distributed task queue, and use the queue with the smallest difference in the sum of the weight coefficients as the target task queue.

[0109] It should be noted that a cluster node is a computing unit in a distributed system, responsible for processing tasks in the task queue assigned to it. A cluster node can be a physical server, a virtual machine, or a container. A distributed task queue is a collection of tasks that distributes tasks to multiple computing nodes for parallel processing, and each task queue contains a set of tasks.

[0110] Exemplarily, obtain the current task queue of each cluster node, calculate the total weight coefficient of each task queue, and select the task queue with the smallest total weight coefficient as the target queue.

[0111] Optionally, for cluster nodes without a defined task queue, initialize a certain number of distributed task queues according to the number and computing power of the cluster nodes to ensure that each node has one or more task queues, thereby achieving parallel processing of tasks.

[0112] Step S33, according to the task allocation strategy, allocate the spatio-temporal data to the target task queue.

[0113] Through a reasonable task allocation strategy, it can be ensured that the workloads among task queues are relatively balanced, avoiding the situation where some queues are overloaded while others are idle, maximizing the resource utilization rate of the system, and improving the processing efficiency.

[0114] It should be noted that the task allocation strategy is an algorithm or rule for allocating spatio-temporal data to different task queues, including the round-robin method, the least number of tasks method, the least total weight coefficient method, etc.

[0115] Optionally, if the difference between the total weight coefficients of each task queue is greater than a preset difference threshold, move some tasks from the task queue with a larger total weight coefficient to the task queue with a smaller total weight coefficient, and recalculate the total weight coefficient of each task queue to ensure that the difference is less than the preset difference threshold.

[0116] This embodiment provides a distributed task processing method based on the complexity of spatio-temporal data. In this embodiment, first, according to the number and computing power of cluster nodes, a corresponding number of distributed task queues are dynamically initialized, improving the flexibility of the system. According to the mapping rule of category labels and weight coefficients, the weight coefficient of each piece of spatio-temporal data is determined, improving the accuracy and efficiency of data processing.

[0117] Based on Embodiment 1, Embodiment 4 of this application proposes a distributed task processing method based on the complexity of spatio-temporal data. Referring to Figure 4 , before step S10, it includes:

[0118] Step S50, extract a preset number of sample data from the spatio-temporal data set and calculate the eigenvalue of each sample data.

[0119] Selecting a certain number of representative sample data from a large amount of spatio-temporal data sets and calculating the eigenvalues of these sample data helps the algorithm to more efficiently understand and process spatio-temporal data, extract key information, and improve the efficiency of model training in the subsequent model training process. By sampling and calculating eigenvalues, the complexity of data processing can be reduced while retaining the main features and trends of the data.

[0120] In this embodiment, the spatio-temporal data set is a data set containing time and space information, used to describe the time and space characteristics of the digital twin model, including geographical coordinates, timestamps, sensor readings, etc. The sample data is a part of the data randomly selected from the spatio-temporal data set for model training and verification. The sample data is representative and can reflect the characteristics of the entire data set. The eigenvalue is a set of numerical values extracted from the sample data, used to characterize the properties of the data. The eigenvalue is the input of the random forest model and can help the model understand and classify the data. The eigenvalues include the number of triangular patches, curvature, surface area to volume ratio, texture complexity, etc.

[0121] As an alternative implementation, a preset number of sample data are randomly selected from the spatio-temporal data set. For the extracted sample data, the geographical location coordinates, timestamps, and other relevant attribute data of the sample are extracted, and the corresponding eigenvalues are calculated according to the analysis requirements.

[0122] Exemplarily, count the number of triangular patches in each sample data, calculate the curvature of each sample data, calculate the ratio of the surface area to the volume of each sample data, use image processing technology to extract the number of pixels and pixel bit depth of each sample data, and calculate the texture complexity.

[0123] As another alternative implementation of sampling, the data is stratified according to the time interval or geographical region of the spatio-temporal data, and then samples are randomly selected from each layer of data to ensure the representativeness of the sample data in the population.

[0124] As another alternative implementation of sampling, for data that shows a uniform distribution in time or space, sample data is extracted from the spatio-temporal data set at a preset interval, and a sample is extracted at a certain time interval or spatial distance.

[0125] Step S60, according to the range of the eigenvalues, label the corresponding class labels for each of the sample data, and form a labeled data set with all the labeled sample data.

[0126] By labeling the sample data with corresponding class labels according to the range of eigenvalue and forming the labeled sample data into a labeled data set, a supervised machine learning training set can be created. The labeled data set is used to train the machine learning model so that it can automatically identify and classify the complexity of new spatio-temporal data, improve the accuracy and generalization ability of the model, and ensure that the model can reliably evaluate the complexity in practical applications.

[0127] It should be noted that the eigenvalue is a set of numerical values extracted from the sample data, used to characterize the characteristics of the data, including the number of triangular patches, curvature, surface area to volume ratio, texture complexity, etc. The class label is a label assigned to each sample data according to the range of eigenvalue, indicating the complexity level of the data. For example, "simple", "general", "complex", etc. The labeled data set is a data set containing sample data and its corresponding class labels, used to train and validate the machine learning model.

[0128] As an alternative implementation, analyze the distribution of eigenvalues, determine the minimum and maximum values of each eigenvalue, and divide the eigenvalue range into different intervals according to business requirements and data characteristics. Each interval corresponds to a complexity level. Traverse each sample data and assign the corresponding class label according to the range to which its eigenvalue belongs. Save the labeled sample data into a data structure to form a complete labeled data set for training and validating the random forest model.

[0129] Exemplarily, statistically analyze the value ranges of the number of triangular patches T, curvature C, surface area to volume ratio R, and texture complexity M. When the data satisfies the ranges between 0 and "0.3*T", 0 and "0.3*C", 0 and "0.3*R", and 0 and "0.3*M", label the data with the "simple" class label; when the data satisfies the ranges between "0.7*T" and T, "0.7*C" and C, "0.7*R" and R, and "0.7*M" and M, label the data with the "complex" class label; and uniformly label all the remaining data with the "general" class label.

[0130] Step S70: Based on the labeled data set, iteratively train the initial random forest model to obtain a trained random forest model.

[0131] Use the labeled data set to iteratively train the initial random forest model to optimize the performance of the model so that it can more accurately classify new data. Through iterative training, the model can gradually learn the features in the data, thereby improving its generalization ability.

[0132] In this embodiment, iterative training is to repeatedly use the training data to update the parameters of the model to minimize the prediction error or loss function. Iterative training involves adjusting parameters such as the number of trees, the depth of the trees, the randomness of feature selection, etc., and evaluating the performance of the model through methods such as cross-validation.

[0133] As an alternative implementation, set the basic parameters of the random forest model, select appropriate performance evaluation metrics, initially train the random forest model using the labeled dataset, evaluate the performance of the model through the cross-validation method, adjust the parameters of the random forest model according to the performance evaluation results, and repeat the training and evaluation processes until the performance of the model reaches stability or meets the preset stop conditions. Use an independent test dataset to evaluate the performance of the trained random forest model to ensure that the performance of the model on the test dataset is consistent with that on the training dataset.

[0134] It should be noted that in the random forest model, the basic parameters include the number of decision trees in the random forest, the maximum depth of the decision trees, the minimum number of samples for splitting, etc. Increasing the number of trees can improve the accuracy of the model, but at the same time increase the computational cost and memory consumption. Too many trees may lead to overfitting, while too few trees may lead to underfitting. The maximum depth of the decision tree limits the maximum number of layers that the tree can grow. Deeper trees can capture more complex features but may also lead to overfitting. Shallow trees are simpler but may not be able to fully capture the features in the data. The minimum number of samples for splitting is the minimum number of samples required for splitting at a node. A larger value can prevent the model from learning the noise in the data but may also lead to underfitting. A smaller value may increase the complexity of the model.

[0135] It should be noted that the performance evaluation metrics include accuracy, recall, F1-score, etc. Accuracy represents the proportion of correctly classified samples in the total samples; recall, also known as sensitivity or true positive rate, represents the proportion of samples that are actually positive and are correctly predicted as positive among all positive samples; the F1-score is the harmonic mean of precision and recall, which is used to comprehensively evaluate the performance of the model. The higher the F1-score, the better the balance achieved by the model between precision and recall.

[0136] Optionally, step S70 includes:

[0137] Step A10, preprocess the labeled dataset and divide the preprocessed labeled dataset into a training dataset and a test dataset.

[0138] Preprocessing the labeled dataset can ensure the quality and consistency of the data. By dividing the dataset, it can ensure that the machine learning model can learn from the training data during the training process and verify the performance of the model on independent test data. This can avoid overfitting of the model to the training data and ensure that the model has good generalization ability on unseen data.

[0139] In this embodiment, the preprocessing is to organize the feature values and class labels in the labeled dataset into corresponding matrices respectively for the convenience of training the machine learning model. The feature matrix is a two-dimensional array, where each row represents the feature vector of a sample and each column represents a feature. The feature matrix is the input of the machine learning model. The label matrix is a one-dimensional array or vector, where each element corresponds to the class label of a sample. The order of the label matrix matches the sample order of the feature matrix.

[0140] As an alternative implementation, extract the feature values and class labels from the labeled dataset, organize the feature values and class labels into a feature matrix and a label matrix respectively, and divide the preprocessed labeled dataset into an 80% training dataset for training and a 20% test dataset for testing.

[0141] Optionally, step A10 includes:

[0142] Step A11, standardize the feature values of the sample data in the labeled dataset and arrange them according to the order of the sample data to form a feature matrix.

[0143] Eliminating the dimensionality differences between different features by standardizing the feature values helps the model better learn the relationships between features and improve the performance of the model.

[0144] As an alternative implementation, standardize each feature in the labeled dataset, calculate the mean and standard deviation of each feature, subtract the mean from each feature value and divide by the standard deviation. Arrange the processed feature values into a feature matrix according to the order of the sample data. Each row of the feature matrix represents a sample data and each column represents a feature.

[0145] Optionally, for standardization, the min-max normalization method can also be used to scale the feature values to the range of 0 to 1.

[0146] Step A12, organize the class labels of the sample data in the labeled dataset into a label matrix and match it with the order of the sample data in the feature matrix.

[0147] Organizing the class labels into a format corresponding to the feature matrix for correct use during the model training process ensures the correspondence between the feature values and class labels of each sample.

[0148] Exemplarily, class labels are extracted from the labeled dataset, and the class labels are arranged in the order of samples to form a label matrix, and it is ensured that the order of the label matrix matches the sample order of the feature matrix.

[0149] Step A13, divide the preprocessed labeled dataset into the training dataset and the test dataset according to a preset ratio.

[0150] Dividing the preprocessed dataset into a training set and a test set enables the evaluation of the model's performance after training the model, helps prevent the model from overfitting the training data, and ensures that the model has good generalization ability on unseen data.

[0151] Exemplarily, the preprocessed labeled dataset is divided into an 80% training dataset and a 20% test dataset.

[0152] Step A20, use the training dataset to perform iterative training on the initial random forest model for a predetermined number of times to obtain a well-fitted random forest model.

[0153] Through multiple iterative trainings, the parameters of the model are optimized, the prediction accuracy and generalization ability of the model are improved, and the model can better learn the relationship between the features and labels in the data, so as to perform better when dealing with new and unseen data.

[0154] In this embodiment, the initial random forest model is an untrained random forest model, which contains a set of initial parameters, including the number of decision trees, the maximum depth, the minimum number of samples for splitting, etc. The well-fitted random forest model is a model whose parameters are optimized after a predetermined number of iterative trainings and can better predict the labels in the training dataset.

[0155] As an alternative implementation, set the initial parameters such as the number of decision trees, the maximum depth, the minimum number of samples for splitting, and the minimum number of samples for leaf nodes of the random forest model, and use the training dataset to perform iterative training on the initial random forest model. In each iteration, the model randomly extracts samples from the training dataset to train each decision tree, and each decision tree randomly selects a part of the features for splitting during the training process to increase the diversity and generalization ability of the model. During the training process, cross-validation is used to evaluate the performance of the model, and according to the evaluation results, the parameters of the model are adjusted to obtain a well-fitted random forest model.

[0156] Exemplarily, in the random forest model, 100 decision trees are constructed, and the minimum number of samples required to split an internal node is set to 2, and the minimum number of samples required for a leaf node is set to 1. According to the number of decision trees, 100 iterative trainings are performed to obtain a well-fitted random forest model.

[0157] Step A30, cross-validating the fitted random forest model, and testing it according to the test data set to obtain the trained random forest model.

[0158] The generalization ability of the fitted random forest model was further evaluated through cross-validation, and an independent test dataset was used to verify the performance of the model to ensure that the model maintained good prediction accuracy on unseen data.

[0159] It should be noted that cross-validation is a statistical analysis method used to evaluate and improve the predictive performance of a model. It divides the original training data set into several subsets, and trains the model by taking one of the subsets as a test set and the remaining subsets as training sets in turn, thereby obtaining multiple performance indicators, and taking the average value as the final performance evaluation.

[0160] As an optional implementation, the training data set is divided into K subsets through the K-fold cross-validation method, K-1 subsets are used for training each time, and the remaining 1 subset is used for validation. The performance index of each validation is calculated, and the average value is taken as the final cross-validation performance evaluation; the fitted random forest model is predicted using the test data set, and the performance index on the test data set is calculated; the performance index on the cross-validation and test data sets is compared to ensure that the model performs consistently on both.

[0161] Optionally, step A30 includes:

[0162] Step A31, performing a predetermined number of cross-validations on the fitted random forest model, and recording the average accuracy of each cross-validation.

[0163] The performance of the model on different subsets is evaluated through cross-validation, which reduces the variance of model evaluation and improves the reliability of model evaluation.

[0164] For example, the training data set is divided into K subsets by the K-fold cross validation method, and K-1 subsets are used for training each time, and the remaining 1 subset is used for validation. This process is repeated K times, and a different subset is selected as the validation set each time. The accuracy index of each validation is calculated, and the average is taken as the final average accuracy.

[0165] Step A32: If the average accuracy reaches the accuracy threshold, the initial random forest model is predicted and evaluated using the test data set to calculate the test accuracy.

[0166] Use an independent test dataset to evaluate the final performance of the model to ensure that the model has good generalization ability on unseen data.

[0167] Exemplarily, check whether the average accuracy of cross-validation reaches a preset accuracy threshold. If the threshold is reached, use the test data set to further predict the model and calculate the accuracy on the test data set.

[0168] Step A33, if the test accuracy reaches the accuracy threshold, determine the trained random forest model.

[0169] Exemplarily, check whether the accuracy on the test data set reaches a preset accuracy threshold. If the threshold is reached, determine that the random forest model is a trained model and save the trained model for subsequent use.

[0170] This embodiment provides a distributed task processing method based on spatio-temporal data complexity. First, this embodiment constructs a high-quality training data set by extracting representative sample data and calculating feature values, ensuring that the model can learn the diversity and complexity of the data during training and improving the generalization ability of the model. Through iterative training, the parameters of the model are optimized, and the structure and parameters of the decision trees in the random forest model are more reasonable, improving the accuracy of the model in predicting the class labels of sample data. Through cross-validation and evaluation of the test data set, it is ensured that the model also has good prediction performance on unseen data, avoiding overfitting and underfitting problems, and improving the reliability of the model in practical applications.

[0171] Exemplarily, to help understand the implementation process of the distributed task processing method based on spatio-temporal data complexity obtained by combining this embodiment with the above Embodiment 1, please refer to Figure 5 , Figure 5 which provides a schematic diagram of the brief process of a distributed task processing method based on spatio-temporal data complexity. Specifically:

[0172] First, collect the spatio-temporal data of the digital twin model. This data contains time and space information and is used to describe various events and objects in the city.

[0173] Sample the collected spatio-temporal data and select 4000 data models as sample data. Calculate the respective triangular patch number T, curvature C, surface area to volume ratio R, and texture complexity M of the sample data.

[0174] Statistically analyze the value ranges of the number of triangular patches T, curvature C, surface area-to-volume ratio R, and texture complexity M mentioned above, and formulate classification criteria. When the data satisfies the ranges of "0 to 0.3*T, 0 to 0.3*C, 0 to 0.3*R, 0 to 0.3*M", label the data with the category label "simple"; when the data satisfies the ranges of "0.7*T to T, 0.7*C to C, 0.7*R to R, 0.7*M to M", label the data with the category label "complex"; and label all the remaining data with the category label "general".

[0175] Furthermore, standardize the eigenvalue of the sample data to eliminate the problem of overly large differences in the numerical ranges of different feature elements. Divide all the above-mentioned labeled data into training sample data and test data according to the ratio of 80% and 20%. Organize the eigenvalues of the sample data into a feature matrix X, and organize the corresponding category labels into a label matrix Y.

[0176] Furthermore, use the training dataset (X, Y) to iteratively train the initial random forest model to obtain a well-fitted result model, including preprocessing the labeled dataset, dividing the training dataset and the test dataset, using the training dataset to perform 100 iterations of training on the model, and performing 5-fold cross-validation on the model. Record the average accuracy rate of the 5-fold cross-validation as the performance of the model on the dataset. When the average accuracy rate meets the standard (90% and above), use the fitted model to predict the test data and evaluate the prediction accuracy rate on the test data. When the prediction accuracy rate meets the standard (90% and above), complete the training of the model.

[0177] According to the trained random forest model, classify and predict the spatio-temporal data, output the category label corresponding to the feature vector, which represents the complexity of the spatio-temporal data. Map the category label into a weight coefficient, and formulate a distributed task queue based on the weight coefficient to ensure the load balance among task queues.

[0178] Finally, distribute the distributed task queue to each cluster node for distributed processing.

[0179] The overall process accurately evaluates the complexity of spatio-temporal data and maps it into weight coefficients, reasonably allocates computing resources, optimizes the load balance among distributed task queues, and improves the computing efficiency and resource utilization rate. At the same time, using the random forest model for classification prediction and passing through cross-validation and test set evaluation enhances the accuracy and generalization ability of the random forest model. It provides an efficient, stable, and scalable spatio-temporal data processing method for the construction of the digital twin city system platform, ensuring the optimal utilization of resources and the efficient execution of tasks in the distributed system.

[0180] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation on the distributed task processing method based on spatio-temporal data complexity of the present application. Any simple transformation in more forms based on this technical concept is within the protection scope of the present application.

[0181] The present application provides a distributed task processing device based on spatio-temporal data complexity. The distributed task processing device based on spatio-temporal data complexity includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the distributed task processing method based on spatio-temporal data complexity in the first embodiment above.

[0182] Reference is made below Figure 6 , which shows a schematic structural diagram of a distributed task processing device suitable for implementing the embodiments of the present application. The distributed task processing device based on spatio-temporal data complexity in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The shown distributed task processing device based on spatio-temporal data complexity is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0183] As Figure 6As shown, the distributed task processing device based on spatio-temporal data complexity may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM, Random Access Memory) 1004. In the random access memory 1004, various programs and data required for the operation of the distributed task processing device based on spatio-temporal data complexity are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD, Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the distributed task processing device based on spatio-temporal data complexity to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a distributed task processing device with various systems based on spatio-temporal data complexity, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0184] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0185] The distributed task processing device based on spatio-temporal data complexity provided by this application adopts the distributed task processing method based on spatio-temporal data complexity in the above-mentioned embodiment, and can solve the technical problem of how to improve the efficiency of distributed task processing. Compared with the prior art, the beneficial effects of the distributed task processing device based on spatio-temporal data complexity provided by this application are the same as those of the distributed task processing method based on spatio-temporal data complexity provided by the above-mentioned embodiment, and other technical features in the distributed task processing device based on spatio-temporal data complexity are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated here.

[0186] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0187] As mentioned above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0188] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the distributed task processing method based on spatio-temporal data complexity in the above-mentioned embodiment.

[0189] The computer-readable storage medium provided by the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.

[0190] The above computer-readable storage medium can be included in a distributed task processing device based on spatio-temporal data complexity; or it can exist independently without being assembled into a distributed task processing device based on spatio-temporal data complexity.

[0191] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by a distributed task processing device based on spatio-temporal data complexity, the distributed task processing device based on spatio-temporal data complexity can write computer program code for performing the operations of the present application in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0192] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0193] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0194] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned distributed task processing method based on spatio-temporal data complexity, and can solve the technical problem of how to improve the efficiency of distributed task processing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the distributed task processing method based on spatio-temporal data complexity provided by the above embodiments, and will not be elaborated here.

[0195] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made using the description of the present application and the content of the accompanying drawings under the technical concept of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A distributed task processing method based on spatiotemporal data complexity, characterized in that: The distributed task processing method based on spatiotemporal data complexity includes: Extracting feature vectors of spatiotemporal data of the digital twin model, and inputting the feature vectors into the trained random forest model; Using the trained random forest model, classify and predict the spatiotemporal data according to the feature vector, and output a category label corresponding to the feature vector, where the category label represents the complexity of the spatiotemporal data; Determine the weight coefficient corresponding to the spatiotemporal data according to the mapping rule between the category label and the weight coefficient, wherein the mapping rule includes: arranging the category labels in descending order according to the complexity to establish a linear mapping relationship function; or assigning the weight coefficient to each category label based on the complexity to create a mapping table between the category label and the corresponding weight coefficient; Formulate a distributed task queue based on the weight coefficient, obtain the distributed task queue corresponding to each cluster node, calculate the sum of the weight coefficients of each distributed task queue, and use the queue with the smallest difference in the sum of the weight coefficients as the target task queue; distribute the spatiotemporal data to the target task queue according to the task allocation strategy to ensure load balancing between the distributed task queues; The distributed task queue is distributed to each of the cluster nodes for distributed processing.

2. The distributed task processing method based on spatiotemporal data complexity according to claim 1, characterized in that: The step of performing classification prediction on the spatiotemporal data according to the feature vector by using the trained random forest model and outputting a category label corresponding to the feature vector, wherein the category label represents the complexity of the spatiotemporal data comprises: Input the input feature vectors into each decision tree in the random forest model respectively; Through each decision tree, classify and predict the input feature vector, and output the prediction result of each decision tree; The prediction results of each decision tree are counted, a voting method is used to obtain a final classification prediction result, and the corresponding category label is output.

3. The distributed task processing method based on spatiotemporal data complexity according to claim 1, characterized in that: The step of extracting the feature vector of the spatiotemporal data of the digital twin model and inputting the feature vector into the trained random forest model includes: Extracting a preset number of sample data from the spatiotemporal data set, and calculating a characteristic value of each of the sample data; According to the range of the characteristic value, label each sample data with a corresponding category label, and form a labeling data set with all the labeled sample data; Based on the labeled data set, the initial random forest model is iteratively trained to obtain a trained random forest model.

4. The distributed task processing method based on spatiotemporal data complexity according to claim 3, characterized in that: The step of iteratively training the initial random forest model based on the labeled data set to obtain a trained random forest model comprises: Preprocessing the labeled data set, and dividing the preprocessed labeled data set into a training data set and a test data set; Using the training data set, performing a predetermined number of iterative training on the initial random forest model to obtain a fitted random forest model; The fitted random forest model is cross-validated and tested according to the test data set to obtain the trained random forest model.

5. The distributed task processing method based on spatiotemporal data complexity according to claim 4, characterized in that: The step of preprocessing the labeled data set and dividing the preprocessed labeled data set into a training data set and a test data set comprises: Standardizing the feature values ​​of the sample data in the labeled data set, and arranging them in order of the sample data to form a feature matrix; Arrange the category labels of the sample data in the labeled data set into a label matrix and match the order of the sample data in the feature matrix; According to a preset ratio, the preprocessed labeled data set is divided into the training data set and the test data set.

6. The distributed task processing method based on spatiotemporal data complexity according to claim 4, characterized in that: The step of cross-validating the fitted random forest model and testing it according to the test data set to obtain the trained random forest model comprises: Perform a predetermined number of cross-validations on the fitted random forest model, and record the average accuracy of each cross-validation; If the average accuracy reaches the accuracy threshold, the initial random forest model is predicted and evaluated using the test data set to calculate the test accuracy; If the test accuracy reaches the accuracy threshold, the trained random forest model is determined.

7. A distributed task processing device based on spatiotemporal data complexity, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the distributed task processing method based on spatiotemporal data complexity as described in any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the distributed task processing method based on spatiotemporal data complexity as described in any one of claims 1 to 6 are implemented.

Images