Data resource processing method, device, electronic device and storage medium

Abstract

The present application provides a data resource processing method, device, electronic device and storage medium, which relate to the field of computer technology. The method includes: first obtaining training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples; obtaining model parameters of a global model to be trained, and dividing the model parameters of the global model to be trained into several parameter sets; then using an improved fireworks algorithm to search for the best among several preset edge servers, and when it is determined that a preset iteration termination condition is met, determining a combination of parameter sets and edge servers; finally receiving parameter sets of local models that have been trained and sent by edge servers in all combinations, and aggregating parameter sets of all local models to obtain model parameters of the global model that has been trained. The above method improves the training accuracy of the global model, thereby improving the accuracy of data resource processing.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data resource processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of the fifth generation mobile communication technology (5G), cloud computing, edge computing and other technologies, we have entered an era of big data traffic, including various video data, image data, various data collected by the Internet of Things, and various industry data, etc. When analyzing and searching for the optimal solution for these massive data resources, it is usually necessary to build a model and improve the adaptability of the model through learning and training of big data.

[0003] In order to improve the learning speed of the model, existing model training methods often use a distributed training framework to achieve rapid learning of model parameters. The specific idea is to iteratively train the local models trained by each edge server in the distribution, so as to achieve rapid convergence of the global model. The advantage of this implementation method is that it avoids the problem of insufficient computing power of a single edge node and enhances the processing power of the edge node.

[0004] However, the existing data resource processing methods still have the technical problem that the global model is easily trapped in the local optimum due to the use of distributed training, which leads to low global model accuracy and unreasonable resource processing results. Summary of the invention

[0005] The present application provides a data resource processing method, device, electronic device and storage medium to solve the technical problem that the existing data resource processing method still has the problem that the global model is prone to fall into the local optimum due to the use of distributed training, resulting in low global model accuracy, which in turn leads to poor data resource processing accuracy.

[0006] According to a first aspect of the present application, a data resource processing method is provided, comprising:

[0007] Acquire training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples;

[0008] Obtaining model parameters of a global model to be trained, and dividing the model parameters of the global model to be trained into a plurality of parameter sets;

[0009] The improved fireworks algorithm is used to search for the best among several preset edge servers, and when it is determined that the preset iteration termination condition is met, the combination of the parameter set and the edge server is determined; wherein the edge server in the combination is used to train the local model corresponding to the parameter set based on the training samples collected by the collection device connected thereto, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases;

[0010] Receive parameter sets of trained local models sent by all edge servers in the combination, and aggregate the parameter sets of all the local models to obtain model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

[0011] Optionally, the optimizing operation among a plurality of preset edge servers by using the improved fireworks algorithm, and determining a combination of the parameter set and the edge server when it is determined that a preset iteration termination condition is met, comprises:

[0012] Based on a plurality of preset edge servers, an initial fireworks population including a plurality of fireworks is randomly generated, and the number of fireworks in the initial fireworks population is recorded; wherein each of the fireworks comprises: any one of the parameter sets and an edge server for training a local model corresponding to the parameter set;

[0013] The fitness value of each firework is calculated according to a preset fitness function, and the number of explosion sparks corresponding to each firework is calculated according to the fitness value of each firework and a preset nonlinear decreasing formula; wherein the preset nonlinear decreasing formula is formula (1):

[0014]

[0015] Among them, S i is the number of explosion sparks, S * Used to limit the number of explosion sparks, and S * =S max *(1-γ T-t ), S max is the preset maximum number of explosion sparks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Y max is the worst value of fireworks’ fitness, f(x i ) is fireworks xi The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant;

[0016] Calculating the explosion amplitude of each firework according to the fitness value of each firework, and generating explosion sparks according to the number of explosion sparks corresponding to each firework and the explosion amplitude of the firework;

[0017] mutating the fireworks to obtain mutated sparks;

[0018] According to the fireworks, the explosion sparks and the variant sparks, the initial fireworks population is updated to obtain a candidate population, and the number of fireworks in the candidate population is determined according to a preset nonlinear increasing formula; wherein the preset nonlinear increasing formula is formula (2):

[0019] N t =int(N*(1+γ T-t )) (2)

[0020] Among them, int is the integer symbol, N t is the number of fireworks in the candidate population generated during the tth iteration, N is the number of fireworks in the initial fireworks population, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, and t is the current number of iterations of the improved fireworks algorithm;

[0021] When it is determined that a preset iteration termination condition is met, the iteration is stopped and the combination of the parameter set and the edge server is determined.

[0022] Optionally, the calculating the explosion amplitude of each firework according to the fitness value of each firework includes:

[0023] According to the fitness value of each firework and the preset explosion amplitude formula, the explosion amplitude of each firework is calculated; wherein the preset explosion amplitude formula is formula (3):

[0024]

[0025] Among them, A i For fireworks x i The explosion amplitude, A * Used to restrict fireworks i The explosion amplitude, and A * =A max *(1-γ T-t ), A max is the preset maximum explosion amplitude of fireworks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Ymin is the optimal fitness value of fireworks, f(x i ) is fireworks x i The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant.

[0026] Optionally, mutating the fireworks to obtain mutated sparks includes:

[0027] According to the explosion amplitude of the fireworks and the preset displacement variation formula, the fireworks are subjected to displacement variation by varying the dimensions of the fireworks to obtain variant sparks; wherein the preset displacement variation formula is formula (4):

[0028] x ik (t+1)=x ik (t)+A i ×rand(-1,1) (4)

[0029] Among them, x ik (t+1) is the mutation spark after displacement mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before the displacement mutation in the kth dimension in the current iteration process, A i is the explosion amplitude of the fireworks, and rand(-1,1) is a random value between (-1,1);

[0030] and / or,

[0031] According to a preset Gaussian variation formula, the fireworks are subjected to Gaussian variation to obtain variation sparks; wherein the preset Gaussian variation formula is formula (5):

[0032] x ik ′(t+1)=x ik (t)·g (5)

[0033] Among them, x ik ′(t+1) is the mutation spark after Gaussian mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before Gaussian mutation in the kth dimension of the current iteration process, and g is a Gaussian distribution with mean 1 and variance 1.

[0034] Optionally, the method further comprises:

[0035] According to the mapping formula corresponding to the preset mapping rule, the fireworks that exceed the preset feasible solution space are mapped so that the mapped fireworks are located within the preset feasible solution space; wherein the mapping formula corresponding to the preset mapping rule is formula (6):

[0036] x ik″(t+1)=x LB,k (t)+A(x UB,k (t)-x LB,k (t))×rand(0,1) (6)

[0037] Among them, x ik ″(t+1) is the fireworks after mapping, x UB,k (t) is the upper limit of the preset feasible solution space, x LB,k (t) The lower limit of the preset feasible solution space, rand(0,1) is a random value between (0,1).

[0038] Optionally, the updating of the initial fireworks population according to the fireworks, the explosion sparks and the variant sparks to obtain a candidate population includes:

[0039] The fireworks, the explosion sparks and the variant sparks are used as candidate fireworks, and the candidate fireworks with the highest fitness value are selected from all the candidate fireworks;

[0040] Based on the Manhattan distance selection strategy, the remaining alternative fireworks are selected from all the alternative fireworks except the alternative fireworks with the highest fitness value; wherein the Manhattan distance selection strategy adopts formula (7):

[0041]

[0042] Among them, P i is the probability of the i-th firework being selected, K is the number of all the alternative fireworks except the one with the highest fitness value, d(x i ,x j ) is the distance between the i-th firework and the j-th firework;

[0043] The candidate fireworks with the highest fitness value and the remaining candidate fireworks constitute a candidate population.

[0044] Optionally, the preset fitness function is formula (8):

[0045]

[0046] Among them, Fitness is the fitness value of fireworks, n is the total number of edge servers, f i It takes 0 or 1. When the value is 0, it means that the i-th edge server is not selected. When the value is 1, it means that the i-th edge server is selected. i The error value between the parameter set of the local model trained by the i-th edge server and the parameter set of the global model.

[0047] According to a second aspect of the present application, a data resource processing device is provided, including:

[0048] Acquire training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples;

[0049] An acquisition and division module is used to acquire model parameters of a global model to be trained, and divide the model parameters of the global model to be trained into a plurality of parameter sets;

[0050] An optimization determination module is used to use the improved fireworks algorithm to perform optimization in a plurality of preset edge servers, and when it is determined that a preset iteration termination condition is met, determine the combination of the parameter set and the edge server; wherein the edge server in the combination is used to train a local model corresponding to the parameter set based on training samples collected by a collection device connected thereto, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases;

[0051] A receiving aggregation module is used to receive parameter sets of trained local models sent by all edge servers in the combination, and aggregate the parameter sets of all the local models to obtain model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

[0052] According to a third aspect of the present application, there is provided an electronic device, comprising: at least one processor and a memory;

[0053] The memory stores computer-executable instructions;

[0054] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the data resource processing method described in the first aspect above.

[0055] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data resource processing method described in the first aspect above.

[0056] The present application provides a data resource processing method, which first obtains training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples; obtains model parameters of a global model to be trained, and divides the model parameters of the global model to be trained into a plurality of parameter sets; then uses an improved fireworks algorithm to perform optimization in a plurality of preset edge servers, and when it is determined that a preset iteration termination condition is met, determines a combination of parameter sets and edge servers; wherein the edge servers in the combination are used to train local models corresponding to the parameter set based on the training samples collected by the collection device connected thereto; finally, receives parameter sets of the trained local models sent by all edge servers in the combination, aggregates the parameter sets of all local models, and obtains model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

[0057] The present application improves the fireworks algorithm from two aspects. On the one hand, the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases. On the other hand, the number of fireworks in the candidate population of the improved fireworks algorithm increases according to a preset nonlinear increasing formula as the number of iterations increases during the iteration process. By using a nonlinear decreasing method to adjust the number of explosion sparks, the model can be prevented from being limited to a local optimum. By using a nonlinear increasing method to adjust the number of fireworks in the population, a faster global search can be achieved in the early stage of the algorithm, local optimization can be fully achieved in the later stage, and the local search capability can be jumped out, thereby achieving a global optimum, and ultimately improving the accuracy of the model parameters of the global model, thereby improving the accuracy of data resource processing.

[0058] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] Figure 1 A flowchart of a data resource processing method provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the model parameter division provided in the embodiment of the present application;

[0062] Figure 3 Provided in the embodiments of this application Figure 1 Schematic diagram of the process of step S103;

[0063] Figure 4 A schematic diagram of the structure of a data resource processing device provided in an embodiment of the present application;

[0064] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0065] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0067] In the era of massive data, data in different scenarios are very different. For example, the communications industry has many scenarios, including: urban villages, densely populated residential areas, office areas, and suburbs. For example, if users need to build a business model, the business model of urban villages is very different from that of office areas or suburbs. If a cloud center server collects all the business data of the entire city and then trains the model, then it will face a problem: should it take the data of urban villages, suburban areas, or densely populated residential areas? Obviously, any single data cannot be discarded at will, so how can we ensure that the model built by the cloud center server conforms to the "big trend", which is to roughly reflect the business development status of the entire city. Therefore, it is necessary to find more "suitable" business data from the massive data to train the business model, in order to achieve the construction of a business "big trend" that can reflect the entire city. Since the data distribution of each scene is independent and different, when building a central model (the central model corresponds to the distributed model, the central model refers to the brain of the city, the central part, and the distributed model is also called the local model, which generally refers to the model for parameter training on the edge side), it is necessary to select useful data for modeling. Otherwise, the training accuracy of the central model will be reduced due to the difference in the distribution of some data. Therefore, the optimal solution refers to a part of the data that achieves a higher accuracy of the central model in the massive data.

[0068] Although the existing data resource processing methods avoid the problem of insufficient computing power of a single edge node by adopting a distributed training method and enhance the processing power of the edge nodes. However, this brings about another important problem: if the data distribution collected by the edge nodes is very different, the parameters of the global model will become unstable as the number of iterations increases. To avoid this situation, it is necessary to try to find some local models that are similar to the global model for iterative training. In addition, another issue must be considered, that is, how to ensure that the parameters of the global model have a certain degree of representativeness or scalability. In other words, not only must edge nodes with a similar data distribution to the global model be selected, but it is also necessary to ensure that the parameters trained by these edge nodes can represent the characteristics of the global model, so as to ensure that the global model is representative. This method can be used to determine the parameters of the total model in the cloud.

[0069] To achieve the above goals, the current common practice is to use the distributed training method of swarm intelligence to process data resources, and use the fireworks algorithm to realize the advantages of population collaborative search to optimize the model parameters of the global model.

[0070] However, the existing fireworks algorithm is prone to falling into local optimality due to its own shortcomings. Therefore, the existing data resource processing methods have technical problems such as low precision of the global model and poor accuracy of data resource processing.

[0071] In order to solve the above technical problems, the overall inventive concept of this application is to provide a method that is applied in the computer field and can improve the accuracy of the global model and thereby achieve accurate processing of data resources.

[0072] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0073] Figure 1 A flowchart of a data resource processing method provided in an embodiment of the present application. Figure 1 As shown, the method of this embodiment includes:

[0074] S101: Acquire training samples collected by a collection device connected to an edge server; different edge servers correspond to different types of training samples;

[0075] S102: Obtain model parameters of the global model to be trained, and divide the model parameters of the global model to be trained into several parameter sets.

[0076] It should be understood that the global model is also called the central model. The parameter set includes at least one model parameter of the global model. Since the number of model parameters of the global model is large, if the "divide and conquer" method is not adopted for training, then in order to train the model parameters of the global model, it will face the problem of large amount of calculation. Therefore, the data resource processing method adopted in this application is based on the framework of distributed training. The model parameters of the global model are first divided into N subtasks, and then the local model is iteratively trained to achieve global rapid convergence. The embodiment of the present application can divide the model parameters of the global model into several subtasks, and set the objective function. The objective function is the following formula (8), which is not repeated here.

[0077] In the embodiment of the present application, each parameter set in S102 corresponds to a subtask, which can be understood as a task of training the model parameters in the parameter set corresponding to the subtask. Figure 2 As shown in Figure 1, the model parameters of the first 10 global models correspond to the first subtask, and so on, the model parameters of the last 10 global models correspond to the tenth subtask.

[0078] It should be noted that the global model and the local model are equal in model architecture and the number of model parameters. If there are 100 model parameters that need to be trained, then in a random case, several of the model parameters can be divided into the first edge server, and the other several model parameters can be divided into the second edge server, and so on, until all model parameters are divided into edge servers for training the corresponding local models.

[0079] All subtasks are used to constitute the total task. For the total task, it is necessary to find a suitable edge server for training. The number of edge servers may be consistent with the number of model parameters, or may not be consistent. For example, the number of model parameters of the global model is 100, and the number of edge servers may be 30 or 100. Furthermore, the present application does not necessarily require that each edge server trains 1 model parameter. The model parameters trained by each edge server may be repeated or may not be repeated. Ultimately, the model parameters trained by different edge servers can correct each other. For example, the first edge server trains the four model parameters 1, 3, 5, and 7, and the second edge server trains the four model parameters 1, 2, 4, and 7. Therefore, the two model parameters 1 and 7 can be mutually corrected by the first edge server and the second edge server.

[0080] S103: Utilize the improved fireworks algorithm to search for the best among several preset edge servers, and when it is determined that the preset iteration termination condition is met, determine the combination of the parameter set and the edge server; wherein the edge server in the combination is used to train a local model corresponding to the parameter set based on the training samples collected by the collection device connected to it, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases.

[0081] It should be understood that the improved fireworks algorithm is a swarm intelligence algorithm, so the distributed training method adopted by the data resource processing method can shorten the training time of the global model. The edge server, or edge node, is deployed on the edge side to collect sample data and train the local model through the collected sample data. The local model refers to a model with the same model architecture and model parameters as the global model. The difference is that the training data used is different, and the local model is trained with data collected by the edge server.

[0082] Since the number of fireworks and the number of explosion sparks in the optimization process of the improved fireworks algorithm are changed and adjusted according to the number of iterations, the improved fireworks algorithm can avoid local optimality and thus improve the training accuracy of the global model.

[0083] S104: Receive parameter sets of trained local models sent by edge servers in all combinations, and aggregate the parameter sets of all local models to obtain model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

[0084] The above aggregation can be understood as merging, which can be done by averaging or by choosing one of two methods. After the improved fireworks algorithm is iterated, for the first subtask, the obtained parameter set and edge server combination are (1, 5) and (1, 6), and the model parameters trained by the fifth edge server and the sixth edge server are very similar. Then the two edge servers upload the training results obtained by executing the first subtask (i.e., the parameter set of the above local model) to the global model, and the global model selects the model parameters of the global model by averaging or choosing one of two methods.

[0085] For example, the architecture of the global model is simple. If there are three model parameters and three local models, the model parameters of the three local models are (1, 1, 1), (1, 0, 1) and (1, 1, 0) respectively. Then, after aggregation, the parameters of the three models of the global model can be (1, 1, 1). The reasons for such aggregation are: first, the principle of minority obeys majority; second, the principle of the law of large numbers. Regardless of which principle is used, it is hoped that the parameters selected by the global model will be as close as possible to the central value range of the model parameters of the local models, so as to ensure that the global model can eventually meet most scenarios. Therefore, the model parameters obtained by the global model fluctuate as much as possible within the average value range of the model parameters trained by the edge server, so that the results obtained by the global model can meet the situations of most scenarios.

[0086] The embodiment of the present application improves the fireworks algorithm from two aspects. On the one hand, the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases. On the other hand, the number of fireworks in the candidate population of the improved fireworks algorithm increases according to a preset nonlinear increasing formula as the number of iterations increases during the iteration process. By using a nonlinear decreasing method to adjust the number of explosion sparks, the model can be prevented from being limited to the local optimum. By using a nonlinear increasing method to adjust the number of fireworks in the population, a faster global search can be achieved in the early stage of the algorithm, local optimization can be fully achieved in the later stage, and the local search capability can be jumped out, thereby achieving the global optimum, and finally improving the accuracy of the model parameters of the global model, thereby ensuring the accuracy of the data resource prediction results.

[0087] Based on the above embodiments, the technical solution of the present application is described in more detail in combination with the following specific embodiments.

[0088] Figure 3 Provided in the embodiments of this application Figure 1 Schematic diagram of the process of step S103 in FIG. Figure 1 Based on the embodiment shown, S103 includes the following steps:

[0089] S301: Based on a number of preset edge servers, randomly generate an initial firework population including multiple fireworks, and record the number of fireworks in the initial firework population; wherein each firework includes: any parameter set and an edge server for training a local model corresponding to the parameter set.

[0090] The above S301 can be understood as the process of initializing a feasible solution. The fireworks algorithm is an iterative algorithm. In the embodiment of the present application, assuming that the number of fireworks in the initial fireworks population is 20, the fireworks population can be initialized to 20 groups of data such as (1, 5), (2, 5), (1, 6), (3, 8), ..., (N, K). The first group of data (1, 5) indicates that the first subtask is executed by the fifth edge server, and the second group of data (2, 5) indicates that the second subtask is assigned to the fifth edge server. Because the edge server has multiple processors, different subtasks can be processed separately, and the same edge server has irrelevant processes when processing the first subtask and the second subtask. Then, the third set of data (1, 6) indicates that the first subtask is assigned to the sixth edge server. It can be seen that the first subtask is assigned to the fifth and sixth edge servers. Since the same subtask is located in different areas in different edge servers, the sample sources of different edge servers are different for the same subtask, so the parameters trained by different edge servers have different numerical results. Similarly, (N, K) indicates that the Nth subtask is executed by the Kth edge server.

[0091] For another example, 100 model parameters are divided into 10 subtasks. According to Fireworks (1, 2), the 1st to 10th model parameters are placed on the second edge server for training, that is, the second edge server is used to train the 1st to 10th parameters. Through the Fireworks algorithm, learning is performed to determine the adaptability of the first model parameter in the sample data obtained by the second edge server. If the adaptability is good, it will be retained. If the adaptability is not good, it will be removed. Then determine the adaptability of other edge servers when training 1-10 parameters. If the adaptability is good, they will be retained.

[0092] In the embodiment of the present application, the same edge node can execute multiple subtasks, and the same subtask can be executed by multiple edge servers at the same time. When multiple subtasks face different training data, it can be determined which model parameters are sensitive to the training data and which parameters are relatively less sensitive to the training data. Generally speaking, model parameters with strong sensitivity are relatively important to the global model, and model parameters with poor sensitivity are relatively unimportant to the global model. In addition, multiple subtasks are executed in different edge servers, and mutual correction can also be performed.

[0093] In the embodiment of the present application, the matching of subtasks and edge servers is not fixed, and subtasks can be assigned to different edge servers through a random algorithm, and then the subsequent steps are executed according to the training results of the edge servers.

[0094] S302: Calculate the fitness value of each firework according to the preset fitness function, and calculate the number of explosion sparks corresponding to each firework according to the fitness value of each firework and the preset nonlinear decreasing formula. The preset nonlinear decreasing formula is formula (1):

[0095]

[0096] Among them, S i is the number of explosion sparks, S * is a constant used to limit the number of explosion sparks, and S * =S max *(1-γ T-t ), S max is the preset maximum number of explosion sparks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Y max is the worst value of fireworks’ fitness, f(x i ) is fireworks x i The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant.

[0097] The embodiment of the present application uses a nonlinear decreasing method to adjust the number of explosion sparks, which can avoid the model from falling into a local optimum to a certain extent. Specifically, before S302, the fitness value of each firework is calculated by the fitness function, and the number of explosion sparks to be generated is determined according to the fitness value of the firework in S302. The better the fitness value of the firework, the more explosion sparks it generates.

[0098] In the above formula (1), S * The value of gradually decreases with the increase of the number of iterations, and is inversely proportional to the population size, which is the number of fireworks in the initial fireworks population or the candidate population. In other words, if the embodiment of the present application appropriately increases the number of fireworks in the candidate population while reducing the number of explosion sparks generated each time, it will cause a phenomenon that the number of algorithm searches is reduced, and the probability of retaining the normal explosion sparks generated by the better explosion sparks is high. Based on this idea, the embodiment of the present application adds S * The following relationship is made with the number of iterations t: * =S max *(1-γ T-t ). max The specific value is set according to business needs, and has different values ​​in different scenarios.

[0099] In order to determine the worst fitness value Y of fireworks max,In each iteration, the training results of the corresponding subtask of each edge server can be calculated,and compared with the model parameters of the global model, and the location of the edge server with the worst fitness can be obtained.

[0100] S303: Calculate the explosion amplitude of each firework according to the fitness value of each firework, and generate explosion sparks according to the number of explosion sparks corresponding to each firework and the explosion amplitude of the firework.

[0101] In the embodiment of the present application, the specific process of calculating the explosion amplitude of fireworks is shown in the following S3031, which will not be repeated here.

[0102] S304: Mutate the fireworks to obtain mutant sparks.

[0103] In the embodiment of the present application, the purpose of the mutation is to increase the diversity of the population and to perform appropriate mutation adjustments on the fireworks in the population. The present application does not specifically limit the mutation method, such as displacement mutation, Gaussian mutation, etc., and the specific mutation process is shown in the following S3041-S3042, which will not be repeated here.

[0104] S305: According to the fireworks, explosion sparks and variant sparks, the initial fireworks population is updated to obtain a candidate population, and the number of fireworks in the candidate population is determined according to a preset nonlinear increasing formula. The preset nonlinear increasing formula is formula (2):

[0105] N t =int(N*(1+γ T-t )) (2)

[0106] Among them, int is the integer symbol, N t is the number of fireworks in the candidate population generated during the tth iteration, N is the number of fireworks in the initial fireworks population, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, and t is the current number of iterations of the improved fireworks algorithm. N is the number of fireworks in the initial fireworks population, for example, N = 20.

[0107] The embodiment of the present application uses a nonlinear increasing method to adjust the population size, thereby achieving faster global search in the early stage of the improved fireworks algorithm and fully realizing local optimization in the later stage, so as to jump out of the local search capability and achieve global optimization.

[0108] S306: When it is determined that a preset iteration termination condition is met, the iteration is stopped, and a combination of a parameter set and an edge server is determined.

[0109] Since a parameter set corresponds to a subtask, the final output of the fireworks algorithm can be a combination of several groups of edge servers and subtasks. According to the combination, it can be known which edge server is used to execute which one or more subtasks. Therefore, according to the combination, it can be known which edge server has the closest training result for which subtask to the result of the global model. For example, for the first subtask, the combination may be (1, 5) and (1, 6), so the training results of the fifth and sixth edge servers are close to the model parameters of the global model.

[0110] In S306, when the preset iteration termination condition is met, a combination of edge servers and subtasks that meets the following formula (8) is found. For the same subtask, when the number of edge servers is one, the model parameters trained by the edge server can be directly used as the model parameters of the global model. When the number of edge servers is multiple and the error rate values ​​of multiple edge servers are similar, the model parameters of these edge servers can be obtained and merged through ensemble learning. Optionally, the weight of ensemble learning adopts weighted average.

[0111] That is to say, after determining the combination, the result corresponding to the combination should also be determined, and used as the parameter set of the trained local model, and then the model parameters of the global model are determined. For example: 100 model parameters are divided into 10 subtasks. If an edge server is used to train the 1st to 10th model parameters in the 1st subtask, then the result is the value of these 10 model parameters. For example, the 5th edge server and the 6th edge server are both used to train the 1st to 10th model parameters, and the difference in the results obtained by these two edge servers is less than 3%. Specifically, the result of the 5th edge server is (1, 0.5, 1, 1, 1, 1, 1, 0, 1), and the result of the 6th edge server is (1, 0.5, 0.97, 1, 1, 1, 1, 1, 0.18, 1). That is, by comparing their different values, the difference between the two sets of model parameters can be determined.

[0112] By executing the above operations S302 to S306, the present application can adjust the population size and the number of explosion sparks in the fireworks algorithm, thereby avoiding the global model from falling into a local optimum.

[0113] In a possible implementation, in S303, the explosion amplitude of each firework is calculated according to the fitness value of each firework, including the following steps: S3031: Calculate the explosion amplitude of each firework according to the fitness value of each firework and a preset explosion amplitude formula; wherein the preset explosion amplitude formula is formula (3):

[0114]

[0115] Among them, A i For fireworks x i The explosion amplitude, A * Used to restrict fireworks i The explosion amplitude, and A * =A max *(1-γ T-t ), A max is the preset maximum explosion amplitude of fireworks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Y min is the optimal fitness value of fireworks, f(x i ) is fireworks x i The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant.

[0116] It can be seen that the calculation of the explosion amplitude of the fireworks in the embodiment of the present application is determined by the maximum explosion amplitude of the fireworks and the fitness value of the fireworks. The larger the fitness value of the fireworks, the smaller the explosion amplitude, and vice versa. min , can be achieved by traversing, for example: traversing 100 times, or 200 times, to find the optimal value of the fitness. Calculate A * The formula is a decay function. As the number of iterations increases, the entire formula is expressed as a decay state.

[0117] If the explosion amplitude is a fixed value, then the global model is likely to fall into a local optimum. Therefore, the embodiment of the present application uses a nonlinear decreasing method to adjust the explosion amplitude, thereby achieving a faster global search in the early stage of the fireworks algorithm and fully realizing local optimization in the later stage, so as to jump out of the local search capability and achieve the global optimum.

[0118] The explosion radius of the fireworks algorithm in the related art is fixed. The fixed explosion radius has certain advantages in the early global search, but the local search ability is weak in the later stage. The explosion amplitude in this application is used to reflect the explosion radius. Compared with the prior art, the explosion amplitude changes with the number of iterations. In the initial iteration process, in order to expand the search ability of the fireworks individual as soon as possible, the explosion amplitude is designed to have a relatively large value to expand the early global search ability. At the end of the iteration, the explosion amplitude is designed to have a relatively small value, so that the fireworks individual can easily jump out of the local search ability in the early stage, and give full play to the local search ability of the fireworks individual in the later stage.

[0119] In a possible implementation, S304: mutating fireworks to obtain mutated sparks includes the following steps S3041 and / or S3042, wherein:

[0120] S3041: According to the explosion amplitude of the fireworks and the preset displacement variation formula, the fireworks are subjected to displacement variation by varying the dimensions of the fireworks to obtain a variation spark; wherein the preset displacement variation formula is formula (4):

[0121] x ik (t+1)=x ik (t)+A i ×rand(-1,1) (4)

[0122] Among them, x ik (t+1) is the mutation spark after displacement mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before the displacement mutation in the kth dimension in the current iteration process, A i is the explosion amplitude of the fireworks, and rand(-1,1) is a random value between (-1,1).

[0123] S3042: Perform Gaussian mutation on the fireworks according to a preset Gaussian mutation formula to obtain mutant sparks; wherein the preset Gaussian mutation formula is formula (5):

[0124] x ik ′(t+1)=x ik (t)·g (5)

[0125] Among them, x ik ′(t+1) is the mutation spark after Gaussian mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before Gaussian mutation in the kth dimension of the current iteration process, and g is a Gaussian distribution with mean 1 and variance 1.

[0126] Through the above two types of mutation operations, the diversity of fireworks in the population can be increased.

[0127] In a possible implementation, the method further includes:

[0128] According to the mapping formula corresponding to the preset mapping rule, the fireworks that exceed the preset feasible solution space are mapped so that the mapped fireworks are located in the preset feasible solution space; wherein the mapping formula corresponding to the preset mapping rule is formula (6):

[0129] x ik ″(t+1)=x LB,k (t)+A(x UB,k (t)-x LB,k(t))×rand(0,1) (6)

[0130] Among them, x ik ″(t+1) is the fireworks after mapping, x UB,k (t) is the upper limit of the preset feasible solution space, x LB,k (t) Preset the lower limit of the feasible solution space, rand(0,1) is a random value between (0,1).

[0131] It should be understood that the preset feasible solution space can be a two-dimensional space, and each feasible solution should be set within the preset feasible solution space, that is, fireworks (x, y) should be set within a certain range. If any solution exceeds the space, it can be judged by the values ​​of x and y. Similarly, the explosion sparks and variant sparks can also be mapped in the same way, which will not be elaborated in this application.

[0132] Through the above mapping method, the effectiveness of fireworks can be guaranteed, and then the effectiveness of the generated population can be guaranteed.

[0133] In a possible implementation, S305, updating the initial fireworks population according to the fireworks, explosion sparks and variant sparks to obtain a candidate population includes the following steps S3051 to S3053, wherein:

[0134] S3051, taking fireworks, explosion sparks and mutation sparks as candidate fireworks, and selecting the candidate fireworks with the highest fitness value from all the candidate fireworks;

[0135] S3052, selecting the remaining alternative fireworks from all the alternative fireworks except the alternative fireworks with the highest fitness value based on the Manhattan distance selection strategy; wherein the Manhattan distance selection strategy adopts formula (7):

[0136]

[0137] Among them, P i is the probability of the i-th firework being selected, K is the number of all alternative fireworks except the one with the highest fitness value, d(x i ,x j ) is the distance between the i-th firework and the j-th firework;

[0138] S3053: The alternative fireworks with the highest fitness value and the remaining alternative fireworks constitute a candidate population.

[0139] In the embodiment of the present application, the closer the individual is to the center point of most fireworks, the greater the probability of being selected and retained; conversely, the smaller the probability of being selected and retained. i The larger it is, the greater the probability of being selected.

[0140] In order to ensure the diversity of samples, the embodiment of the present application pays more attention to random selection in the early stage and to preferential selection in the later stage. Therefore, in the initial stage, in addition to retaining the firework individuals with the best fitness, the embodiment of the present application also retains the remaining candidate fireworks based on the Manhattan distance selection strategy. In the Manhattan distance selection strategy, the probability of a firework individual being selected is related to the position of the individual. Therefore, as the algorithm iteration deepens, it should be considered that the offspring produced by the better individuals are more likely to be selected and retained to the next generation. Therefore, in the present application, after t iterations, after retaining t individuals with the best fitness, the remaining G t -t Select based on Manhattan distance selection strategy.

[0141] The selection strategy for the next generation of fireworks in the related art first selects an individual with the best fitness value, and then the remaining individuals are randomly selected for the next generation of fireworks. This method of randomly selecting individuals to form the next generation of fireworks will result in low algorithm accuracy. Compared with the related art, the embodiment of the present application adds a Manhattan distance selection strategy, which can pass on the good things of the parent generation to the offspring as much as possible during the process of the parent generation generating the offspring, that is, in the process of iteration, try to keep the best part. The present application adopts a dynamic selection strategy, which is mainly based on random selection at the beginning, so as to increase the diversity of the population. In the later stage, the optimal algorithm is adopted to increase the probability of selecting the next generation of excellence.

[0142] In a possible implementation, the preset fitness function is formula (8):

[0143]

[0144] Among them, Fitness is the fitness value of fireworks, n is the total number of edge servers, f i It takes 0 or 1. When the value is 0, it means that the i-th edge server is not selected. When the value is 1, it means that the i-th edge server is selected. i The error value between the parameter set of the local model trained by the i-th edge server and the parameter set of the global model.

[0145] In the embodiment of the present application, the first term in the fitness function is used to indicate whether the edge server is selected to train the local model, and the second term is used to indicate the average error of the model parameters of the local model trained by each selected edge server. i It is the error between the results obtained by the central model and the local model for the same sample. For example, for the same sample, the result obtained by the central model is 1.5, and the result obtained by the edge model is 1, then the error is 0.5.

[0146] In the embodiment of the present application, each firework individual has a fitness function value, and the firework with the smallest fitness function value has the smallest error in the model parameters of the global model. According to formula (8), the smaller the error of the fitness value, the fewer edge servers it selects, and the smaller the average error.

[0147] In summary, the distributed training method adopted by the data resource processing method provided by the present application first sets a fitness function, which makes the model parameters of the local model in the fireworks individual and the model parameters of the global model as small as possible to avoid the phenomenon of unstable model parameters. Secondly, in the process of the parent generation generating the offspring, the embodiment of the present application also hopes that the good things of the parent generation will be inherited to the offspring as much as possible, that is, in the process of iteration, the best part will be kept as much as possible; in addition, in order to expand the search ability of the fireworks individual as soon as possible, the amplitude of the explosion spark should be larger to expand the global search ability in the early stage. At the end, the amplitude of the explosion spark should be smaller, so as to give full play to the local search ability of the fireworks individual in the later stage, so that the fireworks individual can easily jump out of the local search ability in the early stage.

[0148] The embodiments of the present application have the following advantages:

[0149] (1) The fireworks algorithm is optimized and improved by setting a fitness function. The smaller the value of this fitness function, the smaller the difference between the model parameters of the local model in the fireworks individual and the model parameters of the global model, which can avoid the phenomenon of unstable model parameters.

[0150] (2) To prevent the model from falling into a local optimum, the embodiment of the present application uses a nonlinear decreasing method to adjust the explosion amplitude and the number of explosion sparks. At the same time, a nonlinear increasing method is used to adjust the population size, so as to achieve a faster global search in the early stage of the algorithm and fully realize local optimization in the later stage. In this way, it is possible to jump out of the local search capability and achieve the global optimum.

[0151] (3) The embodiment of the present application adopts a dynamic selection strategy, which is mainly based on random selection at the beginning, so as to increase the diversity of the population. In the later stage, the optimal selection algorithm is adopted to increase the probability of excellent individuals being selected and retained to the next generation.

[0152] Figure 4 This is a schematic diagram of the structure of a data resource processing device provided in an embodiment of the present application. The device of this embodiment may be in the form of software and / or hardware. Figure 4 As shown, the data resource processing device provided in this embodiment includes: an acquisition module 41, an acquisition and division module 42, an optimization determination module 43 and a receiving and aggregation module 44. Among them:

[0153] The acquisition module 41 is used to acquire training samples collected by a collection device connected to the edge server; different edge servers correspond to different types of training samples.

[0154] The acquisition and division module 42 is used to obtain the model parameters of the global model to be trained, and divide the model parameters of the global model to be trained into several parameter sets.

[0155] The optimization determination module 43 is used to use the improved fireworks algorithm to perform optimization in a plurality of preset edge servers, and when it is determined that the preset iteration termination condition is met, determine the combination of the parameter set and the edge server; wherein the edge server in the combination is used to train a local model corresponding to the parameter set based on the training samples collected by the collection device connected thereto, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases.

[0156] The receiving aggregation module 44 is used to receive the parameter sets of the trained local models sent by the edge servers in all combinations, and aggregate the parameter sets of all local models to obtain the model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

[0157] The data resource processing device provided in this embodiment can be used to execute the data resource processing method provided in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be elaborated here.

[0158] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0159] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0160] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a receiver 50, a transmitter 51, at least one processor 52 and a memory 53. The electronic device composed of the above components can be used to implement the above-mentioned several specific embodiments of the present application, which will not be described in detail here.

[0161] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, each step of the method in the above embodiment is implemented.

[0162] The embodiment of the present application also provides a computer program product, including computer instructions, which implement the various steps of the method in the above embodiment when executed by a processor.

[0163] Various embodiments of the systems and techniques described above in the present application can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or electronic device.

[0165] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0168] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps disclosed in this application can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document does not limit this.

[0169] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of this application should be included in the protection scope of this application.

Claims

1. A data resource processing method, characterized in that: include: Acquire training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples; Obtaining model parameters of a global model to be trained, and dividing the model parameters of the global model to be trained into a plurality of parameter sets; The improved fireworks algorithm is used to search for the best among several preset edge servers, and when it is determined that the preset iteration termination condition is met, the combination of the parameter set and the edge server is determined; wherein the edge server in the combination is used to train the local model corresponding to the parameter set based on the training samples collected by the collection device connected thereto, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases; Receive parameter sets of trained local models sent by all edge servers in the combination, and aggregate the parameter sets of all the local models to obtain model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

2. The method according to claim 1, characterized in that The improved fireworks algorithm is used to search for the best among a plurality of preset edge servers, and when it is determined that a preset iteration termination condition is met, a combination of the parameter set and the edge server is determined, including: Based on a plurality of preset edge servers, an initial fireworks population including a plurality of fireworks is randomly generated, and the number of fireworks in the initial fireworks population is recorded; wherein each of the fireworks comprises: any one of the parameter sets and an edge server for training a local model corresponding to the parameter set; The fitness value of each firework is calculated according to a preset fitness function, and the number of explosion sparks corresponding to each firework is calculated according to the fitness value of each firework and a preset nonlinear decreasing formula; wherein the preset nonlinear decreasing formula is formula (1): Among them, S i is the number of explosion sparks, S * Used to limit the number of explosion sparks, and S * =S max *(1-γ T-t ), S max is the preset maximum number of explosion sparks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Y max is the worst value of fireworks’ fitness, f(x i ) is fireworks x i The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant; Calculating the explosion amplitude of each firework according to the fitness value of each firework, and generating explosion sparks according to the number of explosion sparks corresponding to each firework and the explosion amplitude of the firework; mutating the fireworks to obtain mutated sparks; According to the fireworks, the explosion sparks and the variant sparks, the initial fireworks population is updated to obtain a candidate population, and the number of fireworks in the candidate population is determined according to a preset nonlinear increasing formula; wherein the preset nonlinear increasing formula is formula (2): N t =int(N*(1+γ T-t )) (2) Among them, int is the integer symbol, N t is the number of fireworks in the candidate population generated during the tth iteration, N is the number of fireworks in the initial fireworks population, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, and t is the current number of iterations of the improved fireworks algorithm; When it is determined that a preset iteration termination condition is met, the iteration is stopped and the combination of the parameter set and the edge server is determined.

3. The method according to claim 2, characterized in that The step of calculating the explosion amplitude of each firework according to the fitness value of each firework comprises: According to the fitness value of each firework and the preset explosion amplitude formula, the explosion amplitude of each firework is calculated; wherein the preset explosion amplitude formula is formula (3): Among them, A i For fireworks x i The explosion amplitude, A * Used to restrict fireworks i The explosion amplitude, and A * =A max *(1-γ T-t ), A max is the preset maximum explosion amplitude of fireworks, γ is a constant, T is the maximum number of iterations of the improved fireworks algorithm, t is the current number of iterations of the improved fireworks algorithm, and Y min is the optimal fitness value of fireworks, f(x i ) is fireworks x i The fitness value, N t is the number of fireworks in the candidate population generated during the tth iteration, and ε is a constant.

4. The method according to claim 2, characterized in that: The step of mutating the fireworks to obtain mutated sparks includes: According to the explosion amplitude of the fireworks and the preset displacement variation formula, the fireworks are subjected to displacement variation by varying the dimensions of the fireworks to obtain variant sparks; wherein the preset displacement variation formula is formula (4): x ik (t+1)=x ik (t)+A i ×rand(-1,1) (4) Among them, x ik (t+1) is the mutation spark after displacement mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before the displacement mutation in the kth dimension in the current iteration process, A i is the explosion amplitude of the fireworks, and rand(-1,1) is a random value between (-1,1); and / or, According to a preset Gaussian variation formula, the fireworks are subjected to Gaussian variation to obtain variation sparks; wherein the preset Gaussian variation formula is formula (5): x ik ′(t+1)=x ik (t)·g (5) Among them, x ik ′(t+1) is the mutation spark after Gaussian mutation in the kth dimension in the next iteration process, x ik (t) is the fireworks before Gaussian mutation in the kth dimension of the current iteration process, and g is a Gaussian distribution with mean 1 and variance 1.

5. The method according to claim 2, characterized in that: The method further comprises: According to the mapping formula corresponding to the preset mapping rule, the fireworks that exceed the preset feasible solution space are mapped so that the mapped fireworks are located within the preset feasible solution space; wherein the mapping formula corresponding to the preset mapping rule is formula (6): x ik ″(t+1)=x LB,k (t)+A(x UB,k (t)-x LB,k (t))×rand(0,1) (6) Among them, x ik ″(t+1) is the fireworks after mapping, x UB,k (t) is the upper limit of the preset feasible solution space, x LB,k (t) The lower limit of the preset feasible solution space, rand(0,1) is a random value between (0,1).

6. The method according to claim 2, characterized in that The updating of the initial fireworks population according to the fireworks, the explosion sparks and the variant sparks to obtain a candidate population includes: The fireworks, the explosion sparks and the variant sparks are used as candidate fireworks, and the candidate fireworks with the highest fitness value are selected from all the candidate fireworks; Based on the Manhattan distance selection strategy, the remaining alternative fireworks are selected from all the alternative fireworks except the alternative fireworks with the highest fitness value; wherein the Manhattan distance selection strategy adopts formula (7): Among them, P i is the probability of the i-th firework being selected, K is the number of all the alternative fireworks except the one with the highest fitness value, d(x i ,x j ) is the distance between the i-th firework and the j-th firework; The candidate fireworks with the highest fitness value and the remaining candidate fireworks constitute a candidate population.

7. The method according to claim 2, characterized in that The preset fitness function is formula (8): Among them, Fitness is the fitness value of fireworks, n is the total number of edge servers, f i It takes 0 or 1. When the value is 0, it means that the i-th edge server is not selected. When the value is 1, it means that the i-th edge server is selected. i The error value between the parameter set of the local model trained by the i-th edge server and the parameter set of the global model.

8. A data resource processing device, characterized in that: include: An acquisition module, used to acquire training samples collected by a collection device connected to an edge server; wherein different edge servers correspond to different types of training samples; An acquisition and division module is used to acquire model parameters of a global model to be trained, and divide the model parameters of the global model to be trained into a plurality of parameter sets; An optimization determination module is used to use the improved fireworks algorithm to perform optimization in a plurality of preset edge servers, and when it is determined that a preset iteration termination condition is met, determine the combination of the parameter set and the edge server; wherein the edge server in the combination is used to train a local model corresponding to the parameter set based on training samples collected by a collection device connected thereto, so that the local model predicts the type of the training sample; the number of explosion sparks generated by the improved fireworks algorithm during the iteration process decreases according to a preset nonlinear decreasing formula as the number of iterations increases, and the number of fireworks in the candidate population of the improved fireworks algorithm during the iteration process increases according to a preset nonlinear increasing formula as the number of iterations increases; A receiving aggregation module is used to receive parameter sets of trained local models sent by all edge servers in the combination, and aggregate the parameter sets of all the local models to obtain model parameters of the trained global model, so that the trained global model can predict the type of data resources collected by any edge server to obtain a prediction result.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the data resource processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data resource processing method according to any one of claims 1 to 7.

Images