A quantitative data processing method based on cloud computing

Abstract

The present invention discloses a quantitative data processing method based on cloud computing, which belongs to the technical field of cloud computing and quantitative data processing. By adopting a deep learning algorithm and a multi-level search algorithm to optimize the parameters of a quantitative data analysis model, the learning of historical data is realized, and rules that can process quantitative data are formed. The quantitative data are then processed according to the rules, thereby realizing automatic processing of quantitative data. This method can not only effectively improve the analysis efficiency of quantitative data, but also eliminate errors caused by human analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing and quantitative data processing, and in particular relates to a quantitative data processing method based on cloud computing. Background Art

[0002] Quantitative data generally refers to data expressed in numerical values, and quantitative data exists in all walks of life. Usually, in order to improve the efficiency of enterprise production, a special functional department will often analyze quantitative data (for example, collect equipment operating parameters during the production process to analyze whether the production is abnormal; for example, classify massive transaction data so that staff can quickly make decisions and optimize strategies according to the classification categories), so as to achieve data classification or generate decision-making suggestions to assist leaders in making production scheduling decisions or optimizing production parameters.

[0003] However, using functional departments to analyze quantitative data not only has the problem of inefficiency, but also leads to different final data classification and analysis results due to differences in personnel. Summary of the invention

[0004] The present invention provides a quantitative data processing method based on cloud computing to solve the problems existing in the prior art.

[0005] A quantitative data processing method based on cloud computing, comprising:

[0006] Crawling quantitative data in a target data source through a web crawler, preprocessing the crawled quantitative data to obtain preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server;

[0007] Construct a quantitative data analysis model in the cloud server, and use a multi-level search algorithm to optimize the parameters of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and deploy the optimized quantitative data analysis model on the cloud server;

[0008] The quantitative data analysis model deployed on the scheduling cloud server analyzes and processes the quantitative data after preprocessing to obtain the quantitative data processing results based on cloud computing.

[0009] Further, crawling the quantitative data in the target data source through a web crawler, preprocessing the crawled quantitative data to obtain the preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server, including:

[0010] Crawling the quantitative data in the target data source through a web crawler, performing missing value processing, outlier processing and normalization processing on the crawled quantitative data, and obtaining the quantitative data after initial processing;

[0011] The initialized quantized data is uniformly stored as a preset data structure to obtain the preprocessed quantized data, and the preprocessed quantized data is stored in the cloud server.

[0012] Furthermore, a quantitative data analysis model is constructed in the cloud server, including: constructing a deep learning model, and using the constructed deep learning model as the quantitative data analysis model.

[0013] Furthermore, a multi-level search algorithm is used to optimize the parameters of the quantitative data analysis model to obtain an optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on a cloud server, including:

[0014] Initialize the model parameters of the quantitative data analysis model to obtain a model parameter vector, and repeatedly obtain multiple model parameter vectors to obtain a population;

[0015] Obtain historical data and label data corresponding to the historical data, and use the historical data and label data corresponding to the historical data to obtain a fitness value corresponding to each model parameter vector;

[0016] For each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform a local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search;

[0017] For the model parameter vector after the local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search;

[0018] For the model parameter vector after the equilibrium search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search;

[0019] For the model parameter vector after the global search, the fitness value of the model parameter vector is re-obtained, and the number of individual expansions is obtained according to the fitness value;

[0020] According to the number of individual expansions, a normal distribution method is used to generate multiple new model parameter vectors, and the new model parameter vectors are merged with the model parameter vectors after global search, and then the new population is selected by the method of survival of the fittest;

[0021] For the new population, obtain the fitness value corresponding to each model parameter vector, and obtain the optimal model parameter vector according to the fitness value corresponding to each model parameter vector;

[0022] Determine whether the algorithm termination condition is met. If so, output the optimal model parameter vector, otherwise return to the local search step;

[0023] The output optimal model parameter vector is used as the final model parameter of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on the cloud server.

[0024] Furthermore, for each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform a local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search, including:

[0025] For each model parameter vector, a random vector is generated: (Δx i,1 ,Δx i,2 ,....,Δx i,d ,...,Δx i,D ), where Δx i,d represents the random parameter corresponding to the d-th dimension parameter in the i-th model parameter vector, i = 1, 2, ..., L, L represents the total number of model parameter vectors, d = 1, 2, ..., D, D represents the total dimension of the parameters in the model parameter vector, and Δx i,d Randomly set α with a probability of 0.5 t or -α t , α t Represents the adaptive step size in the t-th optimization process;

[0026] Get the pseudo gradient of the i-th model parameter vector as:

[0027]

[0028] in, Indicates that the d-th dimension parameter in the i-th model parameter vector corresponds to the gradient, f(*) represents the fitness function, represents the i-th model parameter vector in the t-th optimization process, represents a random vector;

[0029] According to the preset fixed field of view β0, the adaptive step size α in the t+1th optimization process is obtained t+1 , and according to the adaptive step size α t+1 and the pseudo gradient of the i-th model parameter vector The local search for the i-th model parameter vector is:

[0030]

[0031] in, represents the d-th dimension parameter in the i-th model parameter vector during the t-th optimization process, express The corresponding local search value;

[0032] Determine whether the fitness corresponding to the i-th model parameter vector after the local search increases. If so, accept the local search to obtain the model parameter vector after the local search. Otherwise, directly use the original model parameter vector as the model parameter vector after the local search.

[0033] Furthermore, the adaptive step length α in the t+1th optimization process is obtained according to the preset fixed field of view β0 t+1 for:

[0034]

[0035] Among them, α0 represents the initial step size, π represents pi, and T represents the maximum number of optimization times.

[0036] Furthermore, for the model parameter vector after the local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search, including:

[0037] Based on the current optimization times t, the change field of view is obtained as:

[0038]

[0039] Among them, β t represents the change horizon during the t-th optimization process, β t+1 represents the change horizon in the t+1th optimization process, and the initial value of the change horizon is β0, and e represents a natural constant;

[0040] According to the change of field of view β t+1 , get the balance search range as:

[0041]

[0042]

[0043] Among them, lb i,d represents the lower bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, ub i,d represents the upper bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, represents the d-th dimension parameter in the i-th model parameter vector after local search;

[0044] According to the balanced search range, the balanced search for the model parameter vector after the local search is:

[0045]

[0046] Among them, rand(0,1) represents a random number between (0,1), Represents the balance search after

[0047] Determine whether the fitness corresponding to the i-th model parameter vector after the balanced search increases. If so, accept the balanced search to obtain the model parameter vector after the balanced search. Otherwise, directly use the model parameter vector after the local search as the model parameter vector after the balanced search.

[0048] Furthermore, for the model parameter vector after the balance search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search, including:

[0049] According to the model parameter vector after all equilibrium searches, the average position of the population is obtained as Each dimension parameter in is the average value of all other model parameter vectors in the same dimension;

[0050] According to the average position of the population The global search for the model parameter vector after the equilibrium search is:

[0051]

[0052] in, represents the model parameter vector after global search, rand(x min ,x max ) means (x min ,x max ), a random number between min Indicates the lower limit of the global search interval, x max Indicates the upper limit of the global search interval, Indicates the average position The d-th dimension parameter in ;

[0053] Determine whether the fitness corresponding to the i-th model parameter vector after the global search increases. If so, accept the global search to obtain the model parameter vector after the global search. Otherwise, directly use the model parameter vector after the balanced search as the model parameter vector after the global search.

[0054] Furthermore, for the model parameter vector after the global search, the fitness value of the model parameter vector is re-obtained, and the number of individual extensions is obtained according to the fitness value:

[0055]

[0056] Among them, γ i represents the number of individual expansions corresponding to the model parameter vector after the ith global search, ξ(*) represents the rounding function, and f i represents the fitness value corresponding to the model parameter vector after the i-th global search, f max represents the maximum fitness value corresponding to the model parameter vector after global search, f min Represents the minimum fitness value corresponding to the model parameter vector after global search, S max Indicates the preset maximum extended integer, S min Indicates the preset minimum extended integer.

[0057] Furthermore, according to the number of individual expansions, a normal distribution method is used to generate multiple new model parameter vectors, and the new model parameter vectors are merged with the model parameter vectors after the global search, and then the new population is selected by the method of survival of the fittest, including:

[0058] For each model parameter vector after global search, the normal distribution Generate a number of new model parameter vectors around the model parameter vector that is equal to the number of individual extensions;

[0059] Among them, σ t represents the standard deviation, and σ t =((Tt) h / (T h ))*(σ c -σ z )+σ z , h represents the nonlinear harmonic index, σ c Indicates the starting value of the standard deviation, σ z Indicates the end value of standard deviation;

[0060] The new model parameter vector is fused with the model parameter vector after global search to obtain a fused population, and the L model parameter vectors with the largest fitness values ​​in the fused population are retained to obtain a new population.

[0061] The present invention provides a quantitative data processing method based on cloud computing. By adopting a deep learning algorithm and a multi-level search algorithm to optimize the parameters of a quantitative data analysis model, the method realizes the learning of historical data, forms rules that can process quantitative data, and then processes the quantitative data according to the rules, thereby realizing automatic processing of quantitative data. This method can not only effectively improve the analysis efficiency of quantitative data, but also eliminate errors caused by human analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0063] Figure 1 A flowchart of a quantitative data processing method based on cloud computing provided in an embodiment of the present invention.

[0064] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0066] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0067] like Figure 1 As shown, a quantitative data processing method based on cloud computing includes:

[0068] S1. Crawling quantitative data in a target data source through a web crawler, preprocessing the crawled quantitative data to obtain preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server.

[0069] Quantitative data can be digital data (such as transaction data, equipment operation data, etc.), or image data, etc. The target data source is used to characterize the data source to be analyzed. Through a fixed sampling frequency, data within a certain period of time can be collected and the corresponding classification results can be automatically analyzed. For example: by analyzing image data, classification results such as parts classification, product surface defect identification (such as glass and car paint defect identification) can be achieved; or, by analyzing the equipment operation parameters of a production line, it is possible to classify whether the equipment is operating normally; or, by analyzing transaction data, different preset decision analyses can be obtained. Quantitative transaction data processing based on cloud computing can greatly improve data processing speed and analysis efficiency, enabling investors to make trading decisions faster at critical moments.

[0070] S2. Build a quantitative data analysis model in the cloud server, and use a multi-level search algorithm to optimize the parameters of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and deploy the optimized quantitative data analysis model on the cloud server.

[0071] Cloud computing has powerful computing performance. By building a quantitative data analysis model in a cloud server, quantitative data can be quickly analyzed, thereby improving data analysis processing efficiency. Although cloud computing has powerful computing performance, in the process of deploying a quantitative data analysis model, a large amount of resources are required to optimize the model, and if the optimization effect is not good, it will lead to subsequent waste of computer resources and computing power. Therefore, the embodiment of the present invention proposes a multi-level search algorithm, which can effectively improve the optimization speed and optimization effect, thereby effectively saving computer resources.

[0072] S3. The quantitative data analysis model deployed on the scheduling cloud server analyzes and processes the preprocessed quantitative data to obtain the quantitative data processing results based on cloud computing.

[0073] The optimized quantitative data analysis model has the ability to process and analyze quantitative data. Therefore, the optimized quantitative data analysis model can be scheduled to automatically process data, thereby improving data processing efficiency.

[0074] Optionally, in order to improve recognition accuracy (such as image classification accuracy), the parameters of the quantitative data analysis model may be updated according to the data to further optimize the model.

[0075] The present invention provides a quantitative data processing method based on cloud computing. By adopting a deep learning algorithm and a multi-level search algorithm to optimize the parameters of a quantitative data analysis model, the method realizes the learning of historical data, forms rules that can process quantitative data, and then processes the quantitative data according to the rules, thereby realizing automatic processing of quantitative data. This method can not only effectively improve the analysis efficiency of quantitative data, but also eliminate errors caused by human analysis.

[0076] In this embodiment, crawling quantitative data in a target data source through a web crawler, preprocessing the crawled quantitative data to obtain preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server includes:

[0077] Crawling the quantitative data in the target data source through a web crawler, performing missing value processing, outlier processing and normalization processing on the crawled quantitative data, and obtaining the quantitative data after initial processing;

[0078] The initialized quantized data is uniformly stored as a preset data structure to obtain the preprocessed quantized data, and the preprocessed quantized data is stored in the cloud server.

[0079] Normalizing the quantitative data ensures the data recognition rate, thereby saving computer processing resources. It is worth noting that no matter whether the quantitative data is used to optimize the quantitative data analysis model, it should be processed into a unified data length and a unified data structure to ensure that the quantitative data analysis model can normally recognize the quantitative data. For example: when classifying image data, it needs to be processed into a unified size.

[0080] In this embodiment, a quantitative data analysis model is constructed in a cloud server, including: constructing a deep learning model, and using the constructed deep learning model as the quantitative data analysis model.

[0081] Optionally, when the quantized data is numerical data, the quantized data can be organized into vectors, and a deep learning model such as a BP (BackPropagation) neural network, a BRF (Radial Basis Function) neural network, etc. can be used as a quantitative data analysis model to achieve recognition of vector data. The quantized data can also be assembled into matrix data in a fixed format, and a convolutional neural network can be used as a quantitative data analysis model to achieve recognition of vector data. It is worth noting that when the quantized data is an image, it is equivalent to limiting the data input form, and a deep learning model that can recognize image data is required to construct a quantitative data analysis model.

[0082] In this embodiment, a multi-level search algorithm is used to optimize the parameters of the quantitative data analysis model to obtain an optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on a cloud server, including:

[0083] Initialize the model parameters of the quantitative data analysis model to obtain a model parameter vector, and repeatedly obtain multiple model parameter vectors to obtain a population.

[0084] For example, model parameters are randomly generated between an upper limit and a lower limit of model parameters of a quantitative data analysis model, thereby obtaining a model parameter vector.

[0085] The historical data and the label data corresponding to the historical data are obtained, and the fitness value corresponding to each model parameter vector is obtained using the historical data and the label data corresponding to the historical data.

[0086] By taking historical data as input and label data corresponding to the historical data as expected output, the error function value of the quantitative data analysis model can be obtained, and the negative of the error function value can be used as the fitness value. According to this method, the fitness value corresponding to each model parameter vector can be obtained one by one or in parallel.

[0087] For each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search.

[0088] For the model parameter vector after local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search.

[0089] For the model parameter vector after the equilibrium search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search.

[0090] For the model parameter vector after the global search, the fitness value of the model parameter vector is re-obtained, and the individual expansion quantity is obtained according to the fitness value.

[0091] According to the number of individual expansions, the normal distribution method is used to generate multiple new model parameter vectors, and the new model parameter vectors are merged with the model parameter vectors after global search, and then the new population is selected using the method of survival of the fittest.

[0092] For the new population, the fitness value corresponding to each model parameter vector is obtained, and the optimal model parameter vector is obtained according to the fitness value corresponding to each model parameter vector.

[0093] Determine whether the algorithm termination condition is met. If so, output the optimal model parameter vector, otherwise return to the local search step.

[0094] Optionally, determining whether the algorithm termination condition is met may include: determining whether the current optimization times are greater than the maximum optimization times; if so, the algorithm termination condition is met; otherwise, the algorithm termination condition is not met.

[0095] The output optimal model parameter vector is used as the final model parameter of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on the cloud server.

[0096] In the prior art, algorithms such as gradient descent and particle swarm optimization are often used to train deep learning models, which not only has a slow training speed, but also has a poor training effect, resulting in a large waste of computer resources. Therefore, the embodiment of the present invention proposes a multi-level search algorithm to optimize the parameters of the quantitative data analysis model to solve the problems existing in the prior art and improve the classification ability of quantitative data.

[0097] In this embodiment, for each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform a local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search, including:

[0098] For each model parameter vector, a random vector is generated: (Δx i,1 ,Δx i,2 ,....,Δx i,d ,...,Δx i,D ), where Δx i,d represents the random parameter corresponding to the d-th dimension parameter in the i-th model parameter vector, i = 1, 2, ..., L, L represents the total number of model parameter vectors, d = 1, 2, ..., D, D represents the total dimension of the parameters in the model parameter vector, and Δx i,d Randomly set α with a probability of 0.5 t or -α t , α t Represents the adaptive step size in the t-th optimization process;

[0099] Get the pseudo gradient of the i-th model parameter vector as:

[0100]

[0101] in, Indicates that the d-th dimension parameter in the i-th model parameter vector corresponds to the gradient, f(*) represents the fitness function, represents the i-th model parameter vector in the t-th optimization process, represents a random vector;

[0102] According to the preset fixed field of view β0, the adaptive step size α in the t+1th optimization process is obtained t+1 , and according to the adaptive step size α t+1 and the pseudo gradient of the i-th model parameter vector The local search for the i-th model parameter vector is:

[0103]

[0104] in, represents the d-th dimension parameter in the i-th model parameter vector during the t-th optimization process, express The corresponding local search value;

[0105] By searching with an adaptive step size, not only can the search direction be continuously changed, but also the search step size can be continuously changed, thereby achieving a fine search within a local range, which is more helpful in finding the optimal value.

[0106] Determine whether the fitness corresponding to the i-th model parameter vector after the local search increases. If so, accept the local search to obtain the model parameter vector after the local search. Otherwise, directly use the original model parameter vector as the model parameter vector after the local search.

[0107] Local search often does not affect the globality of the algorithm, so the embodiment of the present invention introduces a greedy algorithm to perform greedy selection on the local search, thereby improving the convergence speed of the algorithm.

[0108] In this embodiment, the adaptive step length α in the t+1th optimization process is obtained according to the preset fixed field of view β0. t+1 for:

[0109]

[0110] Among them, α0 represents the initial step size, π represents pi, and T represents the maximum number of optimization times.

[0111] In this embodiment, for the model parameter vector after the local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search, including:

[0112] Based on the current optimization times t, the change field of view is obtained as:

[0113]

[0114] Among them, β t represents the change horizon during the t-th optimization process, β t+1 represents the change horizon in the t+1th optimization process, and the initial value of the change horizon is β0, and e represents a natural constant;

[0115] According to the change of field of view β t+1 , get the balance search range as:

[0116]

[0117]

[0118] Among them, lb i,d represents the lower bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, ub i,d represents the upper bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, represents the d-th dimension parameter in the i-th model parameter vector after local search;

[0119] According to the balanced search range, the balanced search for the model parameter vector after the local search is:

[0120]

[0121] Among them, rand(0,1) represents a random number between (0,1), Represents the balance search after

[0122] Determine whether the fitness corresponding to the i-th model parameter vector after the balanced search increases. If so, accept the balanced search to obtain the model parameter vector after the balanced search. Otherwise, directly use the model parameter vector after the local search as the model parameter vector after the balanced search.

[0123] Changing the field of view by setting β t+1 Balanced search can provide a relatively large search range in the early stage of the algorithm, thereby improving the algorithm's global search capability. t+1 Gradually narrowing, thus gradually narrowing the search range, so that the algorithm has the ability of local fine search. And the randomness of the balanced search is stronger, which can further achieve comprehensive exploration of local areas.

[0124] In this embodiment, for the model parameter vector after the balance search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search, including:

[0125] According to the model parameter vector after all equilibrium searches, the average position of the population is obtained as Each dimension parameter in is the average value of all other model parameter vectors in the same dimension;

[0126] According to the average position of the population The global search for the model parameter vector after the equilibrium search is:

[0127]

[0128] in, represents the model parameter vector after global search, rand(x min ,x max ) means (x min ,x max ), a random number between min Indicates the lower limit of the global search interval, x max Indicates the upper limit of the global search interval, Indicates the average position The d-th dimension parameter in ;

[0129] Determine whether the fitness corresponding to the i-th model parameter vector after the global search increases. If so, accept the global search to obtain the model parameter vector after the global search. Otherwise, directly use the model parameter vector after the balanced search as the model parameter vector after the global search.

[0130] Although the balanced search has a certain global search function, it decays as the algorithm progresses. Therefore, the embodiment of the present invention introduces a global search to enable the algorithm to always have a global search capability, thereby improving the convergence effect.

[0131] Optionally, during the global search, when the fitness corresponding to the i-th model parameter vector after the global search does not increase, the global search may be received with a preset probability, thereby avoiding algorithm stagnation.

[0132] In this embodiment, for the model parameter vector after the global search, the fitness value of the model parameter vector is re-obtained, and the number of individual extensions is obtained according to the fitness value:

[0133]

[0134] Among them, γi represents the number of individual expansions corresponding to the model parameter vector after the ith global search, ξ(*) represents the rounding function, and f i represents the fitness value corresponding to the model parameter vector after the i-th global search, f max represents the maximum fitness value corresponding to the model parameter vector after global search, f min Represents the minimum fitness value corresponding to the model parameter vector after global search, S max Indicates the preset maximum extended integer, S min Indicates the preset minimum extended integer.

[0135] Since high-dimensional mathematical problems are difficult to solve, an embodiment of the present invention proposes a population expansion strategy, which can effectively improve the convergence speed and local search accuracy by generating multiple uniformly distributed new individuals in a fixed area near the model parameter vector and using the method of survival of the fittest for selection.

[0136] In this embodiment, a plurality of new model parameter vectors are generated according to the number of individual expansions using a normal distribution method, and the new model parameter vectors are merged with the model parameter vectors after the global search, and then a new population is selected using the method of survival of the fittest, including:

[0137] For each model parameter vector after global search, the normal distribution Generate a number of new model parameter vectors around the model parameter vector that is equal to the number of individual extensions;

[0138] Among them, σ t represents the standard deviation, and σ t =((Tt) h / (T h ))*(σ c -σ z )+σ z , h represents the nonlinear harmonic index, σ c Indicates the starting value of the standard deviation, σ z Indicates the end value of standard deviation;

[0139] The new model parameter vector is fused with the model parameter vector after global search to obtain a fused population, and the L model parameter vectors with the largest fitness values ​​in the fused population are retained to obtain a new population.

[0140] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0144] A person of ordinary skill in the art can understand that all or part of the steps in realizing the above-mentioned facts and methods can be completed by instructing the relevant hardware through a program, and the program involved or the program described can be stored in a computer-readable storage medium. When the program is executed, it includes the following steps: At this time, the corresponding method steps are derived, and the storage medium can be ROM / RAM, a disk, an optical disk, etc.

[0145] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A quantitative data processing method based on cloud computing, characterized in that: include: Crawling quantitative data in a target data source through a web crawler, preprocessing the crawled quantitative data to obtain preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server; Construct a quantitative data analysis model in the cloud server, and use a multi-level search algorithm to optimize the parameters of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and deploy the optimized quantitative data analysis model on the cloud server; The quantitative data analysis model deployed on the scheduling cloud server analyzes and processes the pre-processed quantitative data to obtain the quantitative data processing results based on cloud computing; Crawling quantitative data in a target data source through a web crawler, preprocessing the crawled quantitative data to obtain preprocessed quantitative data, and storing the preprocessed quantitative data in a cloud server, including: Crawling the quantitative data in the target data source through a web crawler, performing missing value processing, outlier processing and normalization processing on the crawled quantitative data, and obtaining the quantitative data after initial processing; The initialized quantized data is uniformly stored as a preset data structure to obtain the preprocessed quantized data, and the preprocessed quantized data is stored in a cloud server; Constructing a quantitative data analysis model in a cloud server, including: constructing a deep learning model, and using the constructed deep learning model as a quantitative data analysis model; A multi-level search algorithm is used to optimize the parameters of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on the cloud server, including: Initialize the model parameters of the quantitative data analysis model to obtain a model parameter vector, and repeatedly obtain multiple model parameter vectors to obtain a population; Obtain historical data and label data corresponding to the historical data, and use the historical data and label data corresponding to the historical data to obtain a fitness value corresponding to each model parameter vector; For each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform a local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search; For the model parameter vector after the local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search; For the model parameter vector after the equilibrium search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search; For the model parameter vector after the global search, the fitness value of the model parameter vector is re-obtained, and the number of individual expansions is obtained according to the fitness value; According to the number of individual expansions, a normal distribution method is used to generate multiple new model parameter vectors, and the new model parameter vectors are merged with the model parameter vectors after global search, and then the new population is selected by the method of survival of the fittest; For the new population, obtain the fitness value corresponding to each model parameter vector, and obtain the optimal model parameter vector according to the fitness value corresponding to each model parameter vector; Determine whether the algorithm termination condition is met. If so, output the optimal model parameter vector, otherwise return to the local search step; The output optimal model parameter vector is used as the final model parameter of the quantitative data analysis model to obtain the optimized quantitative data analysis model, and the optimized quantitative data analysis model is deployed on the cloud server.

2. The method for quantitative data processing based on cloud computing according to claim 1, characterized in that: For each model parameter vector, a local search method with a fixed field of view and an adaptive step size is used to perform a local search on the model parameter vector, and the local search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the local search, including: For each model parameter vector, a random vector is generated: (Δx i,1 ,Δx i,2 ,....,Δx i,d ,...,Δx i,D ), where Δx i,d represents the random parameter corresponding to the d-th dimension parameter in the i-th model parameter vector, i = 1, 2, ..., L, L represents the total number of model parameter vectors, d = 1, 2, ..., D, D represents the total dimension of the parameters in the model parameter vector, and Δx i,d Randomly set α with a probability of 0.5 t or -α t , α t Represents the adaptive step size in the t-th optimization process; Get the pseudo gradient of the i-th model parameter vector as: in, Indicates that the d-th dimension parameter in the i-th model parameter vector corresponds to the gradient, f(*) represents the fitness function, represents the i-th model parameter vector in the t-th optimization process, represents a random vector; According to the preset fixed field of view β0, the adaptive step size α in the t+1th optimization process is obtained t+1 , and according to the adaptive step size α t+1 and the pseudo gradient of the i-th model parameter vector The local search for the i-th model parameter vector is: in, represents the d-th dimension parameter in the i-th model parameter vector during the t-th optimization process, express The corresponding local search value; Determine whether the fitness corresponding to the i-th model parameter vector after the local search increases. If so, accept the local search to obtain the model parameter vector after the local search. Otherwise, directly use the original model parameter vector as the model parameter vector after the local search.

3. The method for quantitative data processing based on cloud computing according to claim 2, characterized in that: According to the preset fixed field of view β0, the adaptive step size α in the t+1th optimization process is obtained t+1 for: Among them, α0 represents the initial step size, π represents pi, and T represents the maximum number of optimization times.

4. The method for quantitative data processing based on cloud computing according to claim 3, characterized in that: For the model parameter vector after the local search, a global balanced search method with a variable field of view is used to perform a balanced search on the model parameter vector, and the balanced search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the balanced search, including: Based on the current optimization times t, the change field of view is obtained as: Among them, β t represents the change horizon during the t-th optimization process, β t+1 represents the change horizon in the t+1th optimization process, and the initial value of the change horizon is β0, and e represents a natural constant; According to the change of field of view β t+1 , get the balance search range as: Among them, lb i,d represents the lower bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, ub i,d represents the upper bound of the balanced search for the d-th dimension parameter in the i-th model parameter vector, represents the d-th dimension parameter in the i-th model parameter vector after local search; According to the balanced search range, the balanced search for the model parameter vector after the local search is: Among them, rand(0,1) represents a random number between (0,1), Represents the balance search after Determine whether the fitness corresponding to the i-th model parameter vector after the balanced search increases. If so, accept the balanced search to obtain the model parameter vector after the balanced search. Otherwise, directly use the model parameter vector after the local search as the model parameter vector after the balanced search.

5. The method for quantitative data processing based on cloud computing according to claim 4, characterized in that: For the model parameter vector after the equilibrium search, a global search method of population information fusion is used to perform a global search on the model parameter vector, and the global search process is controlled according to the fitness value corresponding to the model parameter vector to obtain the model parameter vector after the global search, including: According to the model parameter vector after all equilibrium searches, the average position of the population is obtained as Each dimension parameter in is the average value of all other model parameter vectors in the same dimension; According to the average position of the population The global search for the model parameter vector after the equilibrium search is: in, represents the model parameter vector after global search, rand(x min ,x max ) means (x min ,x max ), a random number between min Indicates the lower limit of the global search interval, x max Indicates the upper limit of the global search interval, Indicates the average position The d-th dimension parameter in ; Determine whether the fitness corresponding to the i-th model parameter vector after the global search increases. If so, accept the global search to obtain the model parameter vector after the global search. Otherwise, directly use the model parameter vector after the balanced search as the model parameter vector after the global search.

6. The method for quantitative data processing based on cloud computing according to claim 5, characterized in that: For the model parameter vector after global search, the fitness value of the model parameter vector is re-obtained, and the number of individual expansions is obtained according to the fitness value: Among them, γ i represents the number of individual expansions corresponding to the model parameter vector after the ith global search, ξ(*) represents the rounding function, and f i represents the fitness value corresponding to the model parameter vector after the i-th global search, f max represents the maximum fitness value corresponding to the model parameter vector after global search, f min Represents the minimum fitness value corresponding to the model parameter vector after global search, S max Indicates the preset maximum extended integer, S min Indicates the preset minimum extended integer.

7. The method for quantitative data processing based on cloud computing according to claim 6, characterized in that: According to the number of individual expansions, a normal distribution method is used to generate multiple new model parameter vectors, and the new model parameter vectors are merged with the model parameter vectors after global search, and then the new population is selected by the method of survival of the fittest, including: For each model parameter vector after global search, the normal distribution Generate a number of new model parameter vectors around the model parameter vector that is equal to the number of individual extensions; Among them, σ t represents the standard deviation, and σ t =((Tt) h / (T h ))*(σ c -σ z )+σ z , h represents the nonlinear harmonic index, σ c Indicates the starting value of the standard deviation, σ z Indicates the end value of standard deviation; The new model parameter vector is fused with the model parameter vector after global search to obtain a fused population, and the L model parameter vectors with the largest fitness values ​​in the fused population are retained to obtain a new population.

Images