GPU-based factor automatic mining method and system, electronic equipment and storage medium

By migrating data from the CPU to the GPU and running genetic algorithm models on the GPU, the GPLearn framework's inefficiency problem when processing high-dimensional data is solved, and efficient factor mining training and high-dimensional data support is achieved.

CN119027241APending Publication Date: 2024-11-26DOLPHINDB INC (CN)
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410950065.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing GPLearn framework relies on CPU for data processing, which is time-consuming and cannot effectively process high-dimensional data, especially three-dimensional data, resulting in low training efficiency and insufficient factor mining support.

Method used

By migrating training data from the CPU to the GPU and running a genetic algorithm model on the GPU, factor mining training is carried out on the packet data, and the parallel computing power of the GPU is used to improve training efficiency and support factor mining of high-dimensional data.

Benefits of technology

It realizes efficient large-scale data training, reduces training time, can effectively process three-dimensional data, supports factor mining of high-dimensional data, and solves the time-consuming problem caused by CPU dependence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027241A_ABST
    Figure CN119027241A_ABST
Patent Text Reader

Abstract

The invention relates to an automatic factor mining method and system based on a GPU, electronic equipment and a storage medium, and the method comprises the steps: obtaining training data through a CPU, and loading the training data into the GPU; preprocessing the training data in the GPU, and grouping the training data according to a preset grouping rule to obtain grouped data; obtaining a genetic algorithm model generated in the CPU and transferring the genetic algorithm model to the GPU, running the genetic algorithm model by the GPU to carry out factor mining training on the grouped data, and obtaining a factor prediction model when the genetic algorithm model converges; and obtaining to-be-predicted data, and calculating the to-be-predicted data through the factor prediction model to obtain a prediction result. According to the method, the data mining process is distributed to the GPU, so that more efficient large-scale data training is supported, the training efficiency is improved, and the training time is shortened.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data mining technology, and in particular to a GPU-based automatic factor mining method, system, electronic device and storage medium. Background Art

[0002] In the financial field, researchers usually rely on market rules and investment experience to mine factors from massive data to guide the buying and selling decisions of financial derivatives such as stocks. As the available data continues to increase, researchers began to use the GPLearn (Genetic Programming Learn) algorithm to automatically mine potential factors. However, since the GPLearn algorithm requires heuristic search and the search space is huge, the training efficiency is low. In addition, the existing GPLearn framework only supports mining factors from two-dimensional data, and the training support for three-dimensional data is insufficient.

[0003] The traditional gplearn framework mainly relies on the CPU when calculating fitness indicators, which may lead to significant time consumption when training with large amounts of data. In addition, the traditional framework can only process two-dimensional data, such as [time*feature] data; however, for three-dimensional data containing [time*product category*feature], the traditional framework's support is obviously insufficient, which makes it impossible for the traditional framework to mine effective factors from high-dimensional data.

[0004] Currently, there is no corresponding technical solution to solve the problem that relying on CPU for data processing results in a long time consumption and inability to process high-dimensional data. Summary of the invention

[0005] The embodiments of the present application provide a GPU-based automatic factor mining method, system, electronic device and storage medium to at least solve the problem in the related art that data processing relies on the CPU, resulting in a long time consumption and inability to process high-dimensional data.

[0006] In a first aspect, an embodiment of the present application provides a GPU-based automatic factor mining method, comprising:

[0007] The CPU obtains training data and pre-processes the training data.

[0008] The training data is loaded into the GPU, and the pre-processed training data is grouped according to a preset grouping rule to obtain grouped data;

[0009] The genetic algorithm model generated in the CPU is obtained and transferred to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data, and when the genetic algorithm model converges, a factor prediction model is obtained;

[0010] The data to be predicted is obtained, and the data to be predicted is calculated using the factor prediction model to obtain a prediction result.

[0011] In one embodiment, the GPU runs the genetic algorithm model to perform factor mining training on the grouped data, including:

[0012] Step a1, obtaining preset training parameters, wherein the training parameters include population size, preset number of iterations and subtree generation strategy;

[0013] Step a2, performing population initialization processing on the grouped data according to the population quantity to generate initial individuals corresponding to the population quantity;

[0014] Step a3, calculating the fitness index of each of the initial individuals in the GPU;

[0015] Step a4, selecting N initial individuals with the highest fitness as the parent individuals of the next iteration through the elite strategy, and generating new individuals through the subtree generation strategy;

[0016] Step a5, judging whether the iteration termination condition is reached, if so, taking the new individual as the factor prediction model and outputting it; if not, returning to step a3, and calculating the fitness with the new individual as input.

[0017] In one embodiment, the calculating of the fitness index of each of the initial individuals includes:

[0018] Sending the initial individual to the GPU;

[0019] Convert the formula corresponding to the initial individual into the reverse order of the prefix expression to obtain an execution queue, wherein the execution queue includes a data operator and a function operator;

[0020] The GPU reads the execution queue in sequence and performs calculations to obtain calculation results of the initial individuals;

[0021] A preset fitness function and a fitting value are obtained, and the calculation result and the fitting value are used as inputs of the fitness to obtain the fitness index.

[0022] In one embodiment, the step of calculating the data to be predicted by using the factor prediction model to obtain a prediction result includes:

[0023] Acquire prediction data through the CPU, load the prediction data into the GPU, group the data according to preset grouping rules, and obtain prediction grouping data;

[0024] Selecting a formula from the factor prediction model according to the predicted grouping data;

[0025] The formula and the prediction data are input into the GPU for calculation to obtain the prediction result.

[0026] In one embodiment, the training parameters further include a preset threshold and a fitness threshold range, and determining whether the genetic algorithm model converges includes:

[0027] When the number of iterations is greater than or equal to a preset threshold; or when the fitness is not within a preset fitness threshold range, it is determined that the genetic algorithm model has converged.

[0028] In one embodiment, the generating a new individual by using the subtree generation strategy includes:

[0029] Randomly select a pair of parent individuals for crossover operation to generate offspring individuals containing the characteristics of the two parent individuals;

[0030] A preset probability is obtained, and a mutation operation is performed on the newly generated offspring individual with the preset probability to introduce new genetic diversity to obtain the new individual.

[0031] In one embodiment, the preprocessing of the training data includes:

[0032] Cleaning the training data to remove erroneous, duplicate or inconsistent data in the data set;

[0033] Convert the cleaned training data into training data in a preset format;

[0034] Create new feature data based on data in a preset format;

[0035] The feature data is standardized or normalized to obtain preprocessed training data.

[0036] In a second aspect, the embodiment of the present application provides a GPU-based factor automatic mining system, which mainly includes an input module, an automatic mining module, an output module and a prediction module; wherein:

[0037] The input module is used to collect training data and prediction data, pre-process the training data, load the training data into the GPU, and group the training data according to preset grouping rules to obtain grouped data;

[0038] The automatic mining module is used to obtain the genetic algorithm model generated in the CPU and transfer it to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data, and when the genetic algorithm model converges, a factor prediction model is obtained;

[0039] The prediction module is used to obtain the data to be predicted, and calculate the data to be predicted through the factor prediction model to obtain the prediction result;

[0040] The output module is used to output the prediction result.

[0041] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the GPU-based automatic factor mining method as described in the first aspect above is implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the GPU-based automatic factor mining method as described in the first aspect above.

[0043] The GPU-based automatic factor mining method, system, electronic device, and storage medium provided in the embodiments of the present application have at least the following technical effects:

[0044] The training data is obtained through the CPU, and the training data is preprocessed, the training data is loaded into the GPU, and the preprocessed training data is grouped according to the preset grouping rules to obtain grouped data; the genetic algorithm model generated in the CPU is obtained and transferred to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data. When the genetic algorithm model converges, a factor prediction model is obtained; the data to be predicted is obtained, and the data to be predicted is calculated by the factor prediction model to obtain the prediction result. The problem of relying on the CPU for data processing, which leads to a long time consumption and inability to process high-dimensional data, is solved, and support for efficient large-scale data training is achieved.

[0045] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 is a flowchart of a GPU-based automatic factor mining method in an embodiment of the present application;

[0048] Figure 2 is a flow chart of performing factor mining training on the grouped data in one embodiment of the present application;

[0049] Figure 3 is a flow chart of calculating the fitness index of each of the initial individuals in one embodiment of the present application;

[0050] Figure 4 is a structural block diagram of a GPU-based automatic factor mining system in an embodiment of the present application;

[0051] Figure 5 It is a structural block diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0053] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0054] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0055] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0056] The existing gplearn framework faces the problems of low computational efficiency and difficulty in three-dimensional data training. Based on the problems existing in the prior art, the embodiments of the present application provide a GPU-based automatic factor mining method, system, electronic device and storage medium, aiming to overcome the limitations of the prior art.

[0057] In a first aspect, the present application provides a GPU-based automatic factor mining method. Figure 1 This is a flowchart of the GPU-based automatic factor mining method. The specific implementation process is described as follows.

[0058] Step S1, obtaining training data through the CPU and preprocessing the training data. In the embodiment of the present application, the training data are mainly three-dimensional financial data.

[0059] Step S2, loading the training data into the GPU, and grouping the preprocessed training data according to preset grouping rules to obtain grouped data. After reading a large amount of three-dimensional training data, the data is preprocessed first, and then grouped according to the rules set by the researchers, such as grouping according to grouping columns.

[0060] Specifically, during preprocessing, the training data is first cleaned to remove erroneous, duplicate or inconsistent data in the data set; the cleaned training data is then converted into a format suitable for analysis; new features are created or existing features are modified based on the data in a specific format to improve the performance of the model; the feature data is then standardized or normalized, and finally the data set is divided into a training set, a validation set and a test set to evaluate the performance of the model; if there is time series data in the training data, time alignment, sliding window feature extraction or time decomposition may be required for the time series data.

[0061] After preprocessing is completed, if the input data is three-dimensional data, it is necessary to group the data according to the grouping column. For example, if the input data is three-dimensional data [product*time*feature], and the grouping column is product, then after the grouping is completed, the data of the same product will be stored continuously.

[0062] Step S3, obtaining the genetic algorithm model generated in the CPU and transferring it to the GPU, and running the genetic algorithm model by the GPU to perform factor mining training on the grouped data, and when the genetic algorithm model converges, a factor prediction model is obtained.

[0063] This application completes the steps of formula generation and mutation of genetic algorithm on the CPU side, migrates the fitness calculation of the formula to the GPU side, and then executes the genetic algorithm on the GPU to process the input training data in and out, automatically mining the factors in the data. Figure 2 ,The specific mining process is described as follows.

[0064] Step a1, obtain the pre-set training parameters, which include but are not limited to the initial population size, number of iterations, subtree generation strategy, etc. These training parameters are determined by researchers based on experience or previous data processing. Among them, the population size refers to the total number of individuals (candidates for solutions) in the genetic algorithm, and the population size will affect the algorithm's search ability and convergence speed; the number of iterations defines the number of generations executed by the genetic algorithm, and each generation will generate new individuals and evaluate them; in the genetic algorithm, especially when dealing with the representation of tree structures (such as genetic programming), the subtree generation strategy determines how to generate individuals in the initial population or generate new subtrees in the mutation operation.

[0065] In this application, common subtree generation strategies include:

[0066] Growth Method, starting from a single node, gradually adding child nodes until the maximum tree depth is reached;

[0067] Full Growth Method, which generates a fully expanded tree structure from root to leaves;

[0068] Half-and-Half Method: half of the individuals in the population are generated using the full-growth method, and the other half using the growth method.

[0069] Random generation, where nodes and child nodes are randomly selected until a specific tree size or depth is reached.

[0070] Step a2, performing population initialization processing on the grouped data according to the population size, generating initial individuals corresponding to the population size. In the process of initializing the population, assuming that the population size is n, n different initial individuals are randomly generated, and these individuals are all formulas in the form of "add(a, sub(b, c))", which can also be called factors.

[0071] Step a3, calculating the fitness index of each of the initial individuals in the GPU. Figure 3 , send the initial individual to the GPU. If there are many initial individuals, divide the initial individual evenly into multiple sub-GPU modules, and then perform subsequent operations on each sub-GPU module. Specifically, convert the formula corresponding to the initial individual into the reverse order of the prefix expression to obtain an execution queue, which includes data operators and function operators; the GPU reads the execution queue in sequence and performs calculations to obtain the calculation results of the initial individual. Specifically, in the process of reading the execution queue for calculation, each operator is read to determine whether it reaches the end of the expression. If so, it indicates that the calculation is over; if not, determine whether the current expression is a data loading operator (i.e., a data operator). If it is a data operator, the data operator is loaded into the GPU. If it is a function operator, the GPU performs corresponding calculations according to the function operator until the execution queue is read and the calculation results are obtained. Finally, obtain the preset fitness function and fitting value, use the calculation result and fitting value as the input of the fitness, and obtain the fitness index.

[0072] In a preferred embodiment, if there are multiple GPUs, the training data will be evenly divided among the GPUs. Then the formula conversion is performed, that is, the individual formula is converted into the reverse order of the prefix expression, and it is used as the execution queue. For example, the formula "add(a, sub(b, c))" will be converted into the execution queue of "cb sub a add", where a, b, c can be called data loading operators, and sub and add can be called function operators. Finally, the formula and fitness index are calculated, and each GPU sequentially accesses the generated execution queue. If it is a data loading operator, the operator is loaded to the current GPU; if it is a function operator, the calculation operation is completed on the current GPU. It is worth noting that if the input data is three-dimensional data grouped according to the grouping column, the current GPU will only calculate within the same group when performing calculations. After obtaining the formula calculation result, the preset fitness function is called, and the fitness index is calculated with the fitting value, where the fitting value is a parameter used for model fitting, and the fitting value is set by the user. Common fitness functions include RankIC, MSE, etc. Taking RankIC as an example, the fitness index a is equal to RankIC (formula result, fitting value). When multiple GPUs are used for training, the final fitness index is the average of the fitness index of each GPU.

[0073] Step a4, according to the fitness index, the individuals are mutated and selected through the elite strategy as the parent individuals of the next iteration, and the parent individuals are processed and generated through the subtree generation strategy. In the process of individual evolution and mutation, the elite strategy is first adopted to pass the N individuals with the highest fitness to the next generation, and the parameter N can also be set by the staff. Then, according to the fitness index, the individuals with better fitness are selected as the parents of the next generation, and then new individuals are generated through crossover, mutation, etc.

[0074] In a preferred embodiment, the new individuals are generated by the subtree generation strategy, and first, several initial individuals with fitness higher than a preset value are selected as parent individuals; then a pair of parent individuals are randomly selected for a crossover operation to generate offspring individuals containing the characteristics of the two parent individuals; then a preset probability is obtained, and a mutation operation is performed on the newly generated offspring individuals with the preset probability to introduce new genetic diversity to obtain the new individuals.

[0075] Step a5, determine whether the iterative termination condition is reached. If so, the new individual is used as a factor prediction model and output; if not, return to step a3, and calculate the fitness with the new individual as input. In this embodiment, after each cycle, it is necessary to determine whether the termination condition is reached. The predetermined termination condition can be, for example, the number of iterations, the fitness threshold, etc., to determine whether to end the algorithm, for example, when the number of iterations is greater than or equal to the preset threshold; or, when the fitness is not within the preset fitness threshold range, the genetic algorithm model is judged to converge. If the termination condition has been reached, start step a3, otherwise the new individual obtained at the end is output, and after the training is completed, verification data and test data can be obtained for verification. After the verification is passed, the factor prediction model can be obtained for predicting subsequent prediction data. In this embodiment, the factor prediction model can be composed of multiple factor formulas, that is, through the above-mentioned training process, multiple factor formulas with fitness that meets the requirements are obtained.

[0076] Step S4, obtaining the data to be predicted, and calculating the data to be predicted by the factor prediction model to obtain a prediction result. Specifically, the prediction data is obtained by the CPU, and the prediction data is loaded into the GPU, grouped according to a preset grouping rule to obtain prediction grouping data; a formula is selected from the factor prediction model according to the prediction grouping data; the formula and the prediction data are input into the GPU for calculation to obtain the prediction result.

[0077] In one embodiment, after the mining module automatically mines out the factors, these factors are recorded as factor prediction models and saved in the prediction module, which can be directly used for subsequent calculations, and the prediction module can be used to calculate the prediction results for the prediction data. First, the prediction data is loaded to the GPU side using the CPU. If the input data is three-dimensional data, the data needs to be grouped according to the grouping column. For example, if the three-dimensional data of the prediction data [product*time*feature] is used, and the grouping column is a product, then after the grouping is completed, the data of the same product will be stored continuously. Specify the formula to be predicted, that is, select any factor formula in the above-mentioned factor prediction model to process the prediction data to obtain the prediction result, and finally send the prediction result from the GPU to the memory and then output it.

[0078] In summary, the GPU-based automatic factor mining method provided by the embodiment of the present application completes the steps of formula generation and mutation of gplearn on the CPU side, and migrates the fitness calculation of the formula to the GPU side to improve training efficiency and reduce training time. In addition, the present invention effectively supports the training of three-dimensional data by automatically performing group-by-group calculations on the input data, so that factors can be mined from high-dimensional data. Finally, in the case of processing large-scale data, the present invention can distribute the training load to multiple GPUs, thereby supporting more efficient large-scale data training.

[0079] In a second aspect, the present application provides a GPU-based automatic factor mining system. Figure 4 The system mainly includes an input module, an automatic mining module, an output module and a prediction module. The input module is used to collect training data and prediction data, pre-process the training data, load the training data into the GPU, and group them according to preset grouping rules to obtain grouped data; the automatic mining module is used to obtain the genetic algorithm model generated in the CPU and transfer it to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data. When the genetic algorithm model converges, a factor prediction model is obtained; the prediction module is used to obtain the data to be predicted, calculate the data to be predicted through the factor prediction model, and obtain the prediction result; the output module is used to output the prediction result.

[0080] In summary, the existing gplearn framework provided by the present application faces the problem of low computational efficiency and difficulty in training three-dimensional data. The present invention is based on a GPU-accelerated factor automatic mining system, which aims to overcome the deficiencies of the prior art. First, the present invention completes the steps of formula generation and mutation of gplearn on the CPU side, and migrates the fitness calculation of the formula to the GPU side to improve training efficiency and reduce training time. In addition, the present invention effectively supports the training of three-dimensional data by automatically performing inter-group calculations on the input data, so that factors can be mined from high-dimensional data. Finally, in the case of processing large-scale data, the present invention can distribute the training load to multiple GPUs, thereby supporting more efficient large-scale data training.

[0081] It should be noted that the GPU-based automatic factor mining system provided in this embodiment is used to implement the above-mentioned implementation mode, and the descriptions that have been made will not be repeated. As used above, the terms "module", "unit", "sub-unit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the apparatus described in the above embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0082] In a third aspect, an embodiment of the present application provides an electronic device, Figure 5 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 5 As shown, the electronic device may include a processor 11 and a memory 12 storing computer program instructions.

[0083] Specifically, the processor 11 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0084] Among them, the memory 12 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 12 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 12 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 12 may be inside or outside a data processing device. In a specific embodiment, the memory 12 is a non-volatile memory. In a specific embodiment, the memory 12 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0085] The memory 12 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 11 .

[0086] The processor 11 reads and executes computer program instructions stored in the memory 12 to implement any one of the GPU-based automatic factor mining methods in the above embodiments.

[0087] In one embodiment, the electronic device may further include a communication interface 13 and a bus 10. Figure 5 As shown, the processor 11, the memory 12, and the communication interface 13 are connected via a bus 10 and communicate with each other.

[0088] The communication interface 13 is used to realize the communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 13 can also realize data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage and image / data processing workstations.

[0089] The bus 10 includes hardware, software or both, and couples the components of the electronic device to each other. The bus 10 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 10 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 10 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0090] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the GPU-based automatic factor mining method provided in the first aspect is implemented.

[0091] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0092] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the GPU-based automatic factor mining method provided in the first aspect.

[0093] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0094] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0095] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A GPU-based automatic factor mining method, characterized in that: include: Acquiring training data through a CPU and preprocessing the training data; The training data is loaded into the GPU, and the pre-processed training data is grouped according to a preset grouping rule to obtain grouped data; The genetic algorithm model generated in the CPU is obtained and transferred to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data, and when the genetic algorithm model converges, a factor prediction model is obtained; The data to be predicted is obtained, and the data to be predicted is calculated using the factor prediction model to obtain a prediction result.

2. The method according to claim 1, characterized in that The GPU runs the genetic algorithm model to perform factor mining training on the grouped data, including: Step a1, obtaining preset training parameters, wherein the training parameters include population size, preset number of iterations and subtree generation strategy; Step a2, performing population initialization processing on the grouped data according to the population quantity to generate initial individuals corresponding to the population quantity; Step a3, calculating the fitness index of each of the initial individuals in the GPU; Step a4, selecting N initial individuals with the highest fitness as the parent individuals of the next iteration through the elite strategy, and generating new individuals through the subtree generation strategy; Step a5, judging whether the iteration termination condition is reached, if so, taking the new individual as the factor prediction model and outputting it; if not, returning to step a3, and calculating the fitness with the new individual as input.

3. The method according to claim 2, characterized in that The calculation of the fitness index of each initial individual includes: Sending the initial individual to the GPU; Convert the formula corresponding to the initial individual into the reverse order of the prefix expression to obtain an execution queue, wherein the execution queue includes a data operator and a function operator; The GPU reads the execution queue in sequence and performs calculations to obtain calculation results of the initial individuals; A preset fitness function and a fitting value are obtained, and the calculation result and the fitting value are used as inputs of the fitness to obtain the fitness index.

4. The method according to claim 1, characterized in that: The step of calculating the data to be predicted by using the factor prediction model to obtain a prediction result includes: Acquire prediction data through the CPU, load the prediction data into the GPU, group the data according to preset grouping rules, and obtain prediction grouping data; Selecting a formula from the factor prediction model according to the predicted grouping data; The formula and the prediction data are input into the GPU for calculation to obtain the prediction result.

5. The method according to claim 2, characterized in that: The training parameters also include a preset threshold and a fitness threshold range, and judging whether the genetic algorithm model converges includes: When the number of iterations is greater than or equal to a preset threshold; or when the fitness is not within a preset fitness threshold range, it is determined that the genetic algorithm model has converged.

6. The method according to claim 3, characterized in that The generating a new individual by the subtree generation strategy comprises: Randomly select a pair of parent individuals for crossover operation to generate offspring individuals containing the characteristics of the two parent individuals; A preset probability is obtained, and a mutation operation is performed on the newly generated offspring individual with the preset probability to introduce new genetic diversity to obtain the new individual.

7. The method according to claim 1, characterized in that The preprocessing of the training data comprises: Cleaning the training data to remove erroneous, duplicate or inconsistent data in the data set; Convert the cleaned training data into training data in a preset format; Create new feature data based on data in a preset format; The feature data is standardized or normalized to obtain preprocessed training data.

8. A GPU-based automatic factor mining system, characterized in that: The system mainly includes input module, automatic mining module, output module and prediction module; among them, The input module is used to collect training data and prediction data, pre-process the training data, load the training data into the GPU, and group the training data according to preset grouping rules to obtain grouped data; The automatic mining module is used to obtain the genetic algorithm model generated in the CPU and transfer it to the GPU, and the GPU runs the genetic algorithm model to perform factor mining training on the grouped data, and when the genetic algorithm model converges, a factor prediction model is obtained; The prediction module is used to obtain the data to be predicted, and calculate the data to be predicted through the factor prediction model to obtain the prediction result; The output module is used to output the prediction result.

9. An electronic device, characterized in that: The invention comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the GPU-based automatic factor mining method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the GPU-based automatic factor mining method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Factor mining system and method based on GPU (Graphics Processing Unit) and refinement type strong constraint

    CN122412118A

  • Factor mining system and method based on GPU and strong constraint of refinement type

    CN122412118B