Data compiling method and device, storage medium and electronic equipment

Abstract

The invention relates to a data compiling method and device, a storage medium and electronic equipment, and the data compiling method comprises the steps: for obtained to-be-compiled data, obtaining compilable data and non-compilable data in the to-be-compiled data based on a preset compiling feature mapping network; querying a compilation database based on the compilable data, determining that a compilation result corresponding to the compilable data does not exist in the compilation database, and running the non-compilable data to obtain a running result; performing asynchronous compiling on the compilable data to obtain a compiling result of the compilable data; and splicing the running result and the compiling result to obtain a compiling result of the to-be-compiled data, and obtaining an estimated value of the to-be-compiled data based on the compiling result of the to-be-compiled data. The compiling efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of compilation technology, and in particular, to a data compilation method, apparatus, storage medium, and electronic device. An asynchronous compilation strategy for an AI deep learning compiler Background Art

[0002] In an e-commerce system, in order to accurately perform data push, such as content push, to achieve a better content push effect, it is usually necessary to predict indicators such as the click-through rate (CTR) of the push content, so as to determine the content push decision based on the CTR value and the content push cost. In related technologies, in order to improve the accuracy of CTR value estimation, a deep neural network (DNN) model based on deep learning is usually used, and the compiler in the DNN model is used to compile the data input into the DNN model, and then operations are performed based on the compilation result to obtain the estimated value. However, in this method, when using the DNN model to obtain the estimated value, it is necessary to perform Just In Time (JIT) compilation on the data input into the DNN model. After the compilation is completed and the compilation result is obtained, the estimated value operation can be performed based on the compilation result. However, JIT compilation generally takes a long time, resulting in a high delay, low compilation efficiency, long time-consuming to obtain the estimated value, and low estimation efficiency. Summary of the Invention

[0003] In view of this, the present invention provides a data compilation method, apparatus, storage medium, and electronic device.

[0004] Specifically, the present invention is implemented through the following technical solutions:

[0005] According to a first aspect of the present invention, a data compilation method is provided. The data compilation method includes:

[0006] For the obtained data to be compiled, based on a pre-set compilation feature mapping network, obtain the compilable data and non-compilable data in the data to be compiled;

[0007] Query the compilation database based on the compilable data, determine that the compilation result corresponding to the compilable data does not exist in the compilation database, run the non-compilable data, and obtain the running result;

[0008] Asynchronously compile the compilable data to obtain the compilation result of the compilable data;

[0009] Concatenate the running result and the compilation result to obtain the compilation result of the data to be compiled, and obtain the estimated value of the data to be compiled based on the compilation result of the data to be compiled.

[0010] Optionally, obtaining the compilable data and non-compilable data in the data to be compiled based on the pre-set compilation feature mapping network includes:

[0011] Obtaining the to-be-compiled features corresponding to the data to be compiled based on the feature analysis module of the compilation feature mapping network;

[0012] Performing code mapping on the to-be-compiled features based on the feature mapping module of the compilation feature mapping network to obtain code features;

[0013] Matching the code features based on the rule sub-graph module of the compilation feature mapping network, and dividing the data to be compiled into compilable data and non-compilable data.

[0014] Optionally, the rule sub-graph module based on the compilation feature mapping network matches the code features and divides the data to be compiled into compilable data and non-compilable data, including:

[0015] Sequentially extracting the target code features in the code features, and if the target code features conform to the rules of the rule sub-graph module, obtaining the target compilation data in the data to be compiled corresponding to the target code features;

[0016] Concatenating the target compilation data corresponding to each target code feature to obtain the compilable data;

[0017] Removing the compilable data from the data to be compiled to obtain the non-compilable data.

[0018] Optionally, determining that there is no compilation result corresponding to the compilable data in the compilation database includes:

[0019] Querying whether the compilation database stores the data identifier according to the data identifier of the compilable data, and if not stored, determining that there is no compilation result corresponding to the compilable data.

[0020] Optionally, asynchronously compiling the compilable data to obtain the compilation result of the compilable data includes:

[0021] Converting the source code corresponding to the compilable data into intermediate code according to the pre-set code conversion strategy;

[0022] Optimizing the compilable data by using a high-level optimizer to obtain a high-level optimizer computation graph;

[0023] Compiling and performing binary conversion on the high-level optimizer computation graph by using a low-level virtual machine to obtain the compilation result.

[0024] Optionally, the method further includes:

[0025] Store the compilation result of the obtained compilable data in the compilation database.

[0026] Optionally, the method further includes:

[0027] Run the non-compilable data to obtain a running result;

[0028] Determine that the compilation result corresponding to the compilable data exists in the compilation database, and extract the compilation result corresponding to the compilable data;

[0029] Concatenate the running result and the compilation result to obtain the compilation result of the data to be compiled.

[0030] In the data compilation method of this technical solution, for the obtained data to be compiled, based on a pre-set compilation feature mapping network, the compilable data and non-compilable data in the data to be compiled are obtained; based on the compilable data, the compilation database is queried to determine that the compilation result corresponding to the compilable data does not exist in the compilation database, the non-compilable data is run to obtain a running result; the compilable data is asynchronously compiled to obtain the compilation result of the compilable data; the running result and the compilation result are concatenated to obtain the compilation result of the data to be compiled, and the estimated value of the data to be compiled is obtained based on the compilation result of the data to be compiled. In this way, by extracting the data to be compiled into non-compilable data and compilable data, for the compilable data, it is also possible to query whether the compilation result of the compilable data is stored in the compilation database, thereby avoiding the situation of low compilation efficiency caused by the need for compilation each time, effectively reducing the amount of data that needs to be compiled, and thus effectively improving the compilation efficiency.

[0031] According to a second aspect of the present invention, there is provided a data compilation device, the data compilation device including:

[0032] A data partitioning module, configured to obtain the compilable data and non-compilable data in the obtained data to be compiled based on a pre-set compilation feature mapping network;

[0033] A compilation result query module, configured to query the compilation database based on the compilable data, determine that the compilation result corresponding to the compilable data does not exist in the compilation database, run the non-compilable data, and obtain a running result;

[0034] A compilation module, configured to asynchronously compile the compilable data to obtain the compilation result of the compilable data;

[0035] The estimated value obtaining module is configured to splice the operation result and the compilation result to obtain the compilation result of the data to be compiled, and obtain the estimated value of the data to be compiled based on the compilation result of the data to be compiled.

[0036] According to a third aspect of the present invention, there is provided a storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the data compilation method in any possible implementation manner of the first aspect are implemented.

[0037] According to a fourth aspect of the present invention, there is provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the data compilation method in any possible implementation manner of the first aspect are implemented. Description of the Drawings

[0038] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0040] Figure 1 It is a schematic flowchart of a data compilation method provided by an embodiment of the present invention;

[0041] Figure 2 It is another schematic flowchart of a data compilation method provided by an embodiment of the present invention;

[0042] Figure 3 It is a schematic diagram of a data compilation device provided by an embodiment of the present invention;

[0043] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0045] In the related art, when using a DNN model for prediction, it is necessary to first use the compiler in the DNN model to compile the data input into the DNN model. After obtaining the compilation result, operations are performed based on the compilation result to obtain the predicted value. However, since just-in-time compilation generally takes a long time, the compilation efficiency is low.

[0046] When a deep learning model (e.g., a DNN model) performs predicted value operations, multiple rounds of iterative operations are required. Or, the same data may need to be compiled multiple times. As the number of iterations increases, the DNN model structure becomes more and more complex, making the time required for each compilation longer, thus leading to an increase in latency. Moreover, the number of compilations is related to the number of features and the feature dimension of the DNN model. As the number of features and the feature dimension increase, the number of compilations shows an exponential growth, which will also cause the time required for compilation to increase sharply.

[0047] In this embodiment, for the case of obtaining a predicted value through multiple iterative operations, the compilation result corresponding to the first iteration has a relatively small impact on the predicted value. Therefore, a default compilation result is set in advance for the first iteration. When running this default compilation result, the compiler is called to compile the currently to-be-compiled data to obtain the compilation result for the next iteration for storage. In this way, when subsequent compilation based on this currently to-be-compiled data is required, the corresponding compilation result can be directly obtained from the stored result, thereby solving the problems of high service latency and low compilation efficiency during online prediction of a deep neural network model through an asynchronous compilation method.

[0048] See Figure 1 , an embodiment of the present invention provides a data compilation method, which may include the following steps:

[0049] S101. For the obtained to-be-compiled data, based on a pre-set compilation feature mapping network, obtain the compilable data and non-compilable data in the to-be-compiled data;

[0050] In this embodiment, the user inputs the to-be-compiled data into the DNN model to obtain the predicted value of the to-be-compiled data. Taking obtaining the CTR predicted value as an example, as an optional embodiment, the to-be-compiled data includes, but is not limited to: the business type to which the content belongs, the user group type for content placement, the display times of the content in each type of user group, the click times of the content by each type of user group, click timestamp information, the historical click behaviors of each type of user group, historical display times, etc. The DNN model performs operations based on the input to-be-compiled data to obtain the click-through rate predicted values of each type of user group after placing the content corresponding to the to-be-compiled data.

[0051] In this embodiment, as an alternative embodiment, based on a pre-set compilation feature mapping network, obtaining the compilable data and non-compilable data in the data to be compiled includes:

[0052] Based on the feature analysis module of the compilation feature mapping network, obtaining the compilation features corresponding to the data to be compiled;

[0053] Based on the feature mapping module of the compilation feature mapping network, performing code mapping on the compilation features to obtain code features;

[0054] Based on the rule sub-graph module of the compilation feature mapping network, matching the code features and dividing the data to be compiled into compilable data and non-compilable data.

[0055] In this embodiment, by using the compilation feature mapping network, through feature mapping, code mapping, and rule sub-graphing on the data to be compiled, the data to be compiled is split into compilable data and non-compilable data. In this way, by extracting the non-compilable data included in the data to be compiled, for example, the data to be compiled containing constant types, and only compiling the compilable data included in the data to be compiled, the compilation efficiency can be effectively improved.

[0056] In this embodiment, as an alternative embodiment, based on the rule sub-graph module of the compilation feature mapping network, matching the code features and dividing the data to be compiled into compilable data and non-compilable data includes:

[0057] Sequentially extracting the target code features in the code features, and if the target code feature conforms to the rules of the rule sub-graph module, obtaining the target compilation data in the data to be compiled corresponding to the target code feature;

[0058] Concatenating the target compilation data corresponding to each target code feature to obtain the compilable data;

[0059] Excluding the compilable data from the data to be compiled to obtain the non-compilable data.

[0060] In this embodiment, rules for compilable data are pre-set, and through the rule sub-graph module, each target code feature is respectively matched. If the target code feature conforms to the set rules, it is determined that the data to be compiled corresponding to the target code feature is compilable data. In this embodiment, regarding the setting of the rules for compilable data, specific references can be made to relevant technical literature, which are omitted here for detailed description.

[0061] In this embodiment, the compilable data includes, but is not limited to: Accelerated Linear Algebra (XLA) data. Among them, XLA data can obtain regions suitable for just-in-time compilation for compilation through graph optimization methods in subsequent processing flows.

[0062] S102. Query the compilation database based on the compilable data, determine that there is no compilation result corresponding to the compilable data in the compilation database, run the non-compilable data, and obtain a running result;

[0063] In this embodiment, by querying whether there is a corresponding compilation result in the compilation database, if it is determined that there is no compilation result corresponding to the compilable data in the compilation database, notify just-in-time compilation to perform asynchronous compilation on the compilable data, and at the same time execute the non-compilable data to reduce the time required for compilation. As an alternative embodiment, running the non-compilable data to obtain a running result includes: performing binary encoding on the non-compilable data to form a machine language encoding.

[0064] In this embodiment, as an alternative embodiment, determining that there is no compilation result corresponding to the compilable data in the compilation database includes:

[0065] According to the data identifier of the compilable data, query whether the compilation database stores the data identifier. If not, determine that there is no compilation result corresponding to the compilable data.

[0066] In this embodiment, before compiling the compilable data, the data identifier of the compilable data can be searched in the compilation database to determine whether the data identifier of the compilable data exists in the compilation database. If the data identifier of the compilable data exists in the compilation database, it can be determined that the compilable data has been compiled, and the compilation result can be directly obtained. If the data identifier of the compilable data does not exist, asynchronous compilation is performed on the compilable data.

[0067] In this embodiment, as an alternative embodiment, the data identifier includes, but is not limited to: the hash value, label, etc. of the compilable data.

[0068] In this embodiment, for the case where multiple iterations of compilation are required, as another alternative embodiment, determining that there is no compilation result corresponding to the compilable data in the compilation database, running the non-compilable data, and obtaining a running result includes:

[0069] Obtain the first iteration compilation result of the compilable data preset in the compilation database;

[0070] Run the non-compilable data to obtain a running result, and acquire the machine language code of the non-compilable data;

[0071] Run the first iteration compilation result and the machine language code that are concatenated.

[0072] In this embodiment, for the data to be compiled that requires multiple iterations, since the iteration is based on the result of the previous iteration, for the initial iteration, the initial compilation result corresponding to the compilable data does not exist in the compilation database. Since the initial compilation result has a relatively small impact on the iteration, therefore, an initial iteration compilation result can be preset in the compilation database, so that the initial iteration compilation result of the compilable data can be obtained. Based on the first iteration compilation result and the machine language code corresponding to the non-compilable data for running, at the same time, asynchronously compile the compilable data of the current iteration. After the asynchronous compilation is completed, store the compilation result of the asynchronous compilation in the compilation database. Thus, when the next iteration is performed, the corresponding compilation result can be directly obtained from the compilation database, and when the next iteration is performed, asynchronously compile the data to be compiled updated in the current iteration, and so on in a loop, without the need for compilation waiting, thereby improving the compilation efficiency of the iteration.

[0073] S103. Asynchronously compile the compilable data to obtain the compilation result of the compilable data;

[0074] In this embodiment, as an optional embodiment, asynchronously compiling the compilable data to obtain the compilation result of the compilable data includes:

[0075] According to a preset code conversion strategy, convert the source code corresponding to the compilable data into intermediate code;

[0076] Use a high-level optimizer to optimize the compilable data to obtain a high-level optimizer computation graph;

[0077] Use a low-level virtual machine to compile and perform binary conversion on the high-level optimizer computation graph to obtain the compilation result.

[0078] In this embodiment, compilation is performed based on the low-level virtual machine (LLVM, Low Level Virtual Machine) framework. Among them, the front end of the LLVM framework converts the source code corresponding to the compilable data into intermediate code, the high-level optimizer (HLO, High Level Optimizer) optimizes the intermediate code, and the low-level virtual machine (LLVM, Low Level Virtual Machine) at the back end compiles based on the intermediate code to generate a compilation result represented by machine code.

[0079] S104. Concatenate the operation result and the compilation result to obtain the compilation result of the data to be compiled, and obtain the estimated value of the data to be compiled based on the compilation result of the data to be compiled.

[0080] In this embodiment, concatenation is performed according to the positions of the non-compilable data and the compilable data in the data to be compiled respectively to obtain the compilation result of the data to be compiled represented by machine language encoding, and the compilation result of the data to be compiled is run, so that the estimated value of the data to be compiled can be obtained.

[0081] In this embodiment, as an alternative embodiment, the method further includes:

[0082] Store the obtained compilation result of the compilable data in the compilation database.

[0083] In this embodiment, after asynchronous compilation is completed, the compilation result is stored in the compilation database for future use. In this way, after asynchronous compilation of the compilable data is completed, a mapping relationship between the data identifier of the compilable data and the compilation result is constructed in the compilation database based on the data identifier of the compilable data. Thus, when the compilable data needs to be compiled subsequently, the corresponding compilation result can be directly obtained from the compilation database based on the data identifier of the compilable data, thereby improving the compilation efficiency of the data.

[0084] In this embodiment, as another alternative embodiment, the method further includes:

[0085] After running the non-compilable data, if the compilation result of the compilable data has not been obtained yet, send indication information for instructing the client that input the compilable data to perform other operations.

[0086] In this embodiment, after running the non-compilable data, if the compilation result of the compilable data has not been obtained, it indicates that the client needs to wait for the compilation result for the estimated value calculation. To improve the utilization efficiency of the client resources, by sending indication information to the client to indicate that the client does not need to wait and can perform other operations, for example, continue to input the data to be compiled, query the historical compilation results stored in the compilation database, query the waiting duration required for the current compilation, etc.

[0087] In this embodiment, as an alternative embodiment, after asynchronously compiling the compilable data and before obtaining the compilation result of the compilable data, the method further includes:

[0088] Return the compilation status information of the compilable data according to a preset time period.

[0089] In this embodiment, during the asynchronous mutation process, the compilation status information is returned so that the client can determine whether to perform other operations based on the returned compilation status information. The compilation status information is used to indicate the compilation status of compiling the current compilable data. For example, starting compilation, in the process of compilation, and ending compilation. As another alternative embodiment, or the compilation status information may also be the current compilation progress, the remaining time for compilation, etc.

[0090] In this embodiment, as another alternative embodiment, the method further includes:

[0091] Run the non-compilable data to obtain a running result;

[0092] Determine that the compilation database has the compilation result corresponding to the compilable data, and extract the compilation result corresponding to the compilable data;

[0093] Concatenate the running result and the compilation result corresponding to the compilable data to obtain the compilation result of the data to be compiled.

[0094] In this embodiment, check whether the compilation database has the compilation result corresponding to the compilable data. If the compilation result already exists, directly execute it. If not, asynchronously compile the compilable data and at the same time execute the non-compilable data for machine language encoding.

[0095] In this embodiment, for the data to be compiled obtained, based on the pre-set compilation feature mapping network, the compilable data and non-compilable data in the data to be compiled are obtained; based on the compilable data, query the compilation database to determine that the compilation database does not have the compilation result corresponding to the compilable data, run the non-compilable data to obtain a running result; asynchronously compile the compilable data to obtain the compilation result of the compilable data; concatenate the running result and the compilation result to obtain the compilation result of the data to be compiled, and obtain the estimated value of the data to be compiled based on the compilation result of the data to be compiled. In this way, before compiling the data to be compiled, the non-compilable data is extracted from the data to be compiled, which can effectively reduce the amount of data to be compiled, reduce the resource overhead of compilation, and effectively improve the compilation efficiency; further, for the compilable data, by querying whether the compilation result of the compilable data is stored in the compilation database, the situation of low compilation efficiency caused by the need for compilation each time can be avoided; moreover, after the compilation result of the compilable data is not stored in the compilation database, through asynchronous compilation and at the same time running the non-compilable data for machine language code generation, the time required for compilation can be effectively reduced, which can not only ensure that the service time consumption meets the requirements, but also ensure the full utilization of computing power, and realize the online estimation ability for complex deep models.

[0096] Figure 2Another flowchart of a data compilation method provided by an embodiment of the present invention. As Figure 2 shown, the method may include the following steps:

[0097] S201. Receive an inference request, parse the inference request, and obtain data to be compiled;

[0098] In this embodiment, the user initiates an inference request and carries the data to be compiled in the inference request.

[0099] S202. Extract the features of the data to be compiled;

[0100] S203. Perform code mapping on the extracted features to obtain code features;

[0101] In this embodiment, by using a compilation feature mapping network, feature mapping and code mapping are performed on the data to be compiled.

[0102] S204. Based on the code features and pre-set rules, divide the data to be compiled into compilable data and non-compilable data;

[0103] S205. Perform feature routing based on the compilable data and the non-compilable data;

[0104] S206. Determine whether to route to the compilation network. If so, execute step S207; if not, execute step S210;

[0105] S207. Obtain a hash value, query the compilation database based on the hash value to check if there is a hit. If there is a hit, execute step S211; if not, execute step S208;

[0106] S208. Start asynchronous compilation and perform compilation according to the compilation configuration;

[0107] S209. Obtain the compilation result and store the compilation result in the compilation database;

[0108] S210. Route the non-compilable data to the non-compilation network;

[0109] S211. Obtain the compilation result from the compilation database;

[0110] S212. Obtain the inference result based on the non-compilation network and the compilation result.

[0111] This embodiment provides an asynchronous compilation technology at runtime, which can be applied to all scenarios of model inference based on Graphics Processing Unit (GPU) devices.

[0112] Based on the same inventive concept, as Figure 3As shown in the figure, an embodiment of the present invention further provides a data compilation device, which includes:

[0113] A data division module 301, configured to obtain compilable data and non-compilable data in the to-be-compiled data based on a pre-set compilation feature mapping network for the obtained to-be-compiled data;

[0114] In this embodiment, by using the compilation feature mapping network, through feature mapping, code mapping and rule sub-graphing of the to-be-compiled data, the to-be-compiled data is split into compilable data and non-compilable data.

[0115] In this embodiment, as an optional embodiment, the data division module 301 includes:

[0116] A feature extraction unit (not shown in the figure), configured to obtain to-be-compiled features corresponding to the to-be-compiled data based on the feature analysis module of the compilation feature mapping network;

[0117] A mapping unit, configured to perform code mapping on the to-be-compiled features based on the feature mapping module of the compilation feature mapping network to obtain code features;

[0118] A data division unit, configured to match the code features based on the rule sub-graphing module of the compilation feature mapping network and divide the to-be-compiled data into compilable data and non-compilable data.

[0119] In this embodiment, as an optional embodiment, the data division unit is specifically configured to:

[0120] Sequentially extract target code features in the code features, if the target code features conform to the rules of the rule sub-graphing module, obtain target compilation data in the to-be-compiled data corresponding to the target code features;

[0121] Concatenate the target compilation data corresponding to each target code feature to obtain the compilable data;

[0122] Exclude the compilable data from the to-be-compiled data to obtain the non-compilable data.

[0123] A compilation result query module 302, configured to query a compilation database based on the compilable data, determine that there is no compilation result corresponding to the compilable data in the compilation database, run the non-compilable data, and obtain a running result;

[0124] In this embodiment, as an optional embodiment, the compilation result query module 302 includes:

[0125] A query unit, configured to query the compilation database based on the compilable data;

[0126] A judgment unit, configured to query whether the compilation database stores the data identifier according to the data identifier of the compilable data. If not stored, it is determined that there is no compilation result corresponding to the compilable data.

[0127] An operation unit, configured to operate the non-compilable data to obtain an operation result.

[0128] In this embodiment, as an alternative embodiment, the operation unit is specifically configured to: perform binary encoding on the non-compilable data to form a machine language encoding.

[0129] In this embodiment, as an alternative embodiment, the data identifier includes but is not limited to: the hash value, label, etc. of the compilable data.

[0130] A compilation module 303, configured to perform asynchronous compilation on the compilable data to obtain a compilation result of the compilable data.

[0131] In this embodiment, as an alternative embodiment, the compilation module 303 includes:

[0132] A code conversion unit, configured to convert the source code corresponding to the compilable data into intermediate code according to a preset code conversion strategy.

[0133] A code optimization unit, configured to optimize the compilable data by using a high-level optimizer to obtain a high-level optimizer computation graph.

[0134] A code compilation unit, configured to compile and perform binary conversion on the high-level optimizer computation graph by using a low-level virtual machine to obtain the compilation result.

[0135] In this embodiment, as another alternative embodiment, the compilation module 303 further includes:

[0136] A status notification unit, configured to return the compilation status information of the compilable data according to a preset time period.

[0137] In this embodiment, during the process of asynchronous mutation, the compilation status information is returned, so that the client determines whether to perform other operations according to the returned compilation status information.

[0138] An estimated value acquisition module 304, configured to splice the operation result and the compilation result to obtain a compilation result of the data to be compiled, and obtain an estimated value of the data to be compiled based on the compilation result of the data to be compiled.

[0139] In this embodiment, splicing is performed according to the positions of the non-compilable data and the compilable data in the data to be compiled respectively, to obtain a compilation result of the data to be compiled represented by machine language encoding.

[0140] In this embodiment, as an alternative embodiment, the device further includes:

[0141] A compilation result storage module (not shown in the figure), configured to store the compilation result of the obtained compilable data in the compilation database.

[0142] In this embodiment, as another alternative embodiment, the device further includes:

[0143] A compilation splicing module, configured to run the non-compilable data to obtain a running result; determine that the compilation database has the compilation result corresponding to the compilable data, and extract the compilation result corresponding to the compilable data; splice the running result of running the non-compilable data and the extracted compilation result corresponding to the compilable data to obtain the compilation result of the to-be-compiled data.

[0144] In this embodiment, as yet another alternative embodiment, the device further includes:

[0145] An operation prompt module, configured to, after running the non-compilable data, if the compilation result of the compilable data has not been obtained yet, send indication information for instructing the client that inputs the compilable data to perform other operations to the client.

[0146] Based on the same inventive concept, an embodiment of the present invention further provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the data compilation method in any of the above possible implementation manners are implemented.

[0147] Optionally, the storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0148] Based on the same inventive concept, refer to Figure 4 , an embodiment of the present invention further provides an electronic device, including a memory 101 (such as a non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, the steps of the data compilation method in any of the above possible implementation manners are implemented, which is equivalent to the data compilation device as described above. Of course, the processor may also be used to process other data or perform operations. The electronic device may be a device such as a PC, a server, a terminal, etc.

[0149] As Figure 4 shown, the electronic device generally may further include: a memory 103, a network interface 104, and an internal bus 105. In addition to these components, other hardware may also be included, which will not be elaborated herein.

[0150] It should be noted that the above data compilation device can be implemented by software. As a logically meaningful device, it is formed by the processor 102 of the electronic device where it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 and running them.

[0151] Embodiments of the subject matter and functional operations described in this specification can be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules in computer program instructions encoded on a tangible non-transitory program carrier to be executed by a data processing device or to control the operation of a data processing device. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for execution by the data processing device. A computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0152] The processes and logical flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logical flows can also be performed by dedicated logic circuits - such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), and the apparatus can also be implemented as dedicated logic circuits.

[0153] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Generally, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, etc., or the computer will be operatively coupled to such mass storage devices to receive data from them or transmit data to them, or both. However, a computer is not necessarily required to have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name just a few.

[0154] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0155] Although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of what is claimed, but rather as mainly being used to describe the features of specific embodiments of a particular invention. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Additionally, although features may operate in certain combinations and even be initially claimed as such, one or more features from a claimed combination may in some cases be removed from that combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

[0156] Similarly, although operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or sequentially, or that all illustrated operations be performed, to achieve the desired result. In certain cases, multitasking and parallel processing may be advantageous. Additionally, the separation of the various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0157] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the acts recited in the claims may be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the drawings are not necessarily in the particular order or sequential order shown to achieve the desired result. In certain implementations, multitasking and parallel processing may be advantageous.

[0158] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0159] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.

Claims

1. A data compilation method, characterized in that, Including: For the to-be-compiled data obtained, based on a pre-set compilation feature mapping network, obtain the compilable data and non-compilable data in the to-be-compiled data; Query the compilation database based on the compilable data, determine that there is no compilation result corresponding to the compilable data in the compilation database, run the non-compilable data, and obtain a running result; Asynchronously compile the compilable data to obtain the compilation result of the compilable data; Concatenate the running result and the compilation result to obtain the compilation result of the to-be-compiled data, and obtain the estimated value of the to-be-compiled data based on the compilation result of the to-be-compiled data.

2. The data compilation method according to claim 1, wherein The obtaining the compilable data and non-compilable data in the to-be-compiled data based on a pre-set compilation feature mapping network includes: Based on the feature analysis module of the compilation feature mapping network, obtain the to-be-compiled feature corresponding to the to-be-compiled data; Based on the feature mapping module of the compilation feature mapping network, perform code mapping on the to-be-compiled feature to obtain a code feature; Based on the rule sub-graph module of the compilation feature mapping network, match the code feature, and divide the to-be-compiled data into compilable data and non-compilable data.

3. The data compilation method according to claim 2, wherein The matching the code feature by the rule sub-graph module of the compilation feature mapping network and dividing the to-be-compiled data into compilable data and non-compilable data includes: Sequentially extract the target code features in the code feature. If the target code feature conforms to the rules of the rule sub-graph module, obtain the target compilation data in the to-be-compiled data corresponding to the target code feature; Concatenate the target compilation data corresponding to each target code feature to obtain the compilable data; Exclude the compilable data from the to-be-compiled data to obtain the non-compilable data.

4. The data compilation method according to claim 1, wherein The determining that there is no compilation result corresponding to the compilable data in the compilation database includes: According to the data identifier of the compilable data, query whether the compilation database stores the data identifier. If not stored, determine that there is no compilation result corresponding to the compilable data.

5. The data compilation method according to claim 1, wherein The asynchronously compiling the compilable data to obtain the compilation result of the compilable data includes: According to a pre-set code conversion strategy, convert the source code corresponding to the compilable data into intermediate code; Use a high-level optimizer to optimize the compilable data to obtain a high-level optimizer computation graph; Use a low-level virtual machine to compile and perform binary conversion on the high-level optimizer computation graph to obtain the compilation result.

6. The data compilation method according to any one of claims 1 to 5, characterized in that The method further includes: Store the obtained compilation result of the compilable data into the compilation database.

7. The data compilation method according to any one of claims 1 to 5, characterized in that, The method further includes: Determine that there is a compilation result corresponding to the compilable data in the compilation database, and extract the compilation result corresponding to the compilable data.

8. A data compilation device, characterized in that, The data compilation device includes: A data division module, configured to, for the to-be-compiled data obtained, based on a pre-set compilation feature mapping network, obtain the compilable data and non-compilable data in the to-be-compiled data; A compilation result query module, configured to query a compilation database based on the compilable data, determine that there is no compilation result corresponding to the compilable data in the compilation database, run the non-compilable data, and obtain a running result; A compilation module, configured to asynchronously compile the compilable data and obtain a compilation result of the compilable data; An estimated value obtaining module, configured to splice the running result and the compilation result to obtain a compilation result of the data to be compiled, and obtain an estimated value of the data to be compiled based on the compilation result of the data to be compiled.

9. A storage medium, characterized in that, A program or instruction is stored on a storage medium, and when the program or instruction is run by a processor, the steps of the data compilation method according to any one of claims 1 to 7 are implemented.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps of the data compilation method according to any one of claims 1 to 7 are implemented.

Images