Data Processing Method, Apparatus, Computer Device, and Medium
By generating feature calculation graphs in deep learning and using pre-trained neural networks, the inefficiency problem caused by feature engineering code differences is solved, and the training and inference process is integrated, which improves data processing efficiency and accuracy.
Patent Information
- Application Number
- CN202111244924.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-26
- Publication Date
- 2025-07-22
- Estimated Expiration
- 2041-10-26
AI Technical Summary
During the training and inference process of deep learning, large differences in feature engineering codes lead to low data processing efficiency in the inference process, and it is necessary to rewrite the code, affecting efficiency and accuracy.
By pre-generating feature calculation graphs, based on the correlation between multiple feature parameters of structured data, a pre-trained data processing neural network is used to determine the data processing results of data prediction requests, and the integration of training and inference processes is realized.
It reduces the amount of calculations for actual data prediction analysis, improves the efficiency and accuracy of the data processing process, and reduces the risk of development time and code errors.
Smart Images

Figure CN113886614B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of machine learning technology, and particularly to a data processing method, apparatus, computer device, and medium. Background Art
[0002] As a technology that simulates the deep abstract cognitive process of the human brain to achieve complex operations and optimizations of data by a computer, deep learning has been widely applied in various types of industries. A complete deep learning framework includes two main parts, one is training, and the other is inference. And both the training part and the inference part include two main processes: feature engineering and model calculation.
[0003] Among them, feature engineering, as an engineering activity that can extract features from data to the greatest extent for model use, is an important part of the deep learning framework. In related technologies, it is mainly the relevant technical personnel who write code to decompose multiple complex problems in feature engineering into individual processing processes through data pipeline (Pipeline) plugins in the code, and then process each processing process one by one.
[0004] However, the amount of data to be processed in the training process and the inference process varies greatly, which makes the feature engineering code involved in the training process quite different from the feature engineering code involved in the inference process. Therefore, in the inference process, code needs to be rewritten, resulting in low data processing efficiency in the inference process. Summary of the Invention
[0005] To overcome the problems existing in related technologies, this specification provides a data processing method, apparatus, computer device, and medium.
[0006] According to the first aspect of the embodiments of this specification, a data processing method is provided, and the method includes:
[0007] Obtain a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs, where each feature calculation graph is pre-generated based on the association relationship between multiple feature parameters involved in structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of target structured data;
[0008] Based on the target feature calculation graph and the target structured data, use a pre-trained data processing neural network to determine the data processing result corresponding to the data prediction request.
[0009] Combined with any implementation manner provided by this disclosure, pre-generating a feature calculation graph based on the association relationship between multiple feature parameters involved in structured data includes:
[0010] Regarding multiple characteristic parameters involved in structured data, respectively use them as multiple nodes of a feature calculation graph;
[0011] For any two nodes among the multiple nodes, when there is an association relationship between the two characteristic parameters corresponding to the two nodes, create an edge in the feature calculation graph to connect the two nodes, so as to indicate the association relationship between the two characteristic parameters corresponding to the two nodes connected by the edge.
[0012] Combined with any implementation manner provided by the present disclosure, obtaining a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs includes:
[0013] Determine at least one target characteristic parameter involved in the target structured data;
[0014] Based on at least one target characteristic parameter, determine, from multiple feature calculation graphs, a target feature calculation graph including the corresponding nodes of at least one target characteristic parameter.
[0015] Combined with any implementation manner provided by the present disclosure, the target structured data includes data corresponding to multiple target characteristic parameters respectively; based on the target feature calculation graph and the target structured data, use a pre-trained data processing neural network to determine a data processing result corresponding to the data prediction request, including:
[0016] Input the target structured data into the pre-trained data processing neural network, and through the pre-trained data processing neural network, based on the dimensionality characteristics of the target structured data indicated by the target feature calculation graph, process the data corresponding to each target characteristic parameter in the target structured data to obtain a data processing result corresponding to the data prediction request.
[0017] Combined with any implementation manner provided by the present disclosure, the training process of the data processing neural network includes:
[0018] Obtain sample data, where the sample data is structured data involving multiple sample characteristic parameters and is labeled with a sample processing result corresponding to the sample data;
[0019] Input the sample data into the initial neural network, and through the initial neural network, based on the dimensionality characteristics of the sample data indicated by the target feature calculation graph, process the data corresponding to each sample characteristic parameter in the sample data to obtain a prediction result corresponding to the sample data;
[0020] Based on a loss function that can characterize the difference between the prediction result and the sample processing result, train the initial neural network until a preset training completion condition is met to obtain a data processing neural network.
[0021] In combination with any of the embodiments provided in the present disclosure, different association relationships correspond to different algorithms, and the algorithms corresponding to the association relationships are encapsulated as computing tools, and different association relationships correspond to different computing tools;
[0022] Input the sample data into the initial neural network. Through the initial neural network, calculate the dimensional features of the sample data indicated by the target feature calculation graph based on the target features, and process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data, including:
[0023] Based on the association relationship between multiple sample feature parameters involved in the sample data, determine the computing tools involved in the target feature calculation graph;
[0024] Input the sample data into the initial neural network. Through the initial neural network, use the computing tools involved in the target feature calculation graph to process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data.
[0025] In combination with any of the embodiments provided in the present disclosure, the data processing result includes at least one of the following:
[0026] The processing result obtained by analyzing based on the product sales data;
[0027] The processing result obtained by analyzing based on the user behavior data;
[0028] The processing result obtained by analyzing the vehicle trajectory.
[0029] According to the second aspect of the embodiments of the present specification, a data processing device is provided. The device includes:
[0030] An acquisition unit, configured to acquire a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs, where each feature calculation graph is pre-generated based on the association relationship between multiple feature parameters involved in the structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of the target structured data;
[0031] A result determination unit, configured to determine the data processing result corresponding to the data prediction request based on the target feature calculation graph and the target structured data by using a pre-trained data processing neural network.
[0032] In combination with any of the embodiments provided in the present disclosure, the device further includes:
[0033] A generation unit, configured to pre-generate a feature calculation graph based on the association relationship between multiple feature parameters involved in the structured data;
[0034] When used to pre-generate a feature calculation graph based on the association relationships between multiple feature parameters involved in structured data, the generation unit is specifically configured to:
[0035] Take the multiple feature parameters involved in the structured data as multiple nodes of the feature calculation graph respectively; for any two nodes among the multiple nodes, when there is an association relationship between the two feature parameters corresponding to the two nodes, create an edge in the feature calculation graph to connect the two nodes, so as to indicate the association relationship between the two feature parameters corresponding to the two nodes connected by the edge.
[0036] Combined with any one of the embodiments provided in the present disclosure, when used to obtain a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs, the acquisition unit is specifically configured to:
[0037] Determine at least one target feature parameter involved in the target structured data;
[0038] Based on at least one target feature parameter, determine, from multiple feature calculation graphs, a target feature calculation graph including the corresponding nodes of at least one target feature parameter.
[0039] Combined with any one of the embodiments provided in the present disclosure, the target structured data includes data respectively corresponding to multiple target feature parameters;
[0040] When used to determine a data processing result corresponding to a data prediction request based on a target feature calculation graph and target structured data by using a pre-trained data processing neural network, the result determination unit is specifically configured to:
[0041] Input the target structured data into the pre-trained data processing neural network, and through the pre-trained data processing neural network, process the data respectively corresponding to each target feature parameter in the target structured data based on the dimensionality features of the target structured data indicated by the feature calculation graph, so as to obtain a data processing result corresponding to the data prediction request.
[0042] Combined with any one of the embodiments provided in the present disclosure, when training the data processing neural network, the acquisition unit is further configured to acquire sample data, where the sample data is structured data involving multiple sample feature parameters and is labeled with a sample processing result corresponding to the sample data;
[0043] The device further includes:
[0044] A processing unit, configured to input the sample data into an initial neural network, and through the initial neural network, process the data respectively corresponding to each sample feature parameter in the sample data based on the dimensionality features of the sample data indicated by the target feature calculation graph, so as to obtain a prediction result corresponding to the sample data;
[0045] A training unit, configured to train an initial neural network based on a loss function that can characterize the difference between a prediction result and a sample processing result until a preset training completion condition is satisfied, thereby obtaining a data processing neural network.
[0046] Combined with any of the embodiments provided in the present disclosure, different association relationships correspond to different algorithms, and the algorithms corresponding to the association relationships are encapsulated as computing tools, and different association relationships correspond to different computing tools;
[0047] When the processing unit is configured to input sample data into the initial neural network, and through the initial neural network, based on the target feature calculation graph, calculate the dimensional features of the sample data, and process the data corresponding to each sample feature parameter in the sample data to obtain a prediction result corresponding to the sample data, it is specifically configured to:
[0048] Determine the computing tools involved in the target feature calculation graph based on the association relationship between multiple sample feature parameters involved in the sample data;
[0049] Input the sample data into the initial neural network, and through the initial neural network, use the computing tools involved in the target feature calculation graph to process the data corresponding to each sample feature parameter in the sample data to obtain a prediction result corresponding to the sample data.
[0050] Combined with any of the embodiments provided in the present disclosure, the data processing result includes at least one of the following:
[0051] A processing result obtained by analyzing product sales data;
[0052] A processing result obtained by analyzing user behavior data;
[0053] A processing result obtained by analyzing vehicle trajectories.
[0054] According to the third aspect of the embodiments of the present specification, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the operations performed by the above data processing method are implemented.
[0055] According to the fourth aspect of the embodiments of the present specification, a computer-readable storage medium is provided. A program is stored on the computer-readable storage medium, and when the program is executed by the processor, the operations performed by the above data processing method are implemented.
[0056] According to the fifth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the operations performed by the above data processing method are implemented.
[0057] The technical solutions provided by the embodiments of this specification may include the following beneficial effects:
[0058] In the embodiments of this specification, a target feature calculation graph corresponding to a data prediction request is obtained from multiple feature calculation graphs, where each feature calculation graph is pre-generated based on the association relationship between multiple feature parameters of structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of target structured data; based on the target feature calculation graph and the target structured data, a pre-trained data processing neural network is used to determine the data processing result corresponding to the data prediction request. By first determining the prediction direction based on the feature parameters involved in the target structured data and then performing prediction analysis based on the specific data corresponding to the feature parameters, the amount of calculation required for actual data prediction analysis can be effectively reduced, and the efficiency of the data processing process can be correspondingly improved.
[0059] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this specification. Brief Description of the Drawings
[0060] The drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with this specification, and are used together with the specification to explain the principles of this specification.
[0061] Figure 1 is a flowchart of a data processing method shown according to an exemplary embodiment of this specification.
[0062] Figure 2 is a schematic diagram of a feature calculation graph shown according to an exemplary embodiment of this specification.
[0063] Figure 3 is a flowchart of a data processing neural network training process and an inference process shown according to an exemplary embodiment of this specification.
[0064] Figure 4 is a block diagram of a data processing device shown according to an exemplary embodiment of this specification.
[0065] Figure 5 is a schematic diagram of the structure of a computer device shown according to an exemplary embodiment of this specification. Detailed Description of the Embodiments
[0066] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the present disclosure.
[0067] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the", and "said" used in the present disclosure are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
[0068] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".
[0069] The present disclosure provides a data processing method for processing structured data. Among them, structured data is data logically expressed and implemented by a two-dimensional table structure, including product sales data, user behavior data, vehicle trajectories, and so on. That is, the data processing method provided by the present disclosure can be applied to various products that require structured data modeling. For example, product sales data in the To Business (To B) scenario, user behavior prediction in the To Customer (ToC) scenario, vehicle trajectory restoration (such as city-level vehicle trajectory restoration), and so on.
[0070] The above is only an exemplary description of the application scenarios of the present disclosure and does not constitute a limitation on the application scenarios of the present disclosure. In more possible implementation manners, the present disclosure can be applied to various other data processing processes involving structured data.
[0071] The above data processing method can be executed by a computer device. The computer device can be a server, such as a single server, multiple servers, a server cluster, a cloud computing platform, and so on. Optionally, the computer device can also be a terminal device, such as a mobile phone, a tablet computer, a game console, a portable computer, a desktop computer, an advertising machine, an all-in-one computer, and so on. The present disclosure does not limit the specific type of the computer device.
[0072] The above is the relevant introduction to the application scenarios of the present disclosure. Next, in combination with the embodiments of this specification, the data processing method provided by the present disclosure will be described in detail.
[0073] As Figure 1 shown, Figure 1 is a flowchart of a data processing method shown in accordance with an exemplary embodiment of this specification. The method includes the following steps:
[0074] Step 101, obtain a target feature calculation graph corresponding to a data prediction request from a plurality of feature calculation graphs, where each feature calculation graph is pre-generated based on the association relationship between a plurality of feature parameters involved in structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of target structured data.
[0075] Among them, the data prediction request can be triggered based on a data prediction operation of a user on a computer device, or can be automatically generated by the computer device when obtaining target structured data to be subjected to prediction processing (hereinafter also simply referred to as data to be predicted). The present disclosure does not limit this. The data prediction request is used to perform prediction processing on the target structured data to be predicted based on the request to obtain a corresponding data processing result.
[0076] Among them, the data prediction request may carry the target structured data to be predicted, or the data prediction request may carry the data identifier of the target structured data to be predicted, so that the computer device can obtain the target structured data to be predicted based on the data identifier. And the target structured data to be predicted involves a plurality of feature parameters and includes the data corresponding to each feature parameter. Specifically, structured data can be understood as data organized by feature parameters, such as key-value data. In structured data, the number of feature parameters (i.e., the number of keys) can be understood as the number of data dimensions involved in the overall structured data (the so-called data dimensions can include, for example, identity, gender, behavior preferences, etc.), and the data corresponding to each feature parameter (i.e., the value corresponding to the key) can be understood as the value taken on the data dimension characterized by the feature parameter (for example, student - female - musical, or employee - male - outdoor sports, etc.). And the value range of the feature parameter is restricted, or rather, different values of the same feature parameter share specific attributes. In addition, in this article, for the sake of simplicity of expression, the data corresponding to the feature parameter may also be referred to as the feature parameter value.
[0077] Taking the target structured data to be predicted as user behavior data as an example, the characteristic parameters involved in user behavior data include user identification information, user attribute information (such as gender, age, etc.), user behavior preference information (such as preferred Internet access time, preferred news types when browsing news, etc.), and so on. Correspondingly, user behavior data can specifically include data corresponding to user identification (hereinafter also simply referred to as user identification, such as mobile phone number or ID card number), data corresponding to user attribute information (such as male, 38 years old, etc.), data corresponding to user behavior preference information (such as from 15:00 to 18:00, sports news, etc.), and so on.
[0078] By structuring the data based on the characteristic parameters as above to obtain structured data, the following becomes possible: First, based on the characteristic parameters involved in the structured data, the dimensional characteristics of the structured data (Dimensional Characteristic, such as related to user consumption behavior, involving user driving habits, etc.) can be preliminarily predicted; Second, based on the characteristic parameter values included in the structured data, the individual characteristics of the structured data (Individual Characteristic, such as the user likes to order takeout, the user is used to avoiding peak hours when traveling, etc.) can be relatively accurately determined.
[0079] Based on the above original discovery, the present disclosure innovatively proposes that when receiving a data prediction request from a user, the prediction direction can be initially determined based on the characteristic parameters involved in the structured data corresponding to the data prediction request, and then data prediction analysis can be performed in the prediction direction based on the characteristic parameter values included in the structured data corresponding to the data prediction request. By first determining the prediction direction based on the involved characteristic parameters and then performing prediction analysis based on the specific characteristic parameter values, the computational amount required for actual data prediction analysis can be effectively reduced, and the efficiency of prediction analysis and the accuracy of prediction results can be correspondingly improved.
[0080] For example, by obtaining a data prediction request corresponding to user behavior data as the data to be predicted, the data processing result corresponding to the user behavior data (such as the user's click operation result, that is, which recommended content the user may click on) can be predicted based on the received data prediction request.
[0081] Among them, the association relationship at least includes a mapping relationship and an operation relationship. In more possible implementation manners, the association relationship further includes other relationship types, and the present disclosure does not limit this. Taking the association relationship including a mapping relationship and an operation relationship as an example, the following introduces these two relationship types included in the association relationship respectively:
[0082] The mapping relationship can represent whether it is possible to map the data corresponding to another characteristic parameter based on the data corresponding to one or more characteristic parameters. For example, based on the value of the first characteristic parameter, the value of the second characteristic parameter can be determined.
[0083] Optionally, the mapping relationship can be a one-to-one mapping or a many-to-one mapping. That is, based on the data corresponding to different characteristic parameters, the data corresponding to another characteristic parameter determined may be the same. In addition, based on different data corresponding to the same characteristic parameter, the data of another characteristic parameter determined can be the same or different. Still taking the data to be predicted as user behavior data as an example, each user identifier corresponds to a user attribute information. That is, there is a one-to-one mapping between the user identifier and the user attribute information, while the user behavior preference information corresponding to different user attribute information may be the same or different. That is, there is a many-to-one mapping between the user attribute information and the user behavior preference information.
[0084] The operation relationship can indicate the association relationship between two or more characteristic parameters. In other words, the association relationship can represent how to perform arithmetic processing based on the data corresponding to each of the two or more characteristic parameters to determine the data corresponding to another characteristic parameter. For example, how to determine the value of the third characteristic parameter based on the value of the first characteristic parameter and the value of the second characteristic parameter. Among them, the operation relationship can include arithmetic operations, convolutional operations, integral operations, statistical operations, discrete operations, etc. Correspondingly, the arithmetic processing can be arithmetic processing, convolutional processing, integral processing, statistical processing, discrete processing, etc. The present disclosure does not limit this.
[0085] Step 102: Based on the target feature calculation graph and the target structured data, use the pre-trained data processing neural network to determine the data processing result corresponding to the data prediction request.
[0086] Among them, the data processing result includes the processing result obtained by analyzing the product sales data, the processing result obtained by analyzing the user behavior data, and / or the processing result obtained by analyzing the vehicle trajectory.
[0087] The pre-trained data processing neural network can be various types of neural networks. For example, the pre-trained data processing neural network can be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Deep Neural Networks (DNN). Optionally, the pre-trained data processing neural network can also be other types of neural networks. The present disclosure does not limit the specific type of the data processing neural network.
[0088] In a possible implementation, the target structured data is input into a pre-trained data processing neural network. Through the pre-trained data processing neural network, the dimensional features of the target structured data indicated by the target feature calculation graph are calculated based on the target features, and the data corresponding to each target feature parameter in the target structured data is processed (including mapping processing based on the mapping relationship and arithmetic processing based on the arithmetic relationship) to obtain the data processing result corresponding to the data prediction request.
[0089] Taking the pre-trained data processing neural network as a CNN and the arithmetic processing as convolution arithmetic processing as an example, correspondingly, the process of determining the data processing result includes: inputting the target structured data into the CNN serving as the data processing neural network, and through each convolutional layer of the CNN, calculating the dimensional features of the structured data indicated by the target feature calculation graph, and performing convolution processing on the data corresponding to each target feature parameter in the target structured data, so as to obtain the data processing result corresponding to the data prediction request. It should be noted that this process is only an exemplary illustration of determining the data processing result and does not constitute a limitation to the present disclosure.
[0090] The present disclosure can effectively reduce the amount of calculation required for actual data prediction analysis and correspondingly improve the efficiency of the data processing process by first determining the prediction direction based on the feature parameters involved in the target structured data and then performing prediction analysis based on the specific data corresponding to the feature parameters. Moreover, by generating a feature calculation graph based on the association relationship between the feature parameters, and the feature parameters used in the training process and the inference process do not change, so that the generated feature calculation graph can be used in both the training process and the inference process, thus realizing the integration of the training process and the inference process, and being able to realize the automated deployment of the inference service. There is no need to write different codes in the training process and the inference process as in the related art to implement the feature engineering, which saves the development time and can also reduce the risk of code errors.
[0091] After introducing the basic implementation process of the present disclosure, the various non-limiting embodiments of the present disclosure will be specifically introduced below.
[0092] Among them, the feature calculation graph includes multiple nodes, and edges are set between the nodes. In some embodiments, pre-generating a feature calculation graph based on the association relationship between multiple feature parameters involved in the structured data includes the following steps:
[0093] Step 1: Respectively use the multiple feature parameters involved in the structured data as multiple nodes of the feature calculation graph.
[0094] Taking the target structured data to be predicted as user behavior data as an example, the multiple characteristic parameters involved in the user behavior data include the user's age, the user's gender (such as male, female), and the user's behavior preference information (such as preferring to browse entertainment news, preferring to listen to jazz-style music, etc.). Then, the user's age, the user's gender, and the user's behavior preference information are used as multiple nodes of the feature calculation graph.
[0095] Step 2: For any two nodes among the multiple nodes, when there is an association relationship between the two characteristic parameters corresponding to the two nodes, create an edge in the feature calculation graph to indicate the association relationship between the two characteristic parameters corresponding to the two nodes connected by the edge.
[0096] Based on the multiple nodes of the feature calculation graph determined in the above Step 1, among them, there is an operation relationship between the user's gender, the user's age, and the user's behavior preference information. Based on the user's gender and age, the user's behavior preference information can be predicted. And there is a mapping relationship between the user's behavior preference information and the user operation data (which type of news was actually browsed, which type of music was actually listened to). Based on the user's behavior preference information, the user operation data can be predicted. Based on the above relationships, edges are constructed between each node, and the feature calculation graph as shown in Figure 2 can be obtained. See Figure 2 , Figure 2 which is a schematic diagram of a feature calculation graph shown in this specification according to an exemplary embodiment.
[0097] The target structured data to be predicted, the characteristic parameters involved in the target structured data, the association relationships between the characteristic parameters, etc. involved in the above process are all exemplary descriptions and do not constitute a limitation to the present disclosure. In more possible implementation manners, the target structured data to be predicted, the characteristic parameters involved in the target structured data, the association relationships between the characteristic parameters, etc. are of other types, and the present disclosure does not limit this.
[0098] In some embodiments, a feature calculation graph can be constructed based on the characteristic parameters of different structured data and different characteristic parameters of the same structured data, and the constructed feature calculation graph can be stored so that when processing structured data subsequently, the target feature calculation graph to be used can be directly obtained from the stored feature calculation graph.
[0099] In a possible implementation manner, after receiving a data prediction request, the target feature calculation graph corresponding to the data prediction request is obtained from the stored multiple feature calculation graphs.
[0100] For example, a computer device may be associated with a database for storing feature calculation graphs corresponding to different feature parameters. Thus, when the computer device receives a data prediction request, it can directly obtain the target feature calculation graph corresponding to the data prediction request from the database associated with the computer device.
[0101] Among them, obtaining the target feature calculation graph corresponding to the data prediction request from multiple feature calculation graphs includes the following steps:
[0102] Step 1: Determine at least one target feature parameter involved in the target structured data.
[0103] In a possible implementation, the target structured data to be predicted corresponding to the data prediction request only includes data corresponding to the prediction target of the data prediction request. The computer device can determine at least one target feature parameter corresponding to the data prediction request based on the target structured data corresponding to the data prediction request.
[0104] Among them, the prediction target of the data prediction request is used to indicate the result to be obtained by processing the target structured data to be predicted. Still taking the target structured data to be predicted as user behavior data as an example, if the prediction target of the data prediction request is which type of news the user will browse, that is, the prediction target of the data prediction request is to obtain user operation data, then the data corresponding to the data prediction request includes data corresponding to the user's gender, data corresponding to the user's age, and data corresponding to the user's behavior preference information. Based on this, at least one target feature parameter corresponding to the data prediction request can be determined as the user's gender, the user's age, and the user's behavior preference information.
[0105] Optionally, the computer device can also directly determine at least one target feature parameter corresponding to the data prediction request based on the prediction target corresponding to the data prediction request.
[0106] In a possible implementation, the computer device pre-stores the corresponding relationship between the types of each data prediction request and the target feature parameters, so that after receiving a data prediction request, it can directly determine at least one target feature parameter corresponding to the data prediction request based on the stored corresponding relationship and the type of the received data prediction request.
[0107] The above are only exemplary ways to determine at least one target feature parameter corresponding to the data prediction request. In more possible implementations, other ways can also be used to determine at least one target feature parameter corresponding to the data prediction request, and the present disclosure does not limit this.
[0108] Step 2: Based on at least one target feature parameter, determine a target feature calculation graph of the corresponding nodes including at least one target feature parameter from multiple feature calculation graphs.
[0109] In a possible implementation, the computer device determines, based on the at least one target feature parameter that has been determined, a target feature calculation graph of the corresponding nodes including the at least one target feature parameter from the database associated with the computer device.
[0110] The above process stores multiple feature calculation graphs in advance, so that when data prediction is required, the stored feature calculation graphs can be directly obtained without regenerating the feature calculation graphs, thereby improving the data prediction speed and data prediction efficiency.
[0111] In some embodiments, the training process of the data processing neural network includes:
[0112] Step 1: Obtain sample data, where the sample data is structured data involving multiple sample feature parameters and is labeled with the sample processing result corresponding to the sample data.
[0113] Taking the sample data as sample user behavior data as an example, the sample feature parameters involved in the sample data may include sample user behavior preference information (that is, the type of news preferred by the sample), and the sample processing result corresponding to the sample data may be the historical operation data of the sample user (the news actually clicked by the user).
[0114] Step 2: Input the sample data into the initial neural network. Through the initial neural network, based on the dimensionality features of the sample data indicated by the target feature calculation graph, process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data.
[0115] Among them, the initial neural network can be CNN, RNN, DNN, etc. The present disclosure does not limit which specific type of neural network is adopted.
[0116] Still taking the sample data as sample user behavior data, and the sample feature parameters corresponding to the sample data including sample user behavior preference information and the historical operation data of the sample user as an example, the computer device inputs the data corresponding to the sample user behavior preference information included in the sample user behavior data into the initial neural network. Through the initial neural network, based on the mapping relationship between the sample user behavior preference information and the operation data of the sample user in the target feature calculation graph, process the data corresponding to the sample user behavior preference information, thereby obtaining the operation data of the sample user as the prediction result corresponding to the sample data.
[0117] Step 3: Based on a loss function that can characterize the difference between the prediction result and the sample processing result, train the initial neural network until the preset training completion condition is met, and obtain a data processing neural network.
[0118] Continuing with the examples in Step 1 and Step 2 above, the prediction result is the operation data of the sample user, and the sample processing result is the historical operation data of the sample user.
[0119] In a possible implementation, based on the difference between the prediction result and the sample processing result, determine the loss function of the initial neural network, and then based on the determined loss function, update the network parameters of the initial neural network until the preset training completion condition is met, and obtain a data processing neural network.
[0120] Among them, the loss function of the initial neural network can adopt any type of function, and the present disclosure does not limit this. The network parameters include weight parameters. Optionally, the network parameters can also include other types of parameters, and the present disclosure does not limit this.
[0121] It should be noted that the process of updating the network in Step 2 and Step 3 is an iterative process. That is, the computer device inputs the first sample data into the initial neural network, and through the initial neural network, outputs the prediction result corresponding to the first sample data. Based on the difference between the prediction result of the first sample data and the sample processing result of the first sample data, determine the loss function of the initial neural network, and then based on the determined loss function, update the network parameters of the initial neural network to obtain a data processing neural network after the first parameter update; the computer device inputs the second sample data into the data processing neural network after the first parameter update, and through the data processing neural network after the first parameter update, outputs the prediction result corresponding to the second sample data. Based on the difference between the prediction result of the second sample data and the sample processing result of the second sample data, determine the loss function of the data processing neural network after the first parameter update, and then based on the determined loss function, update the network parameters of the data processing neural network after the first parameter update to obtain a data processing neural network after the second parameter update, and then continue to update the network parameters of the data processing neural network after the second parameter update through the third sample data to obtain a data processing neural network after the third parameter update, and so on, until the preset training completion condition is met, and the data processing neural network that meets the training completion condition is used as the trained data processing neural network.
[0122] Among them, the training completion condition is that the loss function meets the set condition, or the number of iterations reaches the set number, etc., and the present disclosure does not limit this.
[0123] It should be noted that the data processing neural network includes a feature calculation engine for performing feature engineering. The above process of training the data processing neural network includes the process of training the feature calculation engine.
[0124] In some embodiments, different association relationships correspond to different algorithms. The algorithms corresponding to the association relationships are encapsulated as calculation tools, and different association relationships correspond to different calculation tools.
[0125] In a possible implementation manner, after encapsulating the algorithm corresponding to the association relationship as a calculation tool, the encapsulated calculation tool is stored so that the stored calculation tool can be directly obtained subsequently, and thus the data corresponding to multiple feature parameters can be processed through the calculation tool.
[0126] Correspondingly, inputting the sample data into the initial neural network, through the initial neural network, based on the dimensionality features of the sample data indicated by the target feature calculation graph, processing the data corresponding to each sample feature parameter in the sample data, and obtaining the prediction result corresponding to the sample data, including the following steps:
[0127] Step 1: Determine the calculation tools involved in the target feature calculation graph based on the association relationship indicated among multiple sample feature parameters involved in the sample data.
[0128] In a possible implementation manner, the computer device obtains the calculation tool corresponding to the association relationship from the stored calculation tools according to the association relationship among multiple sample feature parameters involved in the sample data, and uses it as the calculation tool involved in the target feature calculation graph.
[0129] Step 2: Input the sample data into the initial neural network, and through the initial neural network, use the calculation tools involved in the target feature calculation graph to process the data corresponding to each sample feature parameter in the sample data, and obtain the prediction result corresponding to the sample data.
[0130] The processes of the above various embodiments can be referred to Figure 3 , Figure 3This is a flowchart showing a data processing neural network training process and an inference process according to an exemplary embodiment of the present specification. After constructing a feature computation graph based on structured data, the training of a feature computation engine is performed based on the feature computation graph, thereby realizing the training of the data processing neural network, optimizing the parameters of the data processing neural network, obtaining a data processing neural network with better data processing performance, and then pushing the feature computation graph and the trained data processing neural network to a computer device, so that when the computer device receives a data prediction request, based on the feature computation graph, through the trained data processing neural network and the feature computation inference engine, the data prediction request is processed to realize the inference based on the data prediction request.
[0131] The above Figure 3 The process shown above is only a flowchart description of the present disclosure. For the specific implementation process, reference can be made to the above various embodiments, which will not be elaborated here.
[0132] In some embodiments, the above processes can be implemented through the Photon platform.
[0133] Photon is a machine learning platform based on structured, graph, and graph spectrum big data. It abstracts the common modules such as data, features, training, and deployment of big data machine learning, and provides orchestration and management tools, providing full life cycle support for big data learning from development to production environment deployment. Among them, the main features of the Photon platform include:
[0134] (1) Multi-framework support: Support for deep learning, machine learning, and distributed machine learning frameworks, including deep learning training frameworks (such as TensorFlow, PyTorch, etc.), distributed and single-machine machine learning libraries (such as Spark MLlib, Scikit-learn, XGBoost, LightGBM, etc.); under each framework, unified interfaces and artifacts are defined for links such as data, features, training, and deployment.
[0135] (2) Production environment-oriented: Provide machine learning pipeline (Pipeline) orchestration, metadata (Metadata) management, training scheduling, inference deployment, and monitoring tools to realize the integration of training and inference including data transformation.
[0136] (3) Development and production unification: Support local development and containerized application (Kubernetes, k8s) cluster environment deployment, and apply for resources in different environments through a desktop launcher (Laucher).
[0137] (4) Scalability: Based on k8s, support for distributed computing and training.
[0138] Through the Photon platform, the modularization and standardization of the code library can be achieved, providing support for the training of the production environment and the inference deployment of machine learning, improving the R & D efficiency and the quality of the code, and minimizing the work from R & D to deployment.
[0139] For the Photon platform, the inference engine is an online model prediction service provided by the Photon platform. The inference engine can provide open-source remote procedure call (Google Remote Procedure Call, gRPC) and Representational State Transfer (REST)-compliant API interfaces (Application Programming Interface) for external applications to call for sample prediction.
[0140] The Photon platform can be used to create a training pipeline. When creating a training pipeline, the user can specify whether each module (Board) joins the service (Serving). By default, the data transformation (Transformer) class and model (Estimator) class modules are joined. Each module is connected in the training pipeline and is a single chain. There can be at most one model class module, and it is located at the end of the inference module.
[0141] The Pusher at the end of the training pipeline packages the involved installation packages, libraries, and source code and sends them to the specified storage path (local file system or global storage area network S3). After the InferenceEngine Server side is started, it pulls the Serving package file from the specified storage path, extracts it, and creates a Serving_Pipeline Instance, waits for requests, and starts a subprocess to check at regular intervals whether there is a new Serving package push. When a new Serving package push is detected, it pulls the newly pushed Serving package.
[0142] The above is only an illustration of an exemplary implementation method. In more possible implementation methods, the present disclosure can also be implemented through other platforms or codes, and the present disclosure does not limit this.
[0143] Corresponding to the embodiments of the foregoing method, this specification also provides embodiments of a device and a computer device to which the device is applied.
[0144] As Figure 4 shown, Figure 4 is a block diagram of a data processing device shown in this specification according to an exemplary embodiment. The device includes:
[0145] An acquisition unit 401 is configured to acquire a target feature computation graph corresponding to a data prediction request from multiple feature computation graphs, where each feature computation graph is pre-generated based on the association relationship between multiple feature parameters involved in structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of target structured data;
[0146] A result determination unit 402 is configured to determine a data processing result corresponding to the data prediction request based on the target feature computation graph and the target structured data by using a pre-trained data processing neural network.
[0147] In a possible implementation manner, the apparatus further includes:
[0148] A generation unit is configured to pre-generate a feature computation graph based on the association relationship between multiple feature parameters involved in structured data;
[0149] When the generation unit is configured to pre-generate a feature computation graph based on the association relationship between multiple feature parameters involved in structured data, it is specifically configured to:
[0150] Use multiple feature parameters involved in the structured data as multiple nodes of the feature computation graph respectively; for any two nodes among the multiple nodes, when there is an association relationship between the two feature parameters corresponding to the two nodes, create an edge for connecting the two nodes in the feature computation graph to indicate the association relationship between the two feature parameters corresponding to the two nodes connected by the edge.
[0151] In a possible implementation manner, when the acquisition unit 401 is configured to acquire a target feature computation graph corresponding to a data prediction request from multiple feature computation graphs, it is specifically configured to:
[0152] Determine at least one target feature parameter involved in the target structured data;
[0153] Based on the at least one target feature parameter, determine a target feature computation graph including the corresponding nodes of the at least one target feature parameter from the multiple feature computation graphs.
[0154] In a possible implementation manner, the target structured data includes data corresponding to multiple feature parameters respectively;
[0155] When the result determination unit 402 is configured to determine a data processing result corresponding to the data prediction request based on the target feature computation graph and the target structured data by using a pre-trained data processing neural network, it is specifically configured to:
[0156] Input the target structured data into a pre-trained data processing neural network. Through the pre-trained data processing neural network, based on the dimensional features of the target structured data indicated by the feature calculation graph, process the data corresponding to each target feature parameter in the target structured data to obtain the data processing result corresponding to the data prediction request.
[0157] In a possible implementation manner, when training the data processing neural network, the obtaining unit 401 is further configured to obtain sample data, where the sample data is structured data involving multiple sample feature parameters and is labeled with the sample processing result corresponding to the sample data.
[0158] The device further includes:
[0159] A processing unit, configured to input the sample data into the initial neural network. Through the initial neural network, based on the dimensional features of the sample data indicated by the target feature calculation graph, process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data.
[0160] A training unit, configured to train the initial neural network based on a loss function that can represent the difference between the prediction result and the sample processing result until a preset training completion condition is met to obtain the data processing neural network.
[0161] In a possible implementation manner, different association relationships correspond to different algorithms. The algorithm corresponding to the association relationship is encapsulated as a calculation tool, and different association relationships correspond to different calculation tools.
[0162] When the processing unit is configured to input the sample data into the initial neural network. Through the initial neural network, based on the dimensional features of the sample data indicated by the target feature calculation graph, process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data, it is specifically configured to:
[0163] Determine the calculation tools involved in the target feature calculation graph based on the association relationship between the multiple sample feature parameters involved in the sample data.
[0164] Input the sample data into the initial neural network. Through the initial neural network, use the calculation tools involved in the target feature calculation graph to process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data.
[0165] In a possible implementation manner, the data processing result includes at least one of the following:
[0166] The processing result obtained by analyzing based on the product sales data;
[0167] Processing results obtained by analyzing user behavior data;
[0168] Processing results obtained by analyzing vehicle trajectories.
[0169] For the specific implementation process of the functions and roles of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method, which will not be elaborated here.
[0170] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative efforts.
[0171] The present disclosure also provides a computer device. Refer to Figure 5 , Figure 5 which is a schematic structural diagram of a computer device shown in this specification according to an exemplary embodiment. As Figure 5 shown, the computer device includes a processor 510, a memory 520, and a network interface 530. The memory 520 is used to store computer instructions that can run on the processor 510. The processor 510 is used to implement the data processing method provided in any embodiment of the present disclosure when executing the computer instructions. The network interface 530 is used to implement input and output functions. In more possible implementation manners, the computer device may further include other hardware, which is not limited in the present disclosure.
[0172] The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium can be in various forms. For example, in different examples, the computer-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or a combination thereof. Specifically, the computer-readable medium can also be paper or other suitable media that can print programs. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the data processing method provided in any embodiment of the present disclosure.
[0173] The present disclosure also provides a computer program product, including a computer program which, when executed by a processor, implements the data processing method provided in any embodiment of the present disclosure.
[0174] Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, an apparatus, a terminal, a computer-readable storage medium, or a computer program product. Therefore, one or more embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0175] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiment corresponding to the terminal, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0176] The above describes specific embodiments of this specification. Other embodiments are within the scope of the present disclosure. In some cases, the actions or steps recorded in the present disclosure can be executed in a different order from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0177] The embodiments of the subject matter and the functional operations described in this specification can be implemented in the following: digital electronic circuits, tangible computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier to be executed by a data processing apparatus or to control the operation of the data processing apparatus. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for execution by the data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
[0178] The processes and logical flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform the corresponding functions by operating on input data and generating output. The processes and logical flows can also be performed by, for example, FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit) of special logic circuits, and the apparatus can also be implemented as special logic circuits.
[0179] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Generally, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, etc., or the computer will be operatively coupled to such mass storage devices to receive data therefrom or transfer data thereto, or both. However, a computer is not necessarily required to have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name just a few.
[0180] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special logic circuits.
[0181] Although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of what is claimed, but rather are mainly used to describe the features of specific embodiments of a particular invention. Certain features described in multiple embodiments in this specification can also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment can also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features can operate in certain combinations as described above and are even initially claimed as such, one or more features from the claimed combination can in some cases be removed from the combination, and the claimed combination can be directed to a sub-combination or a variation of the sub-combination.
[0182] Similarly, although the operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or sequentially, or that all illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the above-described embodiments should not be understood to be required in all embodiments, and it should be understood that the program components and systems described can generally be integrated together in a single software product or packaged into multiple software products.
[0183] Accordingly, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the present disclosure. In some cases, the acts recited in the present disclosure may be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the drawings are not necessarily in the particular order or sequential order shown to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.
[0184] Those skilled in the art will readily conceive of other embodiments of the present specification after considering the specification and practicing the invention claimed herein. The present specification is intended to cover any variations, uses, or adaptations of the present specification, which follow the general principles of the present specification and include common general knowledge or conventional technical means in the technical field not claimed in the present application. That is, the present specification is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope.
[0185] The above are only alternative embodiments of the present specification and are not intended to limit the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present specification shall be included within the scope of protection of the present specification.
Claims
1. A data processing method, characterized in that, The method includes: Obtaining a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs, where each of the feature calculation graphs is pre-generated based on the association relationship between multiple feature parameters involved in structured data to indicate the dimensional features of the structured data, and the data prediction request is used to request prediction processing of target structured data, where the target structured data corresponds to at least one of product sales data, user behavior data, and vehicle trajectories; Based on the target feature calculation graph and the target structured data, using a pre-trained data processing neural network, determining a data processing result corresponding to the data prediction request, where the data processing result includes at least one of a processing result obtained by analyzing product sales data, a processing result obtained by analyzing user behavior data, and a processing result obtained by analyzing vehicle trajectories.
2. The method according to claim 1, characterized in that, Pre-generating the feature calculation graph based on the association relationship between multiple feature parameters involved in structured data includes: Taking the multiple feature parameters involved in the structured data as multiple nodes of the feature calculation graph respectively; For any two nodes among the multiple nodes, when there is an association relationship between the two feature parameters corresponding to the two nodes, creating an edge for connecting the two nodes in the feature calculation graph to indicate the association relationship between the feature parameters corresponding to the two nodes connected by the edge.
3. The method according to claim 1, characterized in that, The obtaining of the target feature calculation graph corresponding to the data prediction request from multiple feature calculation graphs includes: Determining at least one target feature parameter involved in the target structured data; Based on the at least one target feature parameter, determining, from the multiple feature calculation graphs, a target feature calculation graph including the nodes corresponding to the at least one target feature parameter.
4. The method according to claim 1, wherein The target structured data includes data corresponding to each of the multiple target feature parameters; The determining of the data processing result corresponding to the data prediction request based on the target feature calculation graph and the target structured data, using a pre-trained data processing neural network, includes: Inputting the target structured data into the pre-trained data processing neural network, and through the pre-trained data processing neural network, processing the data corresponding to each of the target feature parameters in the target structured data based on the dimensional features of the target structured data indicated by the feature calculation graph to obtain the data processing result corresponding to the data prediction request.
5. The method according to claim 1, characterized in that, The training process of the data processing neural network includes: Obtaining sample data, where the sample data is structured data involving multiple sample feature parameters and is labeled with a sample processing result corresponding to the sample data; Inputting the sample data into an initial neural network, and through the initial neural network, processing the data corresponding to each of the sample feature parameters in the sample data according to the dimensional features of the sample data indicated by the target feature calculation graph to obtain a prediction result corresponding to the sample data; Based on a loss function that can characterize the difference between the prediction result and the sample processing result, the initial neural network is trained until a preset training completion condition is satisfied, and the data processing neural network is obtained.
6. The method according to claim 5, characterized in that, Different association relationships correspond to different algorithms, and the algorithms corresponding to the association relationships are encapsulated as computing tools, and different association relationships correspond to different computing tools; The step of inputting the sample data into the initial neural network, through the initial neural network, based on the dimension features of the sample data indicated by the target feature calculation graph, processing the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data includes: Based on the association relationship between multiple sample feature parameters involved in the sample data, determine the computing tools involved in the target feature calculation graph; Input the sample data into the initial neural network, and through the initial neural network, use the computing tools involved in the target feature calculation graph to process the data corresponding to each sample feature parameter in the sample data to obtain the prediction result corresponding to the sample data.
7. A data processing device, characterized in that, The device includes: An acquisition unit, configured to acquire a target feature calculation graph corresponding to a data prediction request from multiple feature calculation graphs, where each feature calculation graph is pre-generated based on the association relationship between multiple feature parameters involved in the structured data to indicate the dimension features of the structured data, and the data prediction request is used to request prediction processing of target structured data, where the target structured data corresponds to at least one of product sales data, user behavior data, and vehicle trajectories; A result determination unit, configured to determine the data processing result corresponding to the data prediction request based on the target feature calculation graph and the target structured data by using a pre-trained data processing neural network, where the data processing result includes at least one of the processing result obtained by analyzing the product sales data, the processing result obtained by analyzing the user behavior data, and the processing result obtained by analyzing the vehicle trajectory.
8. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the operations performed by the data processing method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed by the processor, the operations performed by the data processing method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Graph data processing method and device, electronic equipment and computer storage medium
CN113011282A
Node classification method and apparatus for graph network model, and terminal device
WO2021089012A1