Estimation model training method, data processing method, device and electronic equipment

Abstract

The application discloses a kind of estimation model training method, data processing method, device and electronic equipment, the method includes: obtaining first feature sample, second feature sample and third feature sample;First feature sample is carried out feature enhancement processing, and the above-mentioned three feature samples are carried out first cross fusion processing;Feature enhancement result and first cross fusion result are input after second cross fusion processing and are carried out full connection processing to multilayer full connection layer, the full connection processing result is classified and is handled, and according to the matching degree between the result of classification processing and real result, the model parameter of estimation model is determined, and estimation model is constructed.The application enhances the expression content of specific feature while not affecting the training effect of original large general feature, improves the prediction accuracy of estimation model, and then improves the efficiency of advertisement exposure allocation, and can be widely applied to cloud computing, Internet of Vehicles and other technical fields related to Internet technology.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a predictive model training method, data processing method, apparatus, and electronic device. Background Technology

[0002] Pay-per-click advertising is a new type of online advertising where the target audience independently places and manages the ads, and pays based on the ads' performance. To help the target audience better manage their ads, many advertising platforms predict the ad's exposure over a future period and then provide relevant reference information based on these predictions.

[0003] In related technologies, advertising platforms mainly use the following methods to predict the exposure of advertisements: For an advertisement whose exposure needs to be predicted, the characteristics of the advertisement itself (such as the size and attractiveness of the advertisement material), the targeting characteristics of the advertisement (such as the characteristics of the target audience of the advertisement), and the bidding information are input into a pre-trained regression model, and the exposure of the advertisement in the future is predicted by the regression model.

[0004] To improve the prediction accuracy of regression models, related techniques continuously refine the input features during training. As a result, the coverage of each individual feature is relatively limited. Current regression models generally adopt a uniform embedded expression form, and the model structure does not focus on expressing industry-related advertising features. This leads to a very limited frequency of many specific industry features related to advertising in the samples. Ultimately, the role of specific industry features in the model training process is easily submerged, resulting in low prediction accuracy of the model. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a high-accuracy prediction model training method, data processing method, apparatus, and electronic device to enhance the expression of specific industry-specific features during the prediction model training process and improve the accuracy of advertising exposure prediction.

[0006] On one hand, embodiments of the present invention provide a prediction model training method, including the following steps:

[0007] Obtain a first feature sample, a second feature sample, and a third feature sample; wherein, the first feature sample is used to characterize the specific industry characteristics of the target industry; the second feature sample is used to characterize the object attribute characteristics of the target object; and the third feature sample is used to characterize the general text features of all industries.

[0008] The first feature sample is input into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result. The first feature sample, the second feature sample, and the third feature sample are then subjected to a first cross-fusion process to obtain the first cross-fusion result.

[0009] The feature enhancement result and the first cross-fusion result are then subjected to a second cross-fusion process and input into a multi-layer fully connected layer for full-connected processing to obtain the full-connected processing result. The input data of each fully connected layer in the multi-layer fully connected layer includes the output of the previous fully connected layer and the result of the second cross-fusion process of the feature enhancement result.

[0010] The results of the fully connected processing are classified, and the model parameters of the prediction model are determined based on the degree of matching between the classification results and the actual results. Then, the prediction model is constructed based on the model parameters.

[0011] On the other hand, embodiments of the present invention also provide a data processing method, including the following steps:

[0012] Obtain the advertising data to be estimated;

[0013] Based on the target prediction model, the advertising exposure value of the advertising data to be predicted is predicted, and then the advertising exposure value is used to determine the allocation of advertising resources.

[0014] The target prediction model is determined based on the above prediction model training method.

[0015] On the other hand, embodiments of the present invention also provide a prediction model training apparatus, comprising:

[0016] The first module is used to acquire a first feature sample, a second feature sample, and a third feature sample; wherein, the first feature sample is used to characterize the specific industry characteristics of the target industry; the second feature sample is used to characterize the object attribute characteristics of the target object; and the third feature sample is used to characterize the general text features of all industries.

[0017] The second module is used to input the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and to perform a first cross-fusion processing on the first feature sample, the second feature sample and the third feature sample to obtain the first cross-fusion result;

[0018] The third module is used to perform a second cross-fusion process on the feature enhancement result and the first cross-fusion result, and then input the result into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the result of the second cross-fusion process on the output of the previous fully connected layer and the feature enhancement result;

[0019] The fourth module is used to classify the results of the fully connected processing, and determine the model parameters of the prediction model based on the degree of matching between the classification results and the actual results, and then complete the construction of the prediction model based on the model parameters.

[0020] Optionally, in the prediction model training apparatus of this embodiment, the second module includes a first unit and a second unit. The first unit and the second unit are used in the step of inputting the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, wherein:

[0021] The first unit is used to vectorize the first feature sample to obtain the first feature vector.

[0022] The second unit is used to input the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result; wherein, the feature enhancement module includes at least two fully connected layers.

[0023] Optionally, in the prediction model training apparatus of this embodiment, the first feature sample includes specific feature samples and advertising feature samples, and the first unit includes:

[0024] The first sub-unit is used to vectorize the specific feature samples and advertising feature samples in the first feature sample to obtain the first feature vector.

[0025] Optionally, in the prediction model training apparatus of this embodiment, the second unit includes:

[0026] The second subunit is used to perform fully connected processing on the first feature vector sequentially through the two fully connected layers of the feature enhancement module;

[0027] The third subunit is used to constrain the fully connected processing results of the second fully connected layer through a constraint function, thereby limiting the feature enhancement results to the target range.

[0028] Optionally, in the prediction model training apparatus of this embodiment, the second unit further includes:

[0029] The fourth subunit is used to add activation structures to the feature enhancement module;

[0030] The fifth subunit is used to respond to the activation operation by activating the fully connected processing results of the two fully connected layers in the feature enhancement module through the activation structure, thereby determining the bias information of the feature enhancement results for the construction process of the prediction model.

[0031] Optionally, in the prediction model training device of this embodiment, the first subunit includes:

[0032] The first sub-unit is used to perform gradient isolation processing on the advertising feature samples to obtain the gradient isolation processing result of the advertising feature samples.

[0033] The second subunit is used to vectorize specific feature samples and gradient isolation processing results to obtain the first feature vector.

[0034] Optionally, in the prediction model training device of this embodiment, the second subunit includes:

[0035] The third subunit is used to add a matrix factorization layer between the two fully connected layers of the feature enhancement module;

[0036] The fourth subunit is used to perform matrix decomposition on the result of the first fully connected layer through the matrix decomposition layer, and then the second fully connected layer performs fully connected processing on the result of the matrix decomposition to obtain the feature enhancement result.

[0037] Optionally, in the prediction model training apparatus of this embodiment, the first module includes a third unit and a fourth unit, which are used in the step of obtaining the first feature sample, wherein:

[0038] The third unit is used to obtain candidate feature samples;

[0039] The fourth unit is used to filter candidate feature samples and identify multiple candidate feature samples with feature relevance higher than a threshold as specific feature samples.

[0040] Optionally, in the prediction model training device of this embodiment, the second module further includes a fifth unit and a sixth unit, which are used in the step of performing a first cross-fusion process on the first feature sample, the second feature sample, and the third feature sample to obtain a first cross-fusion result, wherein:

[0041] The fifth unit is used to vectorize the first feature sample, the second feature sample, and the third feature sample to obtain the first feature vector, the second feature vector, and the third feature vector.

[0042] The sixth unit is used to perform a first cross-fusion process on the first feature vector, the second feature vector, and the third feature vector to obtain the first cross-fusion result.

[0043] On the other hand, embodiments of the present invention also provide a data processing apparatus, including:

[0044] The fifth module is used to obtain the advertising data to be estimated;

[0045] The sixth module is used to estimate the advertising exposure value of the advertising data to be estimated based on the target prediction model, and then determine the allocation of advertising resources based on the advertising exposure value.

[0046] The target prediction model is determined based on the aforementioned prediction model training device.

[0047] On the other hand, embodiments of the present invention also provide an electronic device, including a processor and a memory;

[0048] Memory is used to store programs;

[0049] The processor executes the program to implement the prediction model training method or data processing method described above.

[0050] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a program that is executed by a processor to implement the prediction model training method or data processing method described above.

[0051] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the prediction model training method or data processing method described above.

[0052] This invention first acquires a first feature sample representing specific industry characteristics, a second feature sample representing object attribute characteristics, and a third feature sample representing general text characteristics. Then, this invention inputs the first feature sample into a feature enhancement module for feature enhancement processing to obtain the feature enhancement result. Simultaneously, it performs a first cross-fusion processing on the first, second, and third feature samples to obtain a first cross-fusion result. Based on a general prediction model, this invention enhances the representation of the first feature sample, allowing the prediction model training process to better reflect the information of the first feature sample, thus increasing the influence of specific industry characteristics on model training. Next, this invention performs a second cross-fusion processing on the feature enhancement result and the first cross-fusion result, then inputs them into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result. Finally, the fully connected processing result is classified, and the model parameters of the prediction model are determined based on the matching degree between the classification result and the true result. The prediction model is then constructed based on these model parameters. This invention enhances the representation of specific features without affecting the training effect of the original large number of general features, improving the prediction accuracy of the prediction model and thus increasing the efficiency of advertising exposure allocation. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating the implementation principle of the prediction model in the advertising system scenario within related technologies;

[0055] Figure 2 This is a schematic diagram of an implementation environment for training a prediction model in an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of the steps of a prediction model training method provided in an embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram illustrating the process of generating fully connected processing results through a prediction model in an embodiment of the present invention;

[0058] Figure 5 This is a schematic diagram illustrating the implementation principle of feature enhancement processing using multiple gate modules in an embodiment of the present invention;

[0059] Figure 6 This is a schematic diagram illustrating the implementation principle of feature enhancement processing and first cross-fusion processing for the first feature sample in an embodiment of the present invention;

[0060] Figure 7 This is a schematic diagram illustrating the implementation principle of constraining the output range of the feature enhancement module through a constraint function in an embodiment of the present invention;

[0061] Figure 8 This is a schematic diagram illustrating the implementation principle of activating the feature enhancement module through an activation structure in an embodiment of the present invention;

[0062] Figure 9 This is a schematic diagram illustrating the implementation principle of gradient isolation processing for advertising feature samples in an embodiment of the present invention;

[0063] Figure 10 This is a schematic diagram illustrating the implementation principle of matrix decomposition of feature enhancement results through a matrix decomposition layer, as provided in an embodiment of the present invention.

[0064] Figure 11 This is a flowchart illustrating the steps of a data processing method provided in an embodiment of the present invention;

[0065] Figure 12 This is a modular schematic diagram of a prediction model training device provided in an embodiment of the present invention;

[0066] Figure 13 This is a modular schematic diagram of a data processing device provided in an embodiment of the present invention;

[0067] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0069] It should be understood that in the description of the embodiments of the present invention, "multiple" (or "amounts") means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. If "first," "second," etc., are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0070] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0071] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention are explained, and the nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0072] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0073] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning. The display device with image acquisition components shown in this invention mainly relates to computer vision, machine learning / deep learning, autonomous driving, and intelligent transportation.

[0074] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0075] Deep learning, a concept originating from research on artificial neural networks, is exemplified by multilayer perceptrons with multiple hidden layers. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. It is a new field in machine learning research, motivated by the desire to build and simulate neural networks that mimic the human brain's analytical learning processes, interpreting data such as images, sounds, and text by imitating the brain's mechanisms. Commonly used deep learning models include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, Deep Neural Networks (DNNs), and Deep Belief Networks (DBNs). Data propagates in neural networks in two ways: forward propagation (along the path from input to output) and back propagation (from output back to input). During forward propagation, the input information is processed layer by layer through the neural network and transmitted to the output layer. The error between the output value and the expectation is described by the loss function. Then, backpropagation begins, and the partial derivatives of the loss function with respect to the weights of each neuron are calculated layer by layer. This forms the weight gradient data of the loss function with respect to the weight vector, which serves as the basis for updating the weight parameters. The training of the neural network is completed in the process of continuously updating the weight parameters.

[0076] A loss function is a function that maps the values ​​of a random event or its related random variables to non-negative real numbers to represent the "risk" or "loss" of that random event. In applications, loss functions are often used as learning criteria related to optimization problems; that is, the model is solved and evaluated by minimizing the loss function. For example, in machine learning, loss functions are used for parameter estimation of the model, and the loss value obtained based on the loss function can be used to describe the degree of difference between the model's predictions and the actual values. Common loss functions include the mean squared error loss function, the hinge loss function for Support Vector Machines (SVM), and the cross-entropy loss function.

[0077] Based on the aforementioned theoretical foundations, and with the research and advancements in artificial intelligence technology, AI technology has been studied and applied in multiple fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI technology will be applied in more fields and play an increasingly important role.

[0078] To facilitate understanding of the embodiments of the present invention, the following explanations are provided for keywords that may be involved in the embodiments of the present invention:

[0079] OCPX: O stands for Optimized. CPX stands for Cost Per X, where "X" refers to traditional billing models based on different methods, such as Cost Per Click (CPC), Cost Per Mille (CPM), and Cost Per Action (CPA). Examples of conversion effects include: leaving sales leads in forms, app downloads and installations, new registrations, coupon redemptions, adding items to carts, and even placing orders. Combining O with CPX results in OCPX, which optimizes various CPXs for Action conversions. Building upon traditional billing models, it incorporates the data and algorithmic capabilities of the advertising platform, allowing for further optimization of conversion goals.

[0080] OCPX is a popular advertising model in computational advertising. Advertisers set bids on the ad delivery platform based on their conversion goals (product orders or app downloads). The advertising system estimates CTR (click-through rate) and CVR (conversion rate) to calculate the value of traffic and allocate optimal exposure to the target audience. If the system ultimately deems the ad to have the best value (for example, the highest ECPM (ad revenue per thousand impressions)), it will be exposed to the target audience. In practice, it can be further subdivided into several forms according to specific business models, such as single-goal shallow optimization ads, single and dual-goal ads, and ROI (return on investment) ads.

[0081] ECPM (effective cost per mille): An abstract factor in advertising systems that measures the value of a single ad impression. It is generally composed of CTR (Click-Through-Rate) * CVR (Conversion Rate) * target_CPA (preset ad price) * alpha (dynamic weighting by the platform). CTR, CVR, and alpha are all controlled by the advertising platform, while target_CPA is the interface through which advertisers participate in ad allocation by bidding, serving as an incentive and compatibility mechanism.

[0082] Cost of Ads (COA): The cost of an ad divided by the number of conversions it generates. This measures the cost an advertiser needs to pay to achieve a single conversion. When the COA is greater than the target bid, it is considered over-cost; when it is less, it is considered under-cost.

[0083] GMV (Gross Merchandise Volume): The number of conversions (valid downloads or product orders) obtained by the ad * the advertiser's set ad bid, which measures the cost of running the ad.

[0084] Pointwise: This is a common point estimation modeling method. The general process involves scoring the model for a single item. In recommendation systems, positive and negative samples can be labeled directly through the click feedback of the target object.

[0085] With the continuous development of online computational advertising technology, an advertising ecosystem with OCPX as the mainstream has gradually formed. At the same time, with the large-scale application of deep learning in various fields, today's advertising backend retrieval systems can basically be regarded as recommendation systems based on deep learning. Correspondingly, advertising recommendation / delivery can be abstracted into a large-scale bipartite graph matching problem. The advertising platform leverages its data advantages to personalized recommend advertisements that are highly relevant to the target audience and have superior platform value to the target audience. The platform value of the advertisement can be abstracted into an ECPM formula: CTR*CVR*target_CPA*alpha, where CTR and CVR can currently be modeled in a personalized way using deep learning.

[0086] A Multilayer Perceptron (MLP), also known as an Artificial Neural Network (ANN), contains multiple hidden layers in addition to the input and output layers. The simplest MLP has only one hidden layer, making it a three-layer structure. All layers in a Multilayer Perceptron are fully connected. The bottom layer is the input layer, followed by hidden layers, and finally the output layer.

[0087] In related technologies, the behavior of the target audience after ad exposure is typically used as positive and negative feedback. For CTR (Click-Through Rate), this is based on the assumption of ad exposure; clicks after exposure represent positive feedback, indicating the ad is attractive to the target audience, while no clicks represent negative feedback. For CVR (Click-Through Rate), it's based on the initial click; further conversion behavior (such as placing an order or downloading an app) represents positive feedback, indicating that the ad not only initially attracted the target audience but also satisfied their deeper needs. Conversely, clicks without any conversion represent negative feedback. By collecting various positive and negative feedback behaviors of the target audience as positive and negative samples, and using the target audience's personalized information, ad information, and current traffic context as features, personalized modeling is performed using a DNN (Dual Neural Network) model. Specifically, for example... Figure 1As shown, CTR and CVR in advertising systems are generally built upon deep learning for representation learning. This utilizes the positive and negative feedback behaviors of the target object as behavioral samples, and embeds a large number of target object attribute samples, ad samples, traffic scenario samples, and other related features into a unified high-dimensional space. Then, a multilayer perceptron (MLP) is used to perform feature cross-validation of arbitrary dimensions to obtain the predicted output. Mathematically, any form of nonlinear function can be fitted. The final fitted probability value (logit) and the positive / negative representation form a loss function. The resulting gradient guides the iterative solution of model parameters, eventually converging to a stable value to obtain a well-established model. When a new target object request arrives, the established model is used to calculate the CTR and CVR of candidate ads, and then the ECPM is calculated as the ECPM score of the ad to be exposed, selecting the optimal value ad. However, advertising campaigns typically involve advertisers from various industries, and the characteristics of advertisements across these industries differ significantly. For example, within the e-commerce industry, the creative and product feature expressions for beverages and cosmetics differ considerably. Furthermore, with the diversification of promoted applications, the feature expressions for applications, links, or virtual products (such as game equipment and NFT digital collectibles) within the internet industry differ even further from those for various physical goods in the traditional e-commerce industry. Promotional items for the gaming industry are all virtual, while the promotional targets for e-commerce are physical goods. Therefore, the descriptions of these products must have different emphases, making complete uniformity impossible. This results in many advertising-related features appearing with limited frequency in the sample. While deep learning excels in its ability to represent a large number of refined features, as the level of feature abstraction becomes increasingly refined, the coverage of individual features becomes relatively limited. This means that relevant features are only effective in certain specific sub-sectors, leading to the limited expressive power of industry features (such as the characteristics of specific promotional items, including beverages, cosmetics, applications, links, and virtual products in e-commerce and the internet industry) in CTR and CVR modeling.

[0088] Therefore, the solution provided by the embodiments of the present invention involves technologies such as deep learning in artificial intelligence. Based on the constructed deep learning model, the feature enhancement of the acquired specific industry features is performed, and the final prediction model is made to better reflect the influence of the relevant specific industry features through cross-fusion with general features, without affecting the training results of the original large number of general features; for example, general features include the object attribute features of the target object and general text features of all industries, etc.

[0089] It is understood that the prediction model training method provided in this embodiment of the invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in this embodiment of the invention is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet computer, laptop computer, or desktop computer, but it is not limited to these.

[0090] It should be further noted that the terminals involved in the embodiments of the present invention include, but are not limited to, smartphones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0091] In some possible implementations, a computer program capable of implementing the prediction model training method or data processing method provided in the embodiments of the present invention can be deployed and executed on a computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network can form a blockchain system.

[0092] Based on the fact that multiple computer devices can form a blockchain system, the computer device implementing the data processing method in this embodiment of the invention can be a node in the blockchain system. This node stores a deep learning model, which acquires the advertising data to be estimated, and obtains the estimated advertising exposure value through feature enhancement and cross-fusion. Then, based on the advertising exposure value, it determines the allocation of advertising resources. This node, or other nodes corresponding to other devices in the blockchain, can also store the estimated advertising exposure value, the allocation results of advertising resources, and other intermediate feature data obtained during the estimation process.

[0093] like Figure 2 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 2The implementation environment includes at least one terminal 101 and a server 102. Terminal 101 and server 102 can be connected via a network, either wirelessly or via a wired connection, to exchange data. Terminal 101 may be configured with functions such as model pre-training and supervised adjustment of model pre-training parameters to adapt to downstream task requirements. Alternatively, terminal 101 may interact with server 102, and based on the program logic provided by server 102, implement relevant functions adapted to downstream task requirements. Optionally, server 102 may be a backend server for this program logic or a cloud server providing cloud computing and cloud storage services. Terminal 101 can pre-train various prediction models based on acquired feature samples. The pre-trained prediction models can be directly applied to various data / information prediction tasks (including but not limited to advertising exposure prediction, such as CTR prediction and CVR prediction, and ECPM determined based on CTR and CVR). Furthermore, based on the prediction results, corresponding data processing (including but not limited to resource allocation) can be performed according to the needs of downstream tasks. Terminal 101 can be a computer, mobile phone, tablet computer, or other terminal.

[0094] For example, the server can obtain feature samples (such as a first feature sample representing specific industry characteristics of the target industry, a second feature sample representing object attribute characteristics of the target object, and a third feature sample representing general text features of all industries) from the terminal for pre-training of the prediction model. Then, it performs the following processing to construct the trained prediction model: a. Input the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and perform a first cross-fusion processing on the first feature sample, the second feature sample, and the third feature sample to obtain the first cross-fusion result; b. Perform a second cross-fusion processing on the feature enhancement result and the first cross-fusion result, and input them into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the output of the previous fully connected layer and the result after the second cross-fusion processing of the feature enhancement result; c. Perform classification processing on the fully connected processing result, and determine the model parameters of the prediction model according to the degree of matching between the classification processing result and the real result, and then complete the construction of the prediction model according to the model parameters. Understandably, the current model parameters can be updated using evolutionary algorithms, including population-based genetic algorithms. The server can send the pre-trained prediction model to the terminal, which then performs various downstream tasks related to data / information prediction based on the prediction model.

[0095] This model training method can be applied to various scenarios.

[0096] For example, in the scenario of target object behavior prediction: With authorization from the target object, the target object's behavior sequence is processed as a specific feature and combined with the target object's object attribute features and other general features (such as device, geographical, and time features). This information is then sent to server 102 via terminal 101. Server 102 uses the aforementioned method to obtain a pre-trained (target object behavior) prediction model and sends it to terminal 101. Terminal 101 can then use this (target object behavior) prediction model to perform target object demand analysis, thereby providing recommendation services such as game item purchase recommendations, object browsing page layout recommendations, or reading content recommendations. It is understood that the above application scenario is only for illustrative purposes. In practical applications, besides using a server to pre-train the prediction model, other devices with model training capabilities, such as terminals, can also be used to pre-train and build the prediction model. Furthermore, other devices besides terminals can be used to carry the pre-trained prediction model. No limitations are placed on the application scenarios of the model training method provided in this embodiment of the invention.

[0097] exist Figure 2 Based on the implementation environment shown, this embodiment of the invention provides a scenario for predicting ad exposure. In this scenario, the server, through data interaction with the terminal, can obtain the ad resources to be predicted input by the advertiser on the terminal equipped with the target application via the target_CPA interface. Furthermore, based on the target prediction model pre-stored on the server (such as a pre-trained and constructed prediction model), the server predicts the ad exposure value of the ad data to be predicted, and then determines the allocation of ad resources based on the ad exposure value. For example, the target application can be an information interaction platform for advertising promotion, or a web link, mini-program, or application plugin with advertising promotion information interaction functions set in multimedia applications (such as movie applications, short video applications, or music applications, etc.), social applications, game applications, and navigation applications.

[0098] It should be noted that in the various specific implementations of this invention, when it is necessary to obtain and process data related to the object's identity or characteristics, such as the object's feature data, object behavior data, object historical data, and object location information, the object's permission or authorization will be obtained first. Furthermore, the collection, use, and processing of this data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this invention need to obtain sensitive personal information of an object, separate permission or consent from the object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the object's separate permission or consent will the necessary object-related data for the normal operation of embodiments of this invention be obtained.

[0099] like Figure 3The diagram shown is a flowchart of a prediction model training method provided in an embodiment of the present invention. The execution subject of this prediction model training method can be any of the aforementioned computer devices. (Refer to...) Figure 3 The method includes the following steps:

[0100] S301. Obtain the first feature sample, the second feature sample, and the third feature sample;

[0101] The first feature sample is used to characterize the specific industry characteristics of the target industry; the second feature sample is used to characterize the object attribute characteristics of the target object; and the third feature sample is used to characterize the general text features of all industries. It should be noted that, in this embodiment of the invention, the sample is used to characterize the content sample used to train the prediction model, and the content can be any one of text, audio, or image. It is understood that target feature extraction is performed on the sample to obtain the required feature sample. Specifically, the format of the feature sample can include at least one of numbers, vectors, matrices, or tensors, and this application does not limit this.

[0102] For example, taking an advertising promotion scenario, the first feature sample can be determined from the advertising resources to be estimated from the advertiser; the second feature sample can be, with the permission and authorization of the target object, the object attribute features (including but not limited to preference tags and other basic attribute features) related to all promotion objects in the specific promotion scenario or some promotion objects adapted based on preset conditions; the third feature sample can be determined through basic tag information such as date, time, and device type. It should be noted that the description of various feature samples in the above specific scenario is only used to help understand the representational meaning of the relevant feature samples and should not be regarded as a limitation on the embodiments of the present invention. In some foreseeable situations, it can also be used in other scenarios such as target object behavior prediction scenarios, such as, with the authorization of the target object, obtaining the behavior sequence of the target object as a specific feature and combining it with the object attribute features of the target object and other general features (such as device, geographical, time and other feature information).

[0103] S302. Input the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and perform the first cross-fusion processing on the first feature sample, the second feature sample and the third feature sample to obtain the first cross-fusion result;

[0104] It should be noted that, given that the general prediction model simply uses deep learning to perform a large number of refined feature representations, the role of specific industry features (first feature samples) in the model is easily overshadowed. Therefore, this embodiment of the invention focuses on feature enhancement and representation processing of specific industry features based on the general prediction model. By performing a first cross-fusion processing of the first feature sample, the second feature sample, and the third feature sample, the specific industry features are enhanced while avoiding affecting the training results of the original large number of general features (such as the second feature sample representing the object attribute features of the target object and the third feature sample representing the general text features of all industries).

[0105] In this embodiment of the invention, a personalized gate module can be introduced as a feature enhancement module to perform feature enhancement processing on the first feature sample; and an MLP (Multilayer Perceptron) can be used to perform feature cross-fertilization of arbitrary dimensions, further performing feature fusion to achieve the first cross-fusion processing. Feature fusion can be implemented using functions such as concat, concat_ws, and other feature fusion functions. Based on the feature enhancement processing of the first feature sample, and combined with the first cross-fusion processing of the first, second, and third feature samples, the features of a specific industry are fully expressed while maintaining compatibility with common features across multiple industries. The role of feature cross-fertilization is to reduce the weight of unimportant features and lower the weight of redundant features to improve model performance. Various feature samples often imply multimodal information processing. Feature fusion refers to leveraging the complementarity between features to fuse their advantages, thereby improving model performance, given features with different attributes.

[0106] S303. After performing a second cross-fusion process on the feature enhancement result and the first cross-fusion result, the result is input into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result.

[0107] In this multi-layer fully connected layer, the input data for each fully connected layer includes the output of the previous fully connected layer and the result of the feature enhancement process, which is then subjected to a second cross-fusion. It should be noted that multi-layer fully connected layers can be implemented using an MLP with multiple hidden layers, such as... Figure 4As shown, taking a three-layer fully connected MLP (the diagram illustrates the fully connected processing corresponding to the fully connected layers) as an example (it is understood that those skilled in the art can adjust the number of fully connected layers based on actual processing needs, and this invention does not impose specific limitations), for the first fully connected layer of the multi-layer fully connected system, its input is the result of the second cross-fusion processing of the feature enhancement result and the first cross-fusion result. The input of subsequent fully connected layers is the result of the second cross-fusion processing of the output of the previous fully connected layer and the feature enhancement result. That is, for two adjacent fully connected layers, the output of the previous fully connected layer is not directly used as the input of the next fully connected layer. Instead, the corresponding processing result is input to the next fully connected layer only after the feature enhancement result has been introduced and the second cross-fusion processing has been performed in each fully connected layer. The fully connected processing result is the result of the second cross-fusion processing of the output of the last fully connected layer of the multi-layer fully connected system and the feature enhancement result. The output of each fully connected layer is the result of fully connected processing based on the input of that layer.

[0108] It should be noted that in this embodiment of the invention, the second cross-fusion process can employ element-wise multiplication, multiplying the feature enhancement result element-by-element with the first cross-fusion result. It is understood that fully connected processing does not refer to a specific processing step in a particular fully connected layer, but rather to the collective processing of each fully connected layer in a multi-layered system. In practical applications, as deep learning refines its feature abstraction levels, the fully connected processing of multi-layered systems tends to converge. In this embodiment of the invention, by introducing feature enhancement results into each fully connected layer, the aim is to ensure that the first feature sample is always fully expressed, i.e., to obtain more training opportunities. Based on the second cross-fusion process, feature enhancement of specific features is ensured without affecting the overall modeling objective.

[0109] S304. Classify the results of the fully connected processing, and determine the model parameters of the prediction model based on the degree of matching between the classification results and the actual results, and then complete the construction of the prediction model based on the model parameters.

[0110] It should be noted that, in this embodiment of the invention, the classification process may include at least one of data classification based on different prediction scenarios and data confidence screening based on the results of fully connected processing. Furthermore, this embodiment of the invention can also determine the difference (matching degree) between the predicted value (the result of classification processing) and the actual value (the true result) by introducing a loss function during model training. Then, through the penalty mechanism of the loss function, backpropagation is performed to continuously optimize the model parameters of the prediction model. Finally, based on preset training conditions, the final optimized model parameters are determined to complete the construction of the prediction model. These training conditions include, but are not limited to, conditions for determining whether training should stop or exit, such as the number of training iterations reaching a preset threshold and the loss value of the loss function reaching a preset loss threshold.

[0111] For example, in the advertising industry where CTR and CVR prediction modeling is performed based on specific industry characteristics, the results of the fully connected processing can be classified according to data type for different prediction scenarios of CTR and CVR. In some optional embodiments, further filtering and classification are performed based on the confidence level of each data point in the fully connected processing results, eliminating data with significant bias. Then, based on the degree of matching between the classification results and the actual results (where the theoretical basis is solid, the expected results of relevant feature samples can also be used as the matching terms for the classification results), the model parameters of the prediction model are determined. In some optional embodiments, the model parameters can be continuously optimized through backpropagation using a loss function. Based on the determined model parameters, the prediction model is constructed.

[0112] In this embodiment of the invention, a first feature sample representing specific industry characteristics, a second feature sample representing object attribute characteristics, and a third feature sample representing general text characteristics are obtained. Then, the first feature sample is input into a feature enhancement module for feature enhancement processing to obtain the feature enhancement result. Simultaneously, the first, second, and third feature samples undergo a first cross-fusion processing to obtain a first cross-fusion result. Based on a general prediction model, this embodiment enhances the representation of the first feature sample, allowing the prediction model training process to better reflect the information of the first feature sample, thus increasing the influence of specific industry characteristics during model training. Next, the feature enhancement result and the first cross-fusion result undergo a second cross-fusion processing and are input into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result. Finally, the fully connected processing result is classified, and the model parameters of the prediction model are determined based on the matching degree between the classification result and the true result. The prediction model is then constructed based on the model parameters. This invention enhances the representation of specific features without affecting the training effect of the original large number of general features, improving the prediction accuracy of the prediction model and increasing the efficiency of advertising exposure allocation.

[0113] To effectively improve the processing efficiency of computing resources, the embodiments can optimize the data processing and computation process. In some possible embodiments, the step S302, which involves inputting the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, may include, but is not limited to, steps S3021 to S3022.

[0114] S3021. The first feature sample is vectorized to obtain the first feature vector;

[0115] It's important to note that training neural network models in deep learning typically requires massive amounts of data, which necessitates powerful computing capabilities. Vectorizing the first feature samples to obtain the first feature vector can effectively reduce the training time of the neural network. For example, after data vectorization, matrix operations can replace complex and cumbersome loop operations (such as for loops), thereby significantly reducing model training time.

[0116] For example, in this embodiment of the invention, the first feature sample can be vectorized by adding an embedding (embedding) layer after it. emb is a crucial application in deep learning; it literally translates to "vectorization" or "vector mapping." The main function of emb is to transform high-dimensional sparse vectors into low-dimensional dense vectors (i.e., to "represent" an object using a low-dimensional dense vector), facilitating processing by upper-layer deep neural networks. In deep learning prediction scenarios, due to the multimodal nature of feature data, encoding features inevitably leads to sparse feature sample vectors. The structural characteristics of deep learning are unfavorable for processing sparse feature vectors. Therefore, by adding an emb layer to transform these high-dimensional sparse features into low-dimensional dense vectors, the efficiency of deep learning can be significantly improved.

[0117] S3022. Input the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result.

[0118] The feature enhancement module includes at least two fully connected layers. In this embodiment of the invention, a personalized gate module can be introduced as the feature enhancement module to perform feature enhancement processing on the first feature vector, thereby obtaining the feature enhancement result. The structure of the gate module can be considered as a two-layer fully connected structure. The gate module can be understood as a two-layer fully connected operation, where the input and output are both vectors (the dimensions can be different), and the output vector is the feature enhancement result.

[0119] For example, in a scenario where MLP is used to implement the first cross-fusion processing and multiple fully connected layers, in order to adapt to changes in the data dimensions of each layer, the feature enhancement module can use multiple gate modules, such as... Figure 5 As shown, the feature enhancement results referenced in different processing levels (including the first cross-fusion process and each fully connected process) are obtained through multiple different gate modules (illustrated sub-modules). Since the gate modules uniformly use the first feature vector after vectorization of the first feature sample as input, the first fully connected layer of each gate module has the same dimension. However, the feature enhancement results output by each gate module need to undergo a second cross-fusion process (e.g., element-wise multiplication) with the output results of each layer of the MLP (including the first cross-fusion result and the fully connected processing results of each fully connected layer in the multi-layer fully connected layers). Therefore, the second fully connected layer of each gate module maintains the same dimension as the output results of each layer of the MLP.

[0120] In this embodiment of the invention, a first feature vector is obtained by vectorizing the first feature sample; this improves the efficiency of deep learning operations and thus significantly reduces the model training time; then, the first feature vector is input into a feature enhancement module for feature enhancement processing to obtain the feature enhancement result; by using a feature enhancement module (such as a gate module) that includes two fully connected layers, the dimensions of the two fully connected layers can be adaptively adjusted to match the data processing adaptability of deep learning.

[0121] It should be noted that in some embodiments, the first feature sample includes specific feature samples and advertising feature samples. The step S3021 of vectorizing the first feature sample to obtain the first feature vector may include, but is not limited to, step S30211:

[0122] S30211. Vectorize the specific feature samples and advertising feature samples in the first feature sample to obtain the first feature vector;

[0123] It should be noted that by vectorizing the specific feature samples and advertising feature samples of the first feature sample, and transforming them into the first feature vector, the training time of the neural network can be effectively reduced. For example, after vectorizing the data, matrix operations can be used to replace complex and cumbersome loop operations (such as for loop operations), thereby significantly reducing the model training time. Specifically, the specific feature samples can be promotional items in advertising scenarios (such as specific applications or mini-programs in the gaming industry, as well as virtual promotional items such as equipment and trading currency related to game content, and physical goods such as beverages and cosmetics in the e-commerce industry, and related promotional items in other industries); the advertising feature samples can be creative promotional schemes in advertising scenarios (including at least one or more combinations of images, text, audio, and video, such as promotional posters, copywriting, and videos) and advertising identifiers (such as the advertiser's ID and name).

[0124] For example, in advertising promotion scenarios, considering that industry-related characteristics are mainly reflected in promotional materials (specific feature samples) related to marketing objectives, in addition to these industry-specific feature samples, advertising feature samples (including the general basic features of advertising (advertising identifiers) and creative features (creative promotional schemes)) can also be used as input to the feature enhancement module to capture some cross signals within the advertisement. In embodiments of the present invention, such as... Figure 6As shown, the first feature sample is divided into specific feature samples and advertising feature samples. The first feature vector, obtained by vectorizing the specific and advertising feature samples, is then input into the feature enhancement module to generate feature enhancement results, which are used for subsequent model training. Simultaneously, the advertising feature samples also serve as general features, participating in regular model training along with the second and third feature samples (after a first cross-fusion). The process of vectorizing the specific and advertising feature samples to obtain the first feature vector can be as follows: First, two vector results are obtained by vectorizing the specific and advertising feature samples separately. Then, the first feature vector is obtained by synthesizing the two vector results (including but not limited to one or more combinations of splicing, cross-fertilization, or fusion). By enhancing the specific and advertising feature samples in the first feature sample, the weight of specific industry features in the prediction model training is ensured, preventing the meaning of relevant specific feature samples from being overwhelmed during training. This ensures the accuracy of the prediction model's prediction results for specific feature samples (such as promotional materials). At the same time, the participation of advertising feature samples in regular model training ensures the compatibility of the prediction model.

[0125] It should be noted that, in some embodiments, the step S3022, which involves inputting the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, may include, but is not limited to, steps S30221 to S30222:

[0126] S30221. The first feature vector is processed by two fully connected layers of the feature enhancement module in sequence.

[0127] It should be noted that the fully connected processing is used to enhance the first feature vector. The purpose of setting up two fully connected layers and performing fully connected processing in the feature enhancement module is that the fully connected processing in the first fully connected layer is mainly used for feature enhancement, while the fully connected processing in the second fully connected layer, in addition to feature enhancement, is used to further adjust the data dimension of the output feature enhancement result. This achieves data dimension matching and adaptation between the feature enhancement result and the first cross-fusion result, as well as the output results of each fully connected layer in the multi-layer fully connected layer (for the second cross-fusion processing). It is easy to understand that when the data dimensions of each fully connected layer in the multi-layer fully connected layer are different, the corresponding feature enhancement module includes several sub-modules containing two fully connected layers (such as the gate module corresponding to the number of fully connected layers in the multi-layer fully connected layer).

[0128] For example, in this embodiment of the invention, to adapt to changes in data dimensions at each layer, each sub-module of the feature enhancement module performs fully connected processing on the first feature vector through a first fully connected layer to achieve feature enhancement. Subsequently, each sub-module performs fully connected processing on the output of its own first fully connected layer through a second fully connected layer to further achieve feature enhancement. Simultaneously, each sub-module adjusts the data dimension of its output feature enhancement result according to the dimensions of the second fully connected layer and the corresponding MLP layers. It is easy to understand that since the input data of each sub-module is the first feature vector, the dimensions of the first fully connected layer of each sub-module are the same, and the dimension of the feature enhancement result output by each sub-module is the same as the dimension of the output of the corresponding MLP layers (which undergo the second cross-fusion processing). It is understood that the fully connected processing is not a specific processing step for a particular fully connected layer of a sub-module, but rather a general term for the processing of each fully connected layer in each sub-module. In practical applications, the data dimension of the second fully connected layer of each submodule needs to be the same as that of the corresponding MLP layers (including the layers that perform the first cross-fusion process and the fully connected layers of the multi-layer fully connected layers) that undergo the second cross-fusion process.

[0129] S30222. Constrain the fully connected processing results of the second fully connected layer through constraint functions, thereby limiting the feature enhancement results to the target interval.

[0130] It's important to note that constraint functions can be, for example, the sigmoid function and other constraint functions (such as regularization). The purpose of constraint processing is to limit the degrees of freedom in deep learning. Degrees of freedom are an important concept in statistics and engineering, often used to summarize the amount of data used when calculating sample statistics or statistical hypothesis testing statistics. In machine learning, degrees of freedom can refer to the number of parameters in a model, such as the number of coefficients in a linear regression model or the number of weights in a deep learning neural network. It's worth noting that if a machine learning model has too many degrees of freedom (i.e., model parameters), the model will overfit the training set, a well-known fact in statistics. This problem can be overcome by using regularization techniques, such as regularized linear regression and various regularization methods commonly used in deep learning neural network models.

[0131] For example, in predictive modeling, degrees of freedom typically refer to the number of parameters estimated from data in the model. This can encompass the model's coefficients as well as the data used to calculate model error. Higher degrees of freedom mean the model needs to fit more parameters, requiring a larger sample size; therefore, in some cases, it's necessary to use relevant techniques to restrict the degrees of freedom. For example... Figure 7As shown, in some embodiments, to limit the expressive freedom of the entire feature enhancement module, the output range of the second layer needs to be constrained by a constraint function. In this embodiment of the invention, two sigmoid functions are connected after the second fully connected layer of the feature enhancement module (the fully connected layer in the diagram corresponds to the fully connected layer) as constraint functions to constrain the fully connected processing result of the second fully connected layer. Using the form of a 2*sigmoid function, the output range can be limited to the interval [0,2]. It should be noted that the target interval of the feature enhancement result limitation range can be adaptively adjusted based on the application scenario.

[0132] It should be noted that in some embodiments, the step S3022, which involves inputting the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, may also include, but is not limited to, steps S30223 to S30224:

[0133] S30223. Add activation structures to the feature enhancement module;

[0134] It's important to note that activation structures are used to perform relevant activation processing to determine whether to activate the feature enhancement module. For example, when the activation module activates the feature enhancement module, the feature enhancement module performs corresponding feature enhancement processing to enhance the features of the first feature sample, thus allowing the enhanced feature result to be applied to the deep learning model training. Otherwise, if the activation module does not activate the feature enhancement module, the feature enhancement module does not perform feature enhancement processing, and the first feature sample is only applied to the deep learning model training in its normal form. Furthermore, since different industries have their own specific feature enhancement needs (e.g., the gaming industry needs to enhance specific industry features related to games, while the e-commerce industry needs to enhance specific industry features related to e-commerce advertising), the model structure can be made compatible with each industry through activation modules. This means setting up several feature enhancement modules corresponding to specific industry features related to each industry, and setting activation structures for each feature enhancement module. Activation processing of at least one activation structure can enable feature enhancement processing for a specific industry, or activation processing of multiple activation structures can enable feature enhancement processing covering a wider range of industries (or even feature enhancement processing across the entire industry).

[0135] S30224. In response to the activation operation, the fully connected processing results of the two fully connected layers in the feature enhancement module are activated through the activation structure, thereby determining the bias information of the feature enhancement results for the construction process of the prediction model.

[0136] It should be noted that activation operations can be implemented manually. Since different industries have their own feature enhancement needs, in scenarios where activation operations are performed, the operation / associated object trained on the model can be manually configured based on prior knowledge and the specific type of the first feature sample. For example, when the obtained first feature sample is a beverage product, it is generally easy to understand that it is a specific industry feature from the e-commerce industry. Typically, feature enhancement processing can be enabled only for the e-commerce industry. Then, the activation operation can be manually configured to activate the activation structure set in the feature enhancement module corresponding to the e-commerce industry, activating the fully connected processing results of the two fully connected layers in that feature enhancement module. This determines the bias information of the feature enhancement result for the prediction model construction process; the bias information is the prediction enhancement for the e-commerce industry.

[0137] For example, the activation structure can be implemented by adding a mask structure to the feature enhancement module. For instance, by setting the output feature enhancement result to 1 using the mask structure, subsequent feature enhancement results undergo a second cross-fusion with other features (such as the first cross-fusion result or the output results of each layer in a multi-layer fully connected layer). The result remains the original value of the other features, and the feature enhancement result does not produce a bias effect. Taking a scenario where the feature enhancement module uses the 2*sigmoid function as the constraint function as an example, such as... Figure 8 As shown, the mask structure (the activation structure shown in the diagram) is input to the 2*sigmoid function (the constraint function shown in the diagram). The mask structure can be in the form of ordinary 0 / 1. In practical applications, when it is necessary to only apply to e-commerce advertisements and only enhance the specific industry features related to e-commerce advertisements, the mask structure of the feature enhancement module corresponding to other industries other than e-commerce is set to 0 and the output of the second fully connected layer of the relevant gate module is set to 1 (i.e., the feature enhancement result is 1). The mask structure of the feature enhancement module corresponding to the e-commerce industry is set to 1 and the 2*sigmoid function is set to 1, thus retaining the feature enhancement result of the e-commerce industry and making it only effective for e-commerce advertisements.

[0138] It should be noted that, in some embodiments, the step S30211, which involves vectorizing specific feature samples and advertising feature samples in the first feature sample to obtain the first feature vector, may include, but is not limited to, steps S302111 to S302112:

[0139] S302111. Perform gradient isolation processing on the advertising feature samples to obtain the gradient isolation processing result of the advertising feature samples;

[0140] It should be noted that in advertising scenarios, since industry-related features are mainly reflected in the promotional materials (specific feature samples) related to the marketing objectives, in addition to these industry-specific feature samples, advertising feature samples (including the general basic features of advertising (advertising identifiers) and creative features (creative promotional schemes)) can also be used as input to the feature enhancement module to capture some cross signals within the advertisement. It's easy to understand that in the specific application of advertising promotion, the focus is still on the promotional materials (specific feature samples) related to the marketing objectives. However, since advertising feature samples undergo feature enhancement processing and participate in regular model training as general features along with the second and third feature samples (after the first cross-fusion), and participate in the second cross-fusion processing during subsequent model training, to avoid the gradients generated by advertising feature samples in the feature enhancement module affecting the general features during model training, gradient isolation processing is performed on the advertising feature samples while introducing them for feature processing. This ensures that advertising feature samples within the specific industry features do not interfere with the normal training of the model.

[0141] For example, when selecting inputs for the feature enhancement module, in addition to industry-specific promotional materials (specific feature samples), some general basic features of advertisements (advertisement identifiers) and creative features (creative promotion schemes) are also included, i.e., advertisement feature samples. However, these advertisement feature samples are also input into multiple fully connected layers after the first cross-fusion, which has a significant impact. Therefore, as... Figure 9 As shown, gradient isolation is required when advertising feature samples are used as input to the feature enhancement module. Gradient isolation can be achieved using the `stop_gradient` function, a TensorFlow function used to label a tensor during backpropagation, indicating that its gradient should not be calculated during backpropagation. This is useful in many situations, such as when training a model and needing to input known constants whose gradients do not require calculation; `stop_gradient` can be used to label these tensors. Model training can be viewed as an iterative solution process, and gradient isolation can be simply understood as not using the current gradient to update parameters. Gradient isolation ensures that the gradients of advertising feature samples in the feature enhancement module do not affect the training of the multi-layer fully connected model. While this restricts the model's degrees of freedom, it avoids redundant information affecting the feature enhancement module, ensuring that industry-specific features do not interfere with the normal training of the model.

[0142] S302112. Vectorize the specific feature samples and gradient isolation processing results to obtain the first feature vector.

[0143] It should be noted that vectorizing the gradient isolation results obtained by performing gradient isolation processing on specific feature samples and advertising feature samples, and transforming them into the first feature vector, can effectively reduce the training time of neural networks. For example, after data vectorization, matrix operations can be used to replace complex and cumbersome loop operations (such as for loop operations), thereby significantly reducing the model training time.

[0144] For example, in this embodiment of the invention, the first feature vector, obtained by vectorizing the gradient isolation processing result (obtained by gradient isolation processing of specific feature samples and advertising feature samples), is input into the feature enhancement module to generate a feature enhancement result, which is then used for subsequent model training. Simultaneously, the advertising feature samples also serve as general features, participating in regular model training along with the second and third feature samples (after a first cross-fusion). The process of vectorizing the specific feature samples and the gradient isolation processing result to obtain the first feature vector can be as follows: Two vector results are obtained by vectorizing the specific feature samples and the gradient isolation processing result respectively; then, the first feature vector is obtained by synthesizing the two vector results (including but not limited to one or more combinations of splicing, cross-fertilization, or fusion). Feature enhancement is achieved by using gradient isolation processing results obtained from gradient isolation processing of specific feature samples and advertising feature samples. This ensures the weight of specific industry features in the training of the prediction model, preventing the meaning of relevant specific feature samples from being overwhelmed during training. This guarantees the accuracy of the prediction model's prediction results for specific feature samples (such as promotional items). At the same time, the gradient isolation processing results obtained from gradient isolation processing of advertising feature samples are used in regular model training, ensuring the compatibility of the prediction model and avoiding the impact of redundant information on the feature enhancement module. This ensures that industry-specific features do not interfere with the normal training of the model.

[0145] It should be noted that, in some embodiments, the step S30221, in which the first feature vector is processed sequentially through the two fully connected layers of the feature enhancement module, may include, but is not limited to, steps S302211 to S302212:

[0146] S302211. Add a matrix factorization layer between the two fully connected layers of the feature enhancement module;

[0147] It should be noted that the number of industry-specific features requiring enhancement is typically much smaller than the number of regular features (second feature samples representing object attribute features of the target object and third feature samples representing general text features across all industries). The feature enhancement results output by the feature enhancement module need to be matched and adapted to the data dimensions of the first cross-fusion result and the outputs of each fully connected layer in the multi-layered fully connected system (for the second cross-fusion process). In the case of performing the second cross-fusion process between the feature enhancement result and the first cross-fusion result, a relatively large matrix multiplication operation is generated, leading to performance degradation during online inference. Therefore, network structure pruning is necessary. A matrix factorization approach can be introduced by adding a matrix factorization layer between the two fully connected layers of the feature enhancement module to simplify the inference computation.

[0148] For example, in a scenario where the feature enhancement module includes several sub-modules containing two fully connected layers (such as a gate module corresponding to the number of fully connected layers in a multi-layered system), the first sub-module (gate module) and the first cross-fusion result are then subjected to a second cross-fusion process. This easily results in a relatively large matrix multiplication operation, leading to performance degradation during online inference. Therefore, as... Figure 10 As shown, by adding a matrix factorization layer as an intermediate layer between two fully connected layers (and the dimension of this intermediate layer is much smaller than the 2*sig output layer), a large matrix operation is transformed into an operation of two smaller matrices. (For example, if the current layer dimension is M and the next layer dimension is N, then a fully connected operation is an M*N matrix operation with a large number of parameters. However, by adding a matrix factorization layer with a smaller dimension, P, the fully connected operation with a large number of parameters becomes a fully connected operation of two low-dimensional products, with the parameters being M*P + P*N), greatly reducing the computational complexity. It should be noted that other sub-modules (gate modules) can determine whether to add a matrix factorization layer based on the actual application scenario. Generally, due to the convergence trend of multiple fully connected layers, the data dimension for the subsequent second cross-fusion processing is already small enough, so other sub-modules (gate modules) do not need to add a matrix factorization layer. It should also be noted that in some embodiments of this invention, the matrix factorization layer can be placed between the fully connected layer and the constraint function.

[0149] S302212. After performing matrix decomposition on the processing result of the first fully connected layer through the matrix decomposition layer, the second fully connected layer performs fully connected processing on the matrix decomposition result to obtain the feature enhancement result.

[0150] For example, taking a scenario where the dimensions of the two layers before and after the first fully connected layer performing the fully connected processing are M and N respectively, the fully connected processing of the first fully connected layer is typically an M*N matrix operation. Based on this, in this embodiment of the invention, a small-dimensional matrix decomposition layer (dimension P) is added to decompose the fully connected processing of the first fully connected layer into the product of two low-rank matrices, with parameters M*P + P*N, which significantly reduces computational complexity. Furthermore, the matrix decomposition result (the product of low-rank matrices) is then fully connected through a second fully connected layer, further reducing subsequent computational complexity. Therefore, in this embodiment of the invention, the matrix decomposition layer decomposes the high-complexity matrix operation (fully connected processing) with large parameters into the product of two low-rank matrices. By compressing the matrix, a matrix can be stored in as little space as possible, ensuring efficient use of space resources while minimizing information loss, thus reducing computational complexity while ensuring the integrity of information utilization. Furthermore, dimensionality reduction can also reduce the occurrence of the curse of dimensionality.

[0151] It should be noted that in some embodiments, the step of obtaining the first feature sample in step S301 may include, but is not limited to, steps S3011 to S3012:

[0152] S3011. Obtain candidate feature samples;

[0153] It's important to note that for a model, using 10 features is obviously more convenient for interpretation than using 50. Reducing the number of features helps risk control and business understand the model, especially in scenarios like predictive behavior in the advertising industry where interpretability is crucial. The purpose of obtaining candidate feature samples is to list all possible feature samples for inclusion in the model, facilitating subsequent screening to determine the final specific feature samples. It's easy to understand that reducing the number of features requires first identifying all features associated with the model, avoiding ignoring highly interpretable feature samples. Only then can feature screening be performed according to preset conditions to ensure the optimal feature set for inclusion in the model is obtained from the candidate feature samples containing all feature samples.

[0154] For example, in this embodiment of the invention, based on the definition of specific industry characteristics of the target industry and their correlation with the model, all feature samples that meet the definition or have a correlation with the prediction model to be trained can be obtained as candidate feature samples. In some optional embodiments, the candidate feature samples can be further arranged systematically by preset conditions (such as defining matching degree, feature correlation, etc.) to speed up the efficiency of subsequent feature selection.

[0155] S3012. Perform feature screening on candidate feature samples and determine multiple candidate feature samples with feature relevance higher than the threshold as specific feature samples.

[0156] It should be noted that feature selection is a crucial step in risk control modeling. Its goal is to find the optimal subset of features to improve model performance, reduce training time, and enhance interpretability. The purpose of feature selection is:

[0157] 1. Reduce the risk of overfitting. Overfitting means that the model's performance differs significantly between the training and test sets. The main causes of overfitting include: 1) Insufficient training samples and incomplete data; 2) Excessive noise in the training data, which the model learns; 3) Overly complex models, where the model memorizes the patterns in the training data but fails to adapt to unknown prediction data, resulting in poor generalization ability. Having too many features increases model complexity and introduces more noise, making the model more susceptible to learning noise and increasing the risk of overfitting. It's important to note that having more features does not cause overfitting, but rather makes it easier to overfit; the two are not inherently related.

[0158] 2. Improve interpretability. For a model, it is usually easier to interpret a small number of precise features compared to a large number of generalized features. Reducing the number of features helps risk control and business understand the model, especially in scenarios with high interpretability requirements, such as predictive behavior in the advertising industry.

[0159] 3. Improve model training speed and save storage space. Time costs must also be considered during the modeling process, especially when dealing with large amounts of data and using ensemble models; feature selection is therefore essential.

[0160] For example, in this embodiment of the invention, prior knowledge can be used to make relevant judgments to filter candidate feature samples, thereby determining multiple candidate feature samples with feature relevance higher than a threshold as specific feature samples. In addition to determining specific industry features requiring feature enhancement based on prior knowledge, mature feature filtering tools, such as learnable feature selection methods based on feature complexity and variational dropout, can be used to rank all features according to a unified metric, selecting some top-ranked specific industry features (including but not limited to those determined based on a preset metric threshold and a preset feature selection quantity) as input to the feature enhancement module. This can achieve a balance between effectiveness and performance to some extent. It should be noted that in scenarios where the first feature sample includes specific feature samples and advertising feature samples, feature filtering for specific feature samples can be performed using relevant judgment rules formulated based on prior knowledge, and feature filtering for advertising feature samples can be performed using learnable feature selection methods.

[0161] It should be noted that, in some embodiments, the step S302, which involves performing a first cross-fusion process on the first feature sample, the second feature sample, and the third feature sample to obtain a first cross-fusion result, may include, but is not limited to, steps S3023 to S3024:

[0162] S3023. Vectorize the first feature sample, the second feature sample, and the third feature sample to obtain the first feature vector, the second feature vector, and the third feature vector;

[0163] It should be noted that vectorizing the first, second, and third feature samples into first, second, and third feature vectors can effectively reduce the training time of the neural network. For example, after data vectorization, matrix operations can replace complex and cumbersome loop operations (such as for loops), thereby significantly reducing model training time. Taking an advertising promotion scenario as an example, the first feature sample can be determined from the advertiser's estimated advertising resources; the second feature sample can be, with the target object's permission and authorization, the object attribute features (including but not limited to preference tags and other basic attribute features) related to all promotional objects in the specific promotion scenario or some promotional objects adapted based on preset conditions; the third feature sample can be determined through basic tag information such as date, time, and device type. In some embodiments, the first feature sample includes a specific feature sample and an advertising feature sample. The specific feature sample can be promotional items in the advertising promotion scenario (such as specific applications or mini-programs in the game industry, as well as virtual promotional items such as equipment and trading currency related to game content, physical goods such as beverages and cosmetics in the e-commerce industry, and related promotional items in other industries); the advertising feature sample can be creative promotional schemes in the advertising promotion scenario (including at least one or a combination of multiple of images, text, audio and video, such as promotional posters, copywriting and videos, etc.) and advertising identifiers (such as the advertiser's ID identifier and name, etc.).

[0164] For example, the first, second, and third feature samples can be vectorized by adding an embedding layer after them. Taking the prediction model for advertising promotion as an example, the first feature vector is obtained by vectorizing the relevant features of the advertising resources to be predicted from the advertiser. The second feature vector is obtained by vectorizing the relevant features of all promotional objects in the promotion scenario or some promotional objects adapted based on preset conditions (including but not limited to preference tags and other basic attribute features). At the same time, the third feature vector is obtained by vectorizing the relevant features of basic tag information such as date, time, and device type. Then, the first, second, and third feature vectors can be used for subsequent first cross-fusion processing and fully connected processing of multiple fully connected layers for model training. It should be noted that in some embodiments, the first feature vector is not only used for the first cross-fusion process, but also for the feature enhancement process. Correspondingly, the first feature vector includes a fourth feature vector and a fifth feature vector. The fourth feature vector is obtained by vectorizing specific feature samples (and advertising feature samples) and is used for feature enhancement processing; the fifth feature vector is obtained by vectorizing advertising feature samples and is used for the first cross-fusion process.

[0165] S3024. Perform a first cross-fusion process on the first feature vector, the second feature vector, and the third feature vector to obtain the first cross-fusion result.

[0166] It should be noted that the first cross-fusion of the first, second, and third feature vectors can be achieved using an MLP (Multilayer Perceptron). The role of feature cross-fusion is to reduce the weights of unimportant features and lower the weights of redundant features to improve model performance. Since a wide variety of feature samples often involves multimodal information processing, feature fusion refers to leveraging the complementarity between features given different attributes to combine their advantages, thereby improving model performance.

[0167] For example, in a scenario utilizing a multilayer perceptron, the first feature vector, second feature vector, and third feature vector are first subjected to arbitrary-dimensional feature cross-interaction using the multilayer perceptron. Further feature fusion is then performed to achieve the first cross-fusion process. Feature fusion can be achieved using functions such as `concat`, `concat_ws`, and other feature fusion functions, resulting in the first cross-fusion result. This first cross-fusion result serves as the input basis for subsequent multilayer fully connected layers in model training. Because this embodiment of the invention enhances the representation of the first feature sample based on a general prediction model, the training process of the prediction model can better reflect the information of the first feature sample, increasing the influence of specific industry features in the model training process. Therefore, this embodiment of the invention, based on the first cross-fusion result, fully preserves the training effect of the original large number of general features, avoiding the enhancement of specific feature bias.

[0168] Referring to the attached diagrams in the instruction manual Figure 4 The complete implementation process of the prediction model training method in the technical solution of this invention is described below:

[0169] Taking an advertising recommendation scenario as an example, firstly, the embodiment determines a first feature sample based on the advertising resources to be estimated from the advertiser; then, with the permission and authorization of the target object, a second feature sample is determined based on the object attribute features (including but not limited to preference tags and other basic attribute features) related to all promotion objects in the specific promotion scenario or some promotion objects adapted based on preset conditions; simultaneously, a third feature sample is determined based on basic tag information such as date, time, and device type. The first feature sample serves as the feature enhancement target for the subsequent model training process, and the first feature sample, along with the second and third feature samples, also serve as samples for the model's regular training.

[0170] Then, the feature enhancement module performs feature enhancement processing on the first feature sample to obtain the feature enhancement result; and, using MLP (Multilayer Perceptron) to perform feature cross-interaction of arbitrary dimensions, further feature fusion is performed to achieve the first cross-fusion processing to obtain the first cross-fusion result. Feature fusion can be achieved by functions such as concat, concat_ws and other feature fusion functions.

[0171] Based on the enhancement result and the first cross-fusion result, element-wise multiplication is used for the second cross-fusion process. This involves multiplying the feature enhancement result and the first cross-fusion result element-wise, and then inputting the result into a multi-layer fully connected layer (this example uses three fully connected layers, corresponding to three layers of fully connected processing) for fully connected processing, yielding the fully connected processing result. Specifically, the input to the first fully connected layer of the three layers is the result of the second cross-fusion process involving the feature enhancement result and the first cross-fusion result. The inputs to the subsequent second and third fully connected layers are the result of element-wise multiplication of the outputs of the first and second fully connected layers with the feature enhancement result, respectively. The fully connected processing result is essentially the result of element-wise multiplication of the output of the third fully connected layer of the three-layer fully connected layer with the feature enhancement result. The output of each fully connected layer is the result of fully connected processing based on the input of that layer.

[0172] Finally, the results of the fully connected processing are divided into CTR prediction results and CVR prediction results based on different prediction scenarios. Then, the model parameters of the prediction model are determined according to the degree of matching between the CTR prediction results and the actual results (when the data theory is solid, the expected results of relevant feature samples can also be used as the matching items for the classification processing results). Then, the CTR prediction model or CVR prediction model is constructed according to the determined model parameters.

[0173] like Figure 11 As shown, this embodiment of the invention also discloses a data processing method, which may include, but is not limited to, steps T001 to T002:

[0174] T001. Obtain the advertising data to be estimated;

[0175] For example, the implementation can obtain the advertising resources to be estimated input by the advertiser in the target application through the target_CPA interface, so as to further estimate the advertising exposure value of the advertising data to be estimated according to the target estimation model pre-stored on the server (such as a pre-trained and constructed estimation model), and then determine the allocation of advertising resources based on the advertising exposure value. The advertising data to be estimated may include promotional materials for advertising marketing objectives, creative solutions for advertising marketing methods (such as marketing posters, marketing audio / video, etc.), and advertising identifiers related to the advertiser. In some embodiments, it may also include the advertiser's bid for advertising allocation. The target application can be an information interaction platform for advertising promotion, or a web link, mini-program, or application plugin with advertising promotion information interaction function set in multimedia applications (such as film and television applications, short video applications, or music applications, etc.), social applications, game applications, and navigation applications.

[0176] T002. Based on the target prediction model, predict the advertising exposure value of the advertising data to be predicted, and then determine the allocation of advertising resources based on the advertising exposure value.

[0177] It should be noted that the target prediction model is determined according to the aforementioned prediction model training method. The prediction model constructed through pre-training serves as the target model, used to predict the ad exposure value of the ad data to be predicted. Depending on the prediction type, this includes, but is not limited to, CTR prediction and CVR prediction. In some optional embodiments, based on the predicted CTR and CVR, ECPM is further determined using the formula for calculating the ad revenue per thousand impressions: ECPM = CTR * CVR * target_CPA * alpha. Here, target_CPA represents the advertiser's bid in ad allocation and is obtained along with the ad data to be predicted; alpha, the platform's dynamically adjusted weight, can be determined based on the model's pre-training parameters or through real-time monitoring and adjustment based on prior knowledge. Furthermore, ad resources can be allocated based on the determined ad exposure value, facilitating the rationalization of ad resource allocation based on accurate ad exposure value prediction.

[0178] For example, the embodiment uses a pre-trained target prediction model to predict CTR and CVR based on the acquired advertising resources to be predicted. It should be noted that CTR and CVR predictions can be obtained using the same prediction model with different parameter adjustments, or by prediction models trained with different biases. Furthermore, based on the predicted CTR and CVR, the expected advertising revenue per thousand impressions can be determined using the ECPM calculation formula. Then, advertising resources are allocated based on at least one of the predicted CTR, CVR, and calculated ECPM. Furthermore, based on the predicted CTR, CVR, and calculated ECPM, the total transaction value (GMV) can be predicted to determine the expected campaign cost. In some optional embodiments, the advertising resource allocation can be adaptively adjusted based on the expected campaign cost planning scenarios (including over-cost, under-cost, and cost break-even) to determine the final advertising resource allocation scheme.

[0179] The complete process of the data processing method in the embodiments of the present invention is described below with reference to the accompanying drawings:

[0180] The advertising prediction method proposed in this invention is executed on a server equipped with an interface for obtaining advertising data to be predicted (such as the target_CPA interface and the target application that sets the target_CPA interface). First, the terminal that interacts with the aforementioned server obtains the advertising data to be predicted input by the advertiser or other prediction execution object based on the interface for obtaining advertising data to be predicted. Then, the target prediction model that has been pre-trained and constructed in the server is called. The target prediction model is determined according to the aforementioned prediction model training method.

[0181] Using a target prediction model, CTR and CVR are predicted based on the acquired advertising resources to be predicted. Then, based on the predicted CTR and CVR, the expected advertising revenue per thousand impressions can be determined using the ECPM calculation formula. Furthermore, advertising resources are allocated to ad exposure based on at least one of the predicted CTR, CVR, and calculated ECPM. Specifically, based on the predicted CTR, CVR, and calculated ECPM, the total transaction value (GMV) can be predicted to determine the expected campaign cost. In some optional embodiments, the allocation of advertising resources to ad exposure can be adaptively adjusted based on the expected campaign cost planning scenario (including over-cost, under-cost, and cost break-even) to determine the final advertising resource allocation scheme.

[0182] like Figure 12 As shown, this embodiment of the invention also provides a prediction model training device, which includes:

[0183] The first module 1201 is used to acquire a first feature sample, a second feature sample, and a third feature sample; wherein, the first feature sample is used to characterize the specific industry characteristics of the target industry; the second feature sample is used to characterize the object attribute characteristics of the target object; and the third feature sample is used to characterize the general text features of all industries.

[0184] The second module 1202 is used to input the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and to perform a first cross-fusion processing on the first feature sample, the second feature sample and the third feature sample to obtain the first cross-fusion result;

[0185] The third module 1203 is used to perform a second cross-fusion process on the feature enhancement result and the first cross-fusion result, and then input the result into a multi-layer fully connected layer for fully connected processing to obtain a fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the result of the second cross-fusion process on the output of the previous fully connected layer and the feature enhancement result;

[0186] The fourth module 1204 is used to classify the results of the fully connected processing, and determine the model parameters of the prediction model based on the degree of matching between the classification results and the actual results, and then complete the construction of the prediction model based on the model parameters.

[0187] Combined with appendix Figure 12The specific implementation process of the prediction model training device provided by the present invention is described as follows: First, the first module 1201 acquires a first feature sample, a second feature sample, and a third feature sample. Specifically, candidate feature samples can be acquired; the candidate feature samples are screened to determine multiple candidate feature samples with feature relevance higher than a threshold as specific feature samples. Then, the second module 1202 inputs the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and performs a first cross-fusion processing on the first feature sample, the second feature sample, and the third feature sample to obtain the first cross-fusion result. Specifically, the first feature sample can be vectorized to obtain a first feature vector; the first feature vector is input into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result. The third module 1203 performs a second cross-fusion processing on the feature enhancement result and the first cross-fusion result, and then inputs them into a multi-layer fully connected layer for fully connected processing to obtain the fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the output of the previous fully connected layer and the result of the second cross-fusion processing on the feature enhancement result. Finally, the fourth module 1204 classifies the results of the fully connected processing and determines the model parameters of the prediction model based on the degree of matching between the classification results and the actual results. Then, the prediction model is constructed based on these parameters. This embodiment of the invention improves the accuracy of advertising exposure prediction by enhancing the representation of specific industry-specific features during the prediction model training process.

[0188] like Figure 13 As shown, this embodiment of the invention also provides a data processing apparatus, which includes:

[0189] Module 5, 1301, is used to obtain advertising data to be estimated.

[0190] Module 6 1302 is used to estimate the advertising exposure value of the advertising data to be estimated based on the target prediction model, and then determine the allocation of advertising resources based on the advertising exposure value.

[0191] The target prediction model is determined based on the aforementioned prediction model training device.

[0192] Combined with appendix Figure 13 The specific implementation process of the data processing device provided by the present invention is described as follows: First, the fifth module 1301 acquires the advertising data to be estimated. Second, the sixth module 1302 estimates the advertising exposure value of the advertising data to be estimated according to the target estimation model, and then determines the allocation of advertising resources based on the advertising exposure value. Specifically, the target estimation model can be determined according to any one of the estimation model training devices in the above embodiments. The embodiments of the present invention improve the accuracy of advertising exposure estimation through the data processing device.

[0193] like Figure 14 As shown, this embodiment of the invention also provides an electronic device, which includes a processor 1401 and a memory 1402; the memory 1402 stores a program; the processor 1401 executes the program to perform the aforementioned prediction model training method or data processing method; the electronic device has the function of carrying and running the business data processing software system provided in this embodiment of the invention, such as a personal computer (PC), mobile phone, smartphone, personal digital assistant (PDA), wearable device, handheld computer (PPC), tablet computer, vehicle terminal, etc.

[0194] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the aforementioned prediction model training method or data processing method. Simultaneously, this invention also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned prediction model training method or data processing method.

[0195] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 3 The method shown.

[0196] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0197] From the embodiments provided in the above specification, it is clear that the technical solution of the present invention has at least the following beneficial effects:

[0198] The technical solution of this invention inputs a first feature sample into a feature enhancement module for feature enhancement processing to obtain a feature enhancement result. Simultaneously, it performs a first cross-fusion process on the first, second, and third feature samples to obtain a first cross-fusion result. Based on this, this invention, on the basis of a general prediction model, enhances the feature expression of the first feature sample, enabling the prediction model training process to better reflect the information of the first feature sample and increasing the influence of specific industry features in the model training process. Then, in this embodiment, the feature enhancement result and the first cross-fusion result undergo a second cross-fusion process before being input into a multi-layer fully connected layer for fully connected processing to obtain a fully connected processing result. Finally, the fully connected processing result is classified, and the model parameters of the prediction model are determined based on the degree of matching between the classification result and the true result. The prediction model is then constructed based on the model parameters. This invention enhances the expression of specific features without affecting the training effect of a large number of original general features, improving the prediction accuracy of the prediction model and thus improving the efficiency of advertising exposure allocation.

[0199] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0200] Furthermore, although the invention has been described in the context of functional modules, it is understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0201] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0202] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0203] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0204] It is understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0205] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0206] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0207] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for training an advertising exposure prediction model, characterized in that, include: A training sample is obtained, and features are extracted from the training sample to obtain a first feature sample, a second feature sample, and a third feature sample. The training sample includes sample advertising data to be estimated, the object attributes of the target object of the sample advertising data, and general text for all industries in the advertising promotion scenario. The first feature sample is used to characterize the specific industry characteristics of the target industry to which the sample advertising data to be estimated belongs. The second feature sample is used to characterize the object attribute characteristics of the target object of the sample advertising data. The third feature sample is used to characterize the general text features for all industries in the advertising promotion scenario. The first feature sample is input into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result. The first feature sample, the second feature sample and the third feature sample are then subjected to a first cross-fusion processing to obtain the first cross-fusion result. After performing a second cross-fusion process on the feature enhancement result and the first cross-fusion result, the result is input into a multi-layer fully connected layer for full-connected processing to obtain a fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the result of performing the second cross-fusion process on the output of the previous fully connected layer and the feature enhancement result; The results of the fully connected processing are classified to obtain classification results; wherein, the classification results are used to characterize the predicted click pass rate or click conversion rate of the target object to the sample advertising data; Based on the degree of matching between the classification results and the actual results, the model parameters of the advertising exposure prediction model are determined, and then the advertising exposure prediction model is constructed based on the model parameters.

2. The method for training an advertising exposure prediction model according to claim 1, characterized in that, The step of inputting the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result includes: The first feature sample is vectorized to obtain the first feature vector. The first feature vector is input into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result; The feature enhancement module includes at least two fully connected layers.

3. The method for training an advertising exposure prediction model according to claim 2, characterized in that, The first feature sample includes specific feature samples and advertising feature samples. The vectorization process of the first feature sample to obtain the first feature vector includes: The specific feature sample and the advertising feature sample in the first feature sample are vectorized to obtain the first feature vector.

4. The method for training an advertising exposure prediction model according to claim 2, characterized in that, The step of inputting the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result includes: The first feature vector is processed sequentially through the two fully connected layers of the feature enhancement module. The fully connected processing results of the second fully connected layer are constrained by a constraint function, thereby limiting the feature enhancement results to the target range.

5. A method for training an advertising exposure prediction model according to claim 2 or 4, characterized in that, The step of inputting the first feature vector into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result further includes: Add an activation structure to the feature enhancement module; In response to the activation operation, the fully connected processing results of the two fully connected layers in the feature enhancement module are activated through the activation structure, thereby determining the bias information of the feature enhancement results for the construction process of the advertising exposure prediction model.

6. The method for training an advertising exposure prediction model according to claim 3, characterized in that, The step of vectorizing the specific feature sample and the advertising feature sample in the first feature sample to obtain the first feature vector includes: The advertising feature samples are subjected to gradient isolation processing to obtain the gradient isolation processing result of the advertising feature samples; The specific feature sample and the gradient isolation processing result are vectorized to obtain the first feature vector.

7. The method for training an advertising exposure prediction model according to claim 4, characterized in that, The step of sequentially performing fully connected processing on the first feature vector through the two fully connected layers of the feature enhancement module includes: A matrix factorization layer is added between the two fully connected layers of the feature enhancement module; After the matrix factorization layer performs matrix factorization on the result of the first fully connected layer, the second fully connected layer performs fully connected processing on the result of the matrix factorization to obtain the feature enhancement result.

8. The method for training an advertising exposure prediction model according to claim 1, characterized in that, The step of extracting features from the training samples to obtain the first feature sample includes: Feature extraction is performed on the training samples to obtain candidate feature samples; The candidate feature samples are screened to determine multiple candidate feature samples whose feature relevance is higher than a threshold as specific feature samples.

9. The method for training an advertising exposure prediction model according to claim 1, characterized in that, The first cross-fusion process of the first feature sample, the second feature sample, and the third feature sample to obtain the first cross-fusion result includes: The first feature sample, the second feature sample, and the third feature sample are vectorized to obtain a first feature vector, a second feature vector, and a third feature vector. The first feature vector, the second feature vector, and the third feature vector are subjected to a first cross-fusion process to obtain the first cross-fusion result.

10. A method for allocating advertising exposure resources, characterized in that, include: Obtain the target advertising data to be estimated; The advertising exposure value of the target advertising data is estimated based on the advertising exposure prediction model, and then advertising exposure resources are allocated to the target advertising data based on the advertising exposure value; wherein, the advertising exposure value includes the click pass rate or click conversion rate of the target advertising data; The advertising exposure prediction model is determined according to the advertising exposure prediction model training method according to any one of claims 1-9.

11. A training device for an advertising exposure prediction model, characterized in that, include: The first module is used to acquire training samples and extract features from the training samples to obtain a first feature sample, a second feature sample, and a third feature sample. The training samples include sample advertising data to be estimated, the object attributes of the target objects of the sample advertising data, and general text for all industries in the advertising promotion scenario. The first feature sample is used to characterize the specific industry characteristics of the target industry to which the sample advertising data to be estimated belongs. The second feature sample is used to characterize the object attribute characteristics of the target objects of the sample advertising data. The third feature sample is used to characterize the general text features for all industries in the advertising promotion scenario. The second module is used to input the first feature sample into the feature enhancement module for feature enhancement processing to obtain the feature enhancement result, and to perform a first cross-fusion processing on the first feature sample, the second feature sample and the third feature sample to obtain a first cross-fusion result. The third module is used to perform a second cross-fusion process on the feature enhancement result and the first cross-fusion result, and then input the result into a multi-layer fully connected layer for full-connected processing to obtain a fully connected processing result; wherein, the input data of each fully connected layer in the multi-layer fully connected layer includes the result of the second cross-fusion process on the output of the previous fully connected layer and the feature enhancement result. The fourth module is used to classify the results of the fully connected processing to obtain classification results; wherein, the classification results are used to characterize the predicted click pass rate or click conversion rate of the target object to the sample advertising data; based on the degree of matching between the classification results and the actual results, the model parameters of the advertising exposure prediction model are determined, and then the advertising exposure prediction model is constructed based on the model parameters.

12. A device for allocating advertising exposure resources, characterized in that, include: The fifth module is used to obtain the target advertising data to be estimated; The sixth module is used to estimate the advertising exposure value of the target advertising data based on the advertising exposure prediction model, and then allocate advertising exposure resources to the target advertising data according to the advertising exposure value; wherein, the advertising exposure value includes the click pass rate or click conversion rate of the target advertising data; The advertising exposure prediction model is determined by the advertising exposure prediction model training device according to claim 11.

13. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the advertising exposure prediction model training method as described in any one of claims 1 to 9 or the advertising exposure resource allocation method as described in claim 10.

14. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the advertising exposure prediction model training method as described in any one of claims 1 to 9 or the advertising exposure resource allocation method as described in claim 10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the advertising exposure prediction model training method as described in any one of claims 1 to 9 or the advertising exposure resource allocation method as described in claim 10.

Images