Feature processing method

Abstract

The embodiment of the specification provides a feature processing method, wherein the feature processing method comprises: acquiring an initial feature sequence, and selecting a feature to be masked in the initial feature sequence; performing mask processing on the feature to be masked, generating an intermediate feature sequence according to a mask processing result, and predicting a predicted feature corresponding to the feature to be masked according to the intermediate feature sequence; calculating a predicted feature score corresponding to the predicted feature, and updating the initial feature sequence into a target feature sequence according to the predicted feature score and feature scores respectively corresponding to features in the intermediate feature sequence; and inputting the target feature sequence into an object click prediction model for processing to obtain object click prediction information.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of machine learning technology, and in particular to feature processing methods. Background Technology

[0002] With the rapid development of computer and internet technologies, recommender systems have become an indispensable part of e-commerce, social media, and content platforms. Click-through rate (CTR) prediction models, as a core component of recommender systems, are crucial for improving user experience and platform revenue. Traditional CTR prediction primarily relies on a discriminative paradigm, modeling input feature interactions through a simple binary classification objective. However, with the successful application of generative models in natural language processing and computer vision, more and more research is exploring their introduction into CTR prediction to overcome the limitations of the traditional discriminative paradigm. While existing generative models can efficiently learn representations using a generative paradigm during the training phase, they often revert to the standard discriminative paradigm during online inference, failing to fully leverage their powerful generative capabilities to further improve prediction accuracy. This fundamental asymmetry between the training and inference phases limits the potential of the generative paradigm in CTR prediction. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide feature processing methods. One or more embodiments of this specification also relate to feature processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address technical deficiencies in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a feature processing method is provided, comprising: Obtain an initial feature sequence, and select features to be masked from the initial feature sequence; The feature to be masked is subjected to masking processing, an intermediate feature sequence is generated based on the masking processing result, and the predicted feature corresponding to the feature to be masked is predicted based on the intermediate feature sequence. Calculate the predicted feature score corresponding to the predicted feature, and update the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. The target feature sequence is input into the object click prediction model for processing to obtain object click prediction information.

[0005] According to a second aspect of the embodiments of this specification, another feature processing method is provided, including: Obtain the historical transaction behavior feature sequence of the target user, and select the feature to be masked from the historical transaction behavior feature sequence; Masking processing is performed on the features to be masked, an intermediate transaction behavior feature sequence is generated based on the masking processing result, and the predicted features corresponding to the features to be masked are predicted based on the intermediate transaction behavior feature sequence. Calculate the predicted feature score corresponding to the predicted feature, and update the historical transaction behavior feature sequence to the target transaction behavior feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence. The target transaction behavior feature sequence is input into the product click prediction model for processing to obtain product click prediction information, and the target product is recommended to the target user based on the product click prediction information.

[0006] According to a third aspect of the embodiments of this specification, a feature processing apparatus is provided, comprising: The acquisition module is configured to acquire an initial feature sequence and select features to be masked from the initial feature sequence; The masking module is configured to perform masking processing on the feature to be masked, generate an intermediate feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate feature sequence. The update module is configured to calculate the predicted feature score corresponding to the predicted feature, and update the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. The processing module is configured to input the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0007] According to a fourth aspect of the embodiments of this specification, another feature processing apparatus is provided, comprising: The sequence acquisition module is configured to acquire a sequence of historical transaction behavior features of a target user, and select features to be masked from the sequence of historical transaction behavior features. The masking module is configured to perform masking processing on the feature to be masked, generate an intermediate transaction behavior feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate transaction behavior feature sequence. The update sequence module is configured to calculate the predicted feature score corresponding to the predicted feature, and update the historical transaction behavior feature sequence to the target transaction behavior feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence. The product recommendation module is configured to input the target transaction behavior feature sequence into the product click prediction model for processing, obtain product click prediction information, and recommend target products to the target user based on the product click prediction information.

[0008] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described feature processing method.

[0009] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the above-described feature processing method.

[0010] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described feature processing method.

[0011] The feature processing method provided in this embodiment, in order to reduce the impact of noise and improve the prediction accuracy of click prediction information, can process the input features of the model before model prediction. Specifically, after obtaining the initial feature sequence, the features to be masked can be selected first from the initial feature sequence. At this time, masking processing can be performed on the features to be masked, and an intermediate feature sequence can be generated based on the masking processing result. On this basis, the predicted features corresponding to the features to be masked can be predicted based on the inherent correlation between features, thereby fully mining the accurate expression of features before model processing. Afterwards, the predicted feature score corresponding to the predicted feature can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. This achieves accurate representation of real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross features as the denoising object, and using user features as a guide, the product features and user-product cross features are redefined. This allows the model to input accurate and real feature sequences to complete the prediction processing after denoising, thereby effectively improving the model prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information. This allows for generative processing of the model input features before model prediction, effectively reducing noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thus facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results. Attached Figure Description

[0012] Figure 1This is a flowchart of a feature processing method provided in one embodiment of this specification; Figure 2 This is a flowchart of another feature processing method provided in one embodiment of this specification; Figure 3 This is a flowchart illustrating the processing procedure of a feature processing method provided in one embodiment of this specification. Figure 4 This is a schematic diagram of the structure of a feature processing device provided in one embodiment of this specification; Figure 5 This is a schematic diagram of another feature processing device provided in one embodiment of this specification; Figure 6 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0013] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0014] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0016] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0017] The technical solutions provided in this application can employ deep learning models with relatively large parameter scales. However, this large model is merely an example; this application does not limit the number of model parameters supported by the deep learning model used, aiming to meet actual needs. The deep learning models involved in this application can be artificial intelligence-based language models (LM) or multimodal models (MM). First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0018] Symmetric Masking Generative Paradigm for CTR (SGCTR) is an innovative click-through rate (CTR) prediction method that aims to seamlessly integrate the advantages of generative models into the entire CTR prediction process, thereby overcoming the limitations of traditional discriminative paradigms. Its core idea is to employ a generative paradigm in both the training and inference phases. Through symmetric masking and reconstruction mechanisms, the model can more comprehensively capture the complex dependencies in user behavior data, thereby improving the accuracy and robustness of predictions.

[0019] CTR (Click-Through Rate): Click-through rate is a key metric for measuring the effectiveness of digital advertising or recommendation systems. It represents the percentage of users who actually click on an ad or recommendation after it has been shown to them.

[0020] To address the aforementioned technical problems, this specification provides a feature processing method. One or more embodiments of this specification also relate to a feature processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0021] In practical applications, existing generative CTR models suffer from a key problem: they employ an asymmetric paradigm for training and inference. During training, generative objectives are used to enhance the model's understanding of the data distribution and learn robust feature representations. However, during online inference, these models revert to traditional discriminative methods, making predictions based on potentially noisy input samples. This asymmetry means that the robust generative capabilities gained during training are completely discarded during inference, preventing these generative models from fully realizing their predictive potential. Therefore, an effective solution is urgently needed to address this issue.

[0022] The feature processing method provided in this embodiment, in order to reduce the impact of noise and improve the prediction accuracy of click prediction information, can process the input features of the model before model prediction. Specifically, after obtaining the initial feature sequence, the features to be masked can be selected first from the initial feature sequence. At this time, masking processing can be performed on the features to be masked, and an intermediate feature sequence can be generated based on the masking processing result. On this basis, the predicted features corresponding to the features to be masked can be predicted based on the inherent correlation between features, thereby fully mining the accurate expression of features before model processing. Afterwards, the predicted feature score corresponding to the predicted feature can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. This achieves accurate representation of real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross features as the denoising object, and using user features as a guide, the product features and user-product cross features are redefined. This allows the model to input accurate and real feature sequences to complete the prediction processing after denoising, thereby effectively improving the model prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information. This allows for generative processing of the model input features before model prediction, effectively reducing noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thus facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results.

[0023] See Figure 1 , Figure 1 A flowchart of a feature processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0024] Step S102: Obtain the initial feature sequence and select the feature to be masked from the initial feature sequence.

[0025] Step S104: Perform masking processing on the feature to be masked, generate an intermediate feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate feature sequence.

[0026] Step S106: Calculate the predicted feature score corresponding to the predicted feature, and update the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence.

[0027] Step S108: Input the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0028] The feature processing method provided in this embodiment can be applied to click prediction processing in any scenario, such as a product transaction scenario, to predict the click rate of users for each product, and to recommend products that users are interested in (such as products under categories such as electrical appliances, clothing, and daily necessities); or a content browsing scenario, to predict the click rate of users for each published content, and to recommend published content that users are interested in (such as text browsing content, video browsing content, travel guide content, etc.); or a multimedia browsing scenario, to predict the click rate of users for each multimedia resource, and to recommend multimedia resources that users are interested in (such as short videos, novels, music, etc.).

[0029] This embodiment uses the application of feature processing methods in commodity trading scenarios as an example to illustrate the feature processing methods. For descriptions of feature processing methods in other scenarios, please refer to the descriptions in this embodiment. This embodiment will not elaborate further here.

[0030] Specifically, the initial feature sequence refers to a sequence of features constructed based on the model input information, including feature expressions corresponding to at least one dimension. The initial feature sequence can be composed of user features, product features, and user-product cross-features, which are associated with the user dimension, product dimension, and user-product interaction behavior dimension. In practice, user features can be constructed based on user attribute information, such as age, gender, and shopping preferences. Product features can be constructed based on product information corresponding to each candidate product, such as product type, specifications, usage, and price. User-product cross-features can be constructed based on historical user-product interaction behavior information, such as purchases, favorites, and sharing of products over the past month. By constructing a feature sequence using user features, product features, and user-product cross-features, it is possible to redefine product features and user-product cross-features using user features as guidance. This feature denoising allows the model to make predictions based on accurate input features, thereby improving the model's prediction accuracy.

[0031] It should be noted that the user features, product features, and user-product cross-features contained in the initial feature sequence can be constructed by embedding them through the embedding layer in the object click prediction model, so that they can be used for subsequent click-through rate prediction.

[0032] Accordingly, the features to be masked specifically refer to the features in the initial feature sequence that need to be masked. For example, features associated with user attribute information or features corresponding to historical interaction behavior information in the initial feature sequence can be masked to facilitate subsequent prediction of the predicted features corresponding to each masked feature. This achieves the generation of a target feature sequence that matches the real information through iteration, thereby improving the model's prediction accuracy. It should be noted that when selecting the features to be masked from the initial feature sequence, considering that the purpose of selecting the features to be masked is to construct a more accurate feature sequence, one feature to be masked can be selected from the initial feature sequence for processing in each processing cycle. This process is repeated for multiple cycles to update the initial feature sequence into a more accurate target feature sequence. Alternatively, the feature proportion can be used to select the features to be masked from the initial feature sequence in each processing cycle. In specific implementation, this selection strategy can be set according to actual needs, and this embodiment does not impose any limitations on it.

[0033] Accordingly, masking specifically refers to masking the features to be masked in the sequence, which is then used for subsequent feature prediction at that position. Similarly, the intermediate feature sequence refers to the feature sequence obtained after masking the features to be masked in the initial feature sequence, where all features except the features to be masked are inherited from the initial feature sequence. Furthermore, the predicted feature refers to the feature representation obtained by combining the features from the intermediate feature sequence (excluding the masked features to be masked) to predict the features at the masked position. This can be understood as using the inherent relationships between features in the feature sequence to predict the features corresponding to the masked position, which is used to generate features corresponding to each position in the feature sequence using the model's generative capabilities. This improves the accuracy of the mapping between the final target feature sequence and the actual information, and reduces interference from noise.

[0034] Accordingly, the predicted feature score specifically refers to the confidence weight corresponding to the predicted feature, which reflects the degree of similarity between the predicted feature and the true result, and can also indicate whether the predicted feature is reasonable. This allows for subsequent iterative processing to generate the true representation of the feature after each stage masking, thereby improving the model's prediction accuracy. Conversely, the target feature sequence specifically refers to the feature sequence that better matches the true information after iterative correction of each feature. Compared to the initial feature sequence, the target feature sequence has less noise interference, enabling the model to accurately predict click information.

[0035] Accordingly, the object click prediction model specifically refers to a predictive model that, after processing the target feature sequence, predicts the click-through rate of users clicking on information such as products, multimedia resources, and published content. It can be implemented based on a pre-trained and fine-tuned large language model, or it can be implemented using other model structures; this embodiment does not impose any limitations. It can be deployed in any click-through rate prediction scenario to predict user click-through rates for information such as products, multimedia resources, and published content in a specific scenario, so as to recommend information of interest to users. Correspondingly, the object click prediction information is the click-through rate prediction result corresponding to each candidate object. The object can be a product, video, advertisement, music, novel, image, etc., which is set according to the actual service scenario; this embodiment does not impose any limitations.

[0036] Therefore, to mitigate the impact of noise and improve the accuracy of click prediction, the input features of the model can be processed before prediction. Specifically, after obtaining the initial feature sequence, features to be masked can be selected from the initial feature sequence. Masking can then be performed on these features to generate an intermediate feature sequence based on the masking results. Based on this, the predicted features corresponding to the masked features can be predicted according to the intrinsic relationships between features, thus fully exploring the accurate representation of features before model processing. Subsequently, the predicted feature scores corresponding to the predicted features can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature scores and the feature scores corresponding to the features in the intermediate feature sequence. This allows for accurate representation of the real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross-features as the denoising object, and using user features as a guide to redefine product features and user-product cross-features, the model can input accurate and realistic feature sequences to complete the prediction process after denoising, thereby effectively improving the model's prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information.

[0037] For example, to recommend products that users are interested in and improve their shopping experience, shopping platform A predicts the click-through rate (CTR) of each product clicked by the user. First, it can obtain the user's attribute and interaction information on the shopping platform, such as the user's age, candidate product prices, and historical CTR. Based on this information, an initial feature sequence {f1, f2, f3} can be constructed for the user. Then, feature f3, corresponding to the user's historical CTR, can be selected from the initial feature sequence and masked. After masking, an intermediate feature sequence {f1, f2, M} can be obtained. Subsequently, features f1 and f2 can be combined to predict the feature at position M, yielding the feature representation of the masked user's historical CTR. In the process of f3-1, since features f1 and f2 are not masked, their respective confidence levels are 1. However, feature f3-1, after calculation, has a confidence level of S3. Based on this, the features to be masked and predicted in the next round can be selected according to their confidence levels. After n iterations, a more accurate target feature sequence {F1, F2, F3} can be generated. Then, the target feature sequence can be input into the product click-through rate prediction model for processing. The probability of a user clicking on a candidate product is determined to be 0.8. This probability value is greater than the threshold, so the candidate product (such as a shirt from a certain brand) can be recommended to the user to improve the user's shopping experience on the shopping platform and make it easier for them to buy products they are interested in.

[0038] In summary, generative processing of the model input features before model prediction can effectively reduce noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thereby facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results.

[0039] Furthermore, to ensure that the features before model prediction accurately represent actual information and reduce noise interference, the model's generative capabilities can be used to iteratively process the feature sequence. In this embodiment, selecting the features to be masked from the initial feature sequence includes: Selecting features to be masked from the initial feature sequence according to a preset feature masking strategy; wherein, predicting the predicted features corresponding to the features to be masked based on the intermediate feature sequence includes: inputting the intermediate feature sequence into the object click prediction model, wherein the object click prediction model includes a feature processing module; processing the intermediate feature sequence using the feature processing module to obtain the predicted features corresponding to the features to be masked, wherein the feature processing module generates the predicted features based on intermediate features in the intermediate feature sequence.

[0040] Specifically, the preset feature masking strategy refers to the strategy for selecting features to be masked in the initial feature sequence. This strategy controls the number of features to be masked and the selection method, which can be random or sequential. Correspondingly, the feature processing module refers to the module in the object click prediction model used to process the input features. It can not only generate prediction features but also select and mask the features to be masked from the initial input feature sequence.

[0041] Based on this, when selecting features to be masked from the initial feature sequence, a preset feature masking strategy can be used to select features to be masked from the initial feature sequence to avoid the problem of invalid execution of subsequent iterative processing. On this basis, an intermediate feature sequence can be constructed, and then the intermediate feature sequence can be input into the object click prediction model, which includes a feature processing module. At this time, the feature processing module can be used to process the intermediate feature sequence, so that the feature processing module can predict the predicted features corresponding to the features to be masked based on the intermediate features in the intermediate feature sequence, so as to update the feature sequence in the subsequent combination of the predicted features.

[0042] In practice, after obtaining the initial feature sequence, it can be directly input into the object click prediction model for processing. The feature processing module in the model masks and predicts the features so that the feature sequence can be updated to the target feature sequence after subsequent score calculation.

[0043] In practical applications, in addition to the above implementation methods, the feature masking strategy can also be set to dynamically determine the number of features to be masked in each iteration cycle. This can be understood as follows: after each iteration cycle, the corresponding feature sequence will be determined, and the number of masked features to be processed in the next cycle can be dynamically selected by combining the functions in the set of the strategy. Moreover, the number will decrease as the number of iteration cycles increases, so as to gradually optimize to the desired result and thus ensure that the quality of the target feature sequence is higher.

[0044] In summary, driving the model to learn deep relationships between features through mask prediction tasks enhances the robustness of feature representations. The feature processing module reconstructs the masked information based on intermediate features in the context, forcing the model to capture more essential data patterns, thereby improving the model's generalization ability and prediction accuracy, while reducing noise and resulting in higher accuracy in subsequent model prediction processing.

[0045] Furthermore, when calculating the predicted feature score corresponding to the predicted feature, in order to ensure that the predicted feature score can truly express the noise interference level of the predicted feature, the calculation can be completed in conjunction with reference features. In this embodiment, calculating the predicted feature score corresponding to the predicted feature includes: Determine the reference feature corresponding to the feature to be masked; calculate the feature similarity between the reference feature and the predicted feature, and use the feature similarity as the predicted feature score corresponding to the predicted feature.

[0046] Specifically, the reference feature refers to the original value of the feature to be masked in the initial feature sequence. By calculating the similarity between the reference feature and the predicted feature, the confidence level of the prediction result corresponding to the masked position can be reflected, so as to determine the features to be reconstructed in each iteration cycle to generate a high-quality target feature sequence. Correspondingly, the feature similarity refers to the value that reflects the degree of similarity between two features. It can be implemented by the cosine similarity algorithm or by calculating the distance between features. This embodiment does not make any limitation.

[0047] Based on this, when calculating the predicted feature score, the reference feature corresponding to the feature to be masked can be determined from the model first. Since the reference feature is the expected result of the model at the position corresponding to the feature to be masked, the feature similarity between the reference feature and the predicted feature can be calculated. The similarity can characterize the confidence of the predicted feature relative to the feature sequence. Therefore, the feature similarity can be used as the predicted feature score corresponding to the predicted feature for subsequent feature sequence updates.

[0048] In practical applications, the reference feature can be determined by the original value corresponding to the feature to be masked, or by the expected value of the feature processing module. This embodiment does not impose any limitations on this.

[0049] In summary, calculating the predicted feature score by using similarity calculation can ensure that the predicted feature score accurately expresses the confidence level of the predicted feature. This can be used for subsequent iterative feature sequence updates, effectively reducing noise interference in the feature sequence.

[0050] Furthermore, when constructing the target feature sequence, considering that the feature sequence contains multiple features, and each feature needs to be predicted before the overall feature sequence can eliminate the interference caused by noise, it is necessary to construct the target feature sequence iteratively. In this embodiment, before the step of calculating the predicted feature score corresponding to the predicted feature is executed, the following steps are also included: The intermediate feature sequence is updated using the predicted features to obtain a candidate feature sequence; wherein, updating the initial feature sequence to a target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence includes: determining the feature scores corresponding to the candidate features in the candidate feature sequence, sorting the predicted feature score and the feature score; selecting a target candidate feature in the candidate feature sequence according to the sorting result, using the target candidate feature as the feature to be masked, and performing a masking process on the feature to be masked; until a candidate feature sequence that meets the model processing conditions is determined, the candidate feature sequence that meets the model processing conditions is used as the target feature sequence.

[0051] Specifically, the candidate feature sequence refers to the feature sequence obtained by replacing the masked features in the intermediate feature sequence with the predicted features, where the unreplaced features are inherited from the intermediate feature sequence. Correspondingly, the feature score refers to the confidence score of the corresponding feature calculated without masking in the current iteration cycle, which can be set to the default value of 1. Correspondingly, the target candidate feature refers to the feature that needs to be masked in the next iteration cycle. Correspondingly, the model processing condition refers to the condition for determining the target feature sequence, which can be the number of iterations, the confidence score of each feature in the feature sequence being greater than a set threshold, or the condition that each feature is reconstructed; this embodiment does not impose any limitations here.

[0052] Based on this, after obtaining the predicted features, they can be inserted into the intermediate feature sequence at the positions corresponding to the features to be masked, thus obtaining a candidate feature sequence. On this basis, the feature scores corresponding to each candidate feature in the candidate feature sequence can be determined. Then, each feature in the candidate feature sequence can be sorted according to its predicted feature score and the feature score. The sorting result reflects the confidence level of each feature. At this point, a target candidate feature with a lower confidence level can be selected as the feature to be masked, and the masking process described above for the feature to be masked can be repeated. This process can gradually eliminate noise interference and better match the real information in the constructed candidate feature sequence until a candidate feature sequence that meets the model processing conditions is determined. This candidate feature sequence can then be used as the target feature sequence for subsequent prediction processing of object click prediction information.

[0053] In practical applications, the Symmetric Mask Generation Paradigm (SGCTR) for the feature processing module in the object click prediction model mainly involves three key steps: feature generation, feature confidence calculation, and feature redefinition and masking.

[0054] In the feature generation phase, in each iteration, the model first receives an input sample Xt with a mask. For all masked features in Xt, the model generates the corresponding feature values ​​in parallel. For example, the feature generated at a specific position i is represented as... Here, Gt is the feature processing module with generative capabilities. Then, feature confidence is calculated based on the predicted features. At this point, for each masked position i, the features generated by the feature processing module can be used... Its corresponding original input features The similarity score is calculated and can be used as the prediction score, i.e., the confidence score, for position i. =cos( , For feature locations that are not masked, their confidence level can be directly set to 1.

[0055] Furthermore, after obtaining the confidence level corresponding to each feature in the feature sequence, the number of features to be masked in the next iteration cycle can be calculated based on the mask scheduling function γ, which is expressed by the formula lt=[γ( [N], where N is the total number of features in the feature sequence, T is the total number of iterations, and t is the current iteration number. Afterwards, each feature can be sorted according to its confidence score. Then, the lt features with the lowest confidence scores are selected as the features to be masked in the next iteration cycle, while the remaining features remain unchanged, to construct the feature sequence for the next iteration cycle. The feature update can be implemented using the following formulas (1) and (2): (1) (2) in, This represents the input feature sequence for the next iteration cycle. This represents the feature value at position i in the next iteration cycle. This represents the confidence score at position i in the current iteration cycle. This represents the smaller values ​​of the lt confidence scores. By iterating through T cycles, the final, accurately represented target feature sequence can be obtained, which can then be used for click-through rate prediction.

[0056] Following the previous example, after obtaining the initial feature sequence {f1, f2, f3}, we can select the feature f3 corresponding to the user's historical click rate from the initial feature sequence for masking. After masking, we can obtain the intermediate feature sequence {f1, f2, M}. Then, we can combine features f1 and f2 to predict the feature at position M, and obtain the feature expression f3-1 of the masked user's historical click rate. In this process, since features f1 and f2 are not masked, their respective confidence levels are 1. Feature f3-1 can be similar to feature f3, and its corresponding confidence level is determined to be S3 based on the calculation result. Then, the confidence level S3 can be sorted with the confidence levels corresponding to features f1 and f2 respectively. At the same time, the mask scheduling function γ is used to determine that the number of features to be masked in the next iteration cycle is 1. Therefore, based on the sorting results, the feature with the lower confidence level is selected as the feature to be processed in the next iteration cycle. At this time, the feature to be masked in the next cycle is still the feature f3-1 of the user's historical click rate. Based on this, the above process can be repeated. After T cycles, a more accurate target feature sequence {F1, F2, F3} can be generated for subsequent click rate prediction processing of candidate products.

[0057] In summary, by updating the feature sequence through iterative processing, noise removal can be performed on the features corresponding to each position in each iteration cycle, making the final target feature sequence more closely match the real information, thereby effectively improving the accuracy of downstream click-through rate prediction.

[0058] In practical implementation, to ensure that the feature processing module can perform feature masking and iterative updates during the application phase, it needs to be specifically trained during the training phase. In this embodiment, the training of the feature processing module includes: Obtain sample feature sequences and determine the corresponding behavior labels; input the sample feature sequences into the initial feature processing module, and use the initial feature processing module to perform feature forward processing and feature backward processing on the sample feature sequences, and determine the model prediction features based on the processing results; optimize the initial feature processing module based on the behavior labels and the model prediction features until the feature processing module that meets the training stopping condition is obtained.

[0059] Specifically, the sample feature sequence refers to the feature sequence used during the model training phase, and its structure is the same as the initial feature sequence, which will not be elaborated upon in this embodiment. Correspondingly, the behavior label specifically refers to the corresponding click-through rate. The initial feature processing module specifically refers to the feature processing module that has not yet been trained. Specifically, feature forward processing refers to the state transition operation on the sample feature sequence, with the purpose of masking and prediction; correspondingly, feature backward processing refers to the operation of constructing the prediction features corresponding to the mask positions. The training stopping condition refers to the condition for stopping the training of the feature processing module, including but not limited to loss value comparison conditions, validation set verification conditions, or iteration count conditions; this embodiment does not impose any limitations on these conditions.

[0060] Based on this, during the training process of the feature processing module in the model, the other model parameters besides the feature processing module can be frozen first. On this basis, sample feature sequences can be obtained, and the corresponding behavior labels can be determined. Then, the sample feature sequences can be input into the initial feature processing module, which performs forward and backward feature processing on the sample feature sequences to determine the model's predicted features based on the processing results. After that, the initial feature processing module can be optimized based on the behavior labels and the model's predicted features, and the training process can be repeated until a feature processing module that meets the training stopping condition is obtained.

[0061] In summary, by fully training the feature processing module to enable it to perform feature masking and prediction, its deployment allows the input features of the model before click-through rate prediction to more accurately represent relevant information, thereby reducing noise interference and improving the model's prediction accuracy.

[0062] Based on this, the feature processing module can learn predictive capabilities through state transitions and prediction during training. In this embodiment, the step of using the initial feature processing module to perform forward and backward feature processing on the sample features, and determining the model's predictive features based on the processing results, includes: The initial feature processing module is used to perform state transitions on the sample features contained in the sample feature sequence, and intermediate sample feature sequences are determined based on the state transition results; the initial feature processing module is used to process the state transition features in the intermediate sample feature sequence, and model prediction features are determined based on the processing results.

[0063] Specifically, state transition can be understood as the operation of transforming sample features contained in the sample feature sequence between a masked state and the original state, with the aim of enabling the feature processing module to learn about mask processing capabilities. Correspondingly, the intermediate sample feature sequence specifically refers to the sample feature sequence obtained after feature transformation processing.

[0064] Therefore, the forward and backward processing of features is essentially a process of state transition and prediction. Specifically, the initial feature processing module can be used to perform state transitions on the sample features contained in the sample feature sequence. At this point, the intermediate sample feature sequence can be determined based on the state transition results. Then, the initial feature processing module is used to process the state transition features in the intermediate sample feature sequence, and the model prediction features can be determined based on the processing results for subsequent model parameter tuning.

[0065] In practical implementation, during the training phase of the feature processing module, SGCTR can employ a discrete diffusion process to predict mask features. This allows the feature processing module to learn the dependencies between features in the samples, enabling it to master reasonable feature combinations. Specifically, this phase can utilize the forward noise addition and backward denoising processes of discrete diffusion to model the joint probability distribution of features, thereby enhancing the feature processing module's ability to capture the global structure of each sample. Furthermore, by shifting the objective from traditional behavior generation to modeling the distribution of positive and negative samples, the training task can be controlled to align with the CTR objective.

[0066] During training, the input format of the feature processing module is X=[F, y]={ }={x k}, where y is the behavior label. The forward process means that at each time step, each feature has only two states: the original state and the state after conversion to absorption (i.e., masked). This can be expressed by the following formula (3): ( | )= (3) in, This represents the state of feature k at time step t. This represents the original state of feature k. Let M be a function representing the mask probability of feature k at time step t. M represents the mask state.

[0067] Reverse process: The goal of the feature processing module is to learn the ratio (X) t ,t)= To construct the inverse process, where, express In the next state of the forward process, the relevant parameters can be determined by the following formulas (4) and (5): = p0 ( | (4) ( ,t)= p0 ( | (5) Among them, X t This represents the input sample at time step t. This represents the next state predicted by the model at time step t. This represents the set of features masked at time step t. p0 represents the set of features that are not masked at time step t, and p0 is the original data distribution.

[0068] During the training of the feature processing module, the loss function of the following formula (6) can be introduced: (6) In practical implementation, the forward time sampling function λ(t) = As t approaches T, the proportion of mask will increase.

[0069] Based on the above loss function, the feature processing module can predict masked features and unmasked features during pre-training to learn the joint distribution of features. In this process, considering the nature of the recommendation system data, the existence of high cardinality ID features will lead to an excessively large output space. Calculating the complete softmax function in the feature space is computationally intensive. Therefore, to solve this problem, a sampled softmax function can be used to replace the standard softmax function. The true distribution can be approximated by evaluating it on a random subset. The formulas are as follows: Formulas (7) and (8): (7) (8) Where, q θ G represents the feature distribution predicted by the feature processing module, G is the aggregate representation of the unmasked features, and Sk represents the set of possible values ​​for feature k.

[0070] In summary, by using forward and backward processing during model training, the model can learn masking and feature prediction capabilities, which in turn enables it to perform feature iteration processing after deployment, resulting in less noise in the final input feature sequence.

[0071] After obtaining the object click prediction information, the system can determine the objects that the user is interested in based on the object click prediction information and recommend them to the user. In this embodiment, after the step of inputting the target feature sequence into the object click prediction model for processing to obtain the object click prediction information is executed, the system further includes: Determine the candidate object set and target user corresponding to the initial feature sequence; filter at least one target object from the candidate object set according to the object click prediction information, and construct target object recommendation information corresponding to at least one target object; send the target object recommendation information to the target user according to a preset recommendation strategy.

[0072] Specifically, the candidate object set refers to the set of objects to be recommended to the target user. This set can be a collection of products, images, multimedia resources, etc. The candidate object set constructed varies depending on the scenario, and this embodiment does not impose any limitations. Correspondingly, the target user refers to the user associated with the initial feature sequence, and the target object refers to the object selected from the candidate object set for recommendation to the user. The recommendation strategy refers to the strategy for recommending target objects to the user, which can specify the recommendation time, recommendation method, recommendation format, etc. The recommendation method can be SMS, email, pop-up window, etc., and the recommendation format can be text, images, or a combination of text and images, etc.

[0073] Based on this, after obtaining the object click prediction information, the candidate object set and target user corresponding to the initial feature sequence can be determined. At this time, at least one target object can be selected from the candidate object set according to the object click prediction information, and target object recommendation information corresponding to at least one target object can be constructed. On this basis, the target object recommendation information is sent to the target user according to the preset recommendation strategy, thereby completing the recommendation operation to the user, effectively reaching the user and improving the user's participation experience.

[0074] Using the previous example, after obtaining the click-through rate of users relative to the products sold on the platform, we can filter out m products to be recommended from the products sold on the platform based on this click-through rate. Then, we can construct the recommendation information corresponding to each recommended product, so that when users enter the shopping platform application, the details of the m recommended products can be displayed in the form of a pop-up window, so that users can make purchases and improve the user's shopping experience.

[0075] In summary, by combining object click prediction information with target object filtering and recommendation, we can ensure that the objects recommended to users are those that they are interested in, thereby improving the user's participation experience in service scenarios.

[0076] Furthermore, to ensure that the target feature sequence contains less noise and more accurately represents the true information, it can be compared with the reference feature sequence corresponding to the original information set. In this embodiment, before the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information, the method further includes: Obtain the original information set corresponding to the initial feature sequence, input the original information set into the feature embedding model for processing, and obtain a reference feature sequence; compare the reference feature sequence with the target feature sequence; if the target feature sequence meets the prediction conditions according to the comparison result, execute the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0077] Specifically, the original information set refers to the set of original information corresponding to the initial feature sequence before it is constructed, including but not limited to user attribute information and behavioral information. Correspondingly, the feature embedding model refers to a model that performs feature embedding processing on the original information set, and the resulting feature sequence is the reference feature sequence. Comparing the reference feature sequence with the initial feature sequence can be understood as comparing their similarity. Higher similarity results in lower denoising performance for the target feature sequence, and vice versa.

[0078] Based on this, after obtaining the target feature sequence, it can be verified. In this process, the original information set corresponding to the initial feature sequence can be obtained first. At this time, the original information set can be input into the feature embedding model for processing to obtain the reference feature sequence. Then, the reference feature sequence can be compared with the target feature sequence. The comparison result can reflect the similarity between the target feature sequence and the reference feature sequence. The higher the similarity, the worse the optimization result of the target feature sequence. Conversely, the lower the similarity, the better the optimization result of the target feature sequence. Therefore, if the target feature sequence meets the prediction conditions based on the comparison result, it can be said that the target feature sequence has eliminated a lot of noise, making the noise interference smaller and more in line with the real information expression. Then, the step of inputting the target feature sequence into the object click prediction model for processing to obtain the object click prediction information can be executed.

[0079] In summary, by comparing the updated target feature sequence with the reference feature sequence, the feature sequence before model processing can be verified, thereby avoiding inaccurate prediction accuracy caused by inaccurate feature processing and effectively improving service stability.

[0080] In summary, the feature processing method provided in this embodiment proposes a Symmetric Masked Generative Paradigm (SGCTR) for CTR prediction. By learning feature dependencies to acquire generative capabilities, SGCTR applies these capabilities to iteratively redefine the features of input samples during online inference, thereby effectively mitigating the impact of noisy features and improving the prediction accuracy of the model.

[0081] The feature processing method provided in this embodiment, in order to reduce the impact of noise and improve the prediction accuracy of click prediction information, can process the input features of the model before model prediction. Specifically, after obtaining the initial feature sequence, the features to be masked can be selected first from the initial feature sequence. At this time, masking processing can be performed on the features to be masked, and an intermediate feature sequence can be generated based on the masking processing result. On this basis, the predicted features corresponding to the features to be masked can be predicted based on the inherent correlation between features, thereby fully mining the accurate expression of features before model processing. Afterwards, the predicted feature score corresponding to the predicted feature can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. This achieves accurate representation of real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross features as the denoising object, and using user features as a guide, the product features and user-product cross features are redefined. This allows the model to input accurate and real feature sequences to complete the prediction processing after denoising, thereby effectively improving the model prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information. This allows for generative processing of the model input features before model prediction, effectively reducing noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thus facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results.

[0082] See Figure 2 , Figure 2 A flowchart of another feature processing method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0083] Step S202: Obtain the historical transaction behavior feature sequence of the target user, and select the feature to be masked from the historical transaction behavior feature sequence.

[0084] Step S204: Perform masking processing on the feature to be masked, generate an intermediate transaction behavior feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate transaction behavior feature sequence.

[0085] Step S206: Calculate the predicted feature score corresponding to the predicted feature, and update the historical transaction behavior feature sequence to the target transaction behavior feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence.

[0086] Step S208: Input the target transaction behavior feature sequence into the product click prediction model for processing to obtain product click prediction information, and recommend target products to the target user based on the product click prediction information.

[0087] This embodiment provides another feature processing method. For any content not described in detail, please refer to the same or corresponding descriptions in the above embodiments. This embodiment will not elaborate further here.

[0088] Specifically, the target user refers to the user to whom product recommendations are to be made; this can be understood as a user registered on the shopping platform. The historical transaction behavior feature sequence refers to the feature sequence constructed by combining user attribute information and historical transaction behavior information. The product click prediction model refers to the probability prediction model that a user will click on candidate products. Correspondingly, the target product refers to the product that can be recommended to the user, selected from the candidate product set based on the product click prediction information. The number of target products can be determined according to actual needs. If there are multiple target products, they can be sorted according to the click prediction information, and the top-k target products can be selected to improve the user's shopping experience.

[0089] Based on this, in a product recommendation scenario, the historical transaction behavior feature sequence of the target user can be obtained first, and features to be masked can be selected from this sequence. Then, masking processing can be performed on these features, generating an intermediate transaction behavior feature sequence based on the masking results. The predicted features corresponding to the masked features can then be predicted based on this intermediate sequence. Based on this, the predicted feature score corresponding to the predicted feature can be calculated. According to the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence, the historical transaction behavior feature sequence is updated to the target transaction behavior feature sequence. At this point, the target transaction behavior feature sequence can be input into a product click prediction model for processing to obtain product click prediction information. Downstream services can then recommend target products to the target user based on this product click prediction information, thereby improving the user's shopping experience and ensuring that the recommended products better meet the user's shopping needs.

[0090] The following is in conjunction with the appendix Figure 3 Taking the application of the feature processing method provided in this specification in a product recommendation scenario as an example, the feature processing method will be further explained. Figure 3 The present specification shows a flowchart of a feature processing method according to an embodiment, which includes the following steps.

[0091] Step S302: Obtain the initial feature sequence and select the features to be masked in the initial feature sequence according to the preset feature masking strategy.

[0092] Step S304: Input the intermediate feature sequence into the object click prediction model, wherein the object click prediction model includes a feature processing module.

[0093] Step S306: The intermediate feature sequence is processed by the feature processing module to obtain the predicted features corresponding to the features to be masked. The feature processing module generates the predicted features based on the intermediate features in the intermediate feature sequence.

[0094] Step S308: Update the intermediate feature sequence using the predicted features to obtain the candidate feature sequence.

[0095] Step S310: Determine the reference feature corresponding to the feature to be masked, calculate the feature similarity between the reference feature and the predicted feature, and use the feature similarity as the predicted feature score corresponding to the predicted feature.

[0096] Step S312: Determine the feature scores corresponding to the candidate features in the candidate feature sequence, and sort the predicted feature scores and feature scores.

[0097] Step S314: Select the target candidate feature from the candidate feature sequence according to the sorting result, and use the target candidate feature as the masking feature.

[0098] Step S316: Until a candidate feature sequence that meets the model processing conditions is determined, the candidate feature sequence that meets the model processing conditions is taken as the target feature sequence.

[0099] Step S318: Input the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0100] Step S320: Determine the recommended products based on the object click prediction information and recommend the recommended products to the user.

[0101] The feature processing method provided in this embodiment, in order to reduce the impact of noise and improve the prediction accuracy of click prediction information, can process the input features of the model before model prediction. Specifically, after obtaining the initial feature sequence, the features to be masked can be selected first from the initial feature sequence. At this time, masking processing can be performed on the features to be masked, and an intermediate feature sequence can be generated based on the masking processing result. On this basis, the predicted features corresponding to the features to be masked can be predicted based on the inherent correlation between features, thereby fully mining the accurate expression of features before model processing. Afterwards, the predicted feature score corresponding to the predicted feature can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. This achieves accurate representation of real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross features as the denoising object, and using user features as a guide, the product features and user-product cross features are redefined. This allows the model to input accurate and real feature sequences to complete the prediction processing after denoising, thereby effectively improving the model prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information. This allows for generative processing of the model input features before model prediction, effectively reducing noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thus facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results.

[0102] Corresponding to the above method embodiments, this specification also provides embodiments of feature processing apparatus. Figure 4 A schematic diagram of a feature processing apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes: The acquisition module 402 is configured to acquire an initial feature sequence and select features to be masked from the initial feature sequence; The masking module 404 is configured to perform masking processing on the feature to be masked, generate an intermediate feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate feature sequence. The update module 406 is configured to calculate the predicted feature score corresponding to the predicted feature, and update the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. The processing module 408 is configured to input the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0103] In an optional embodiment, selecting the feature to be masked from the initial feature sequence includes: Selecting features to be masked from the initial feature sequence according to a preset feature masking strategy; wherein, predicting the predicted features corresponding to the features to be masked based on the intermediate feature sequence includes: inputting the intermediate feature sequence into the object click prediction model, wherein the object click prediction model includes a feature processing module; processing the intermediate feature sequence using the feature processing module to obtain the predicted features corresponding to the features to be masked, wherein the feature processing module generates the predicted features based on intermediate features in the intermediate feature sequence.

[0104] In an optional embodiment, calculating the predicted feature score corresponding to the predicted feature includes: Determine the reference feature corresponding to the feature to be masked; calculate the feature similarity between the reference feature and the predicted feature, and use the feature similarity as the predicted feature score corresponding to the predicted feature.

[0105] In an optional embodiment, before the step of calculating the predicted feature score corresponding to the predicted feature is performed, the method further includes: The intermediate feature sequence is updated using the predicted features to obtain a candidate feature sequence; wherein, updating the initial feature sequence to a target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence includes: determining the feature scores corresponding to the candidate features in the candidate feature sequence, sorting the predicted feature score and the feature score; selecting a target candidate feature in the candidate feature sequence according to the sorting result, using the target candidate feature as the feature to be masked, and performing a masking process on the feature to be masked; until a candidate feature sequence that meets the model processing conditions is determined, the candidate feature sequence that meets the model processing conditions is used as the target feature sequence.

[0106] In an optional embodiment, the training of the feature processing module includes: Obtain sample feature sequences and determine the corresponding behavior labels; input the sample feature sequences into the initial feature processing module, and use the initial feature processing module to perform feature forward processing and feature backward processing on the sample feature sequences, and determine the model prediction features based on the processing results; optimize the initial feature processing module based on the behavior labels and the model prediction features until the feature processing module that meets the training stopping condition is obtained.

[0107] In an optional embodiment, the step of using the initial feature processing module to perform forward and backward feature processing on the sample features, and determining the model prediction features based on the processing results, includes: The initial feature processing module is used to perform state transitions on the sample features contained in the sample feature sequence, and intermediate sample feature sequences are determined based on the state transition results; the initial feature processing module is used to process the state transition features in the intermediate sample feature sequence, and model prediction features are determined based on the processing results.

[0108] In an optional embodiment, after the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information is executed, the method further includes: Determine the candidate object set and target user corresponding to the initial feature sequence; filter at least one target object from the candidate object set according to the object click prediction information, and construct target object recommendation information corresponding to at least one target object; send the target object recommendation information to the target user according to a preset recommendation strategy.

[0109] In an optional embodiment, before the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information is executed, the method further includes: Obtain the original information set corresponding to the initial feature sequence, input the original information set into the feature embedding model for processing, and obtain a reference feature sequence; compare the reference feature sequence with the target feature sequence; if the target feature sequence meets the prediction conditions according to the comparison result, execute the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information.

[0110] The feature processing device provided in this embodiment, in order to reduce the impact of noise and improve the prediction accuracy of click prediction information, can process the input features of the model before model prediction. Specifically, after obtaining the initial feature sequence, features to be masked can be selected from the initial feature sequence. At this time, masking processing can be performed on the features to be masked, and intermediate feature sequences can be generated based on the masking processing results. On this basis, the predicted features corresponding to the features to be masked can be predicted based on the inherent correlation between features and the intermediate feature sequences, thereby fully mining the accurate expression of features before model processing. Afterwards, the predicted feature scores corresponding to the predicted features can be calculated, and the initial feature sequence can be updated to the target feature sequence based on the predicted feature scores and the feature scores corresponding to the features in the intermediate feature sequence. This achieves accurate representation of real behavioral sequence information through the target feature sequence. In this process, by using the initial feature sequence containing user features, product features, and user-product cross features as the denoising object, and using user features as a guide, the product features and user-product cross features are redefined. After denoising processing, the model can input accurate and real feature sequences to complete the prediction processing, thereby effectively improving the model prediction accuracy. The target feature sequence can then be input into the object click prediction model for processing to obtain object click prediction information. This allows for generative processing of the model input features before model prediction, effectively reducing noise in the final target feature sequence. Using this as model input can effectively improve the model prediction accuracy, thus facilitating downstream services to perform efficient and accurate recommendation processing based on the prediction results.

[0111] The above is an illustrative scheme of a feature processing device according to this embodiment. It should be noted that the technical solution of this feature processing device and the technical solution of the feature processing method described above belong to the same concept. For details not described in detail in the technical solution of the feature processing device, please refer to the description of the technical solution of the feature processing method described above.

[0112] Corresponding to the above method embodiments, this specification also provides embodiments of feature processing apparatus. Figure 5 A schematic diagram of another feature processing apparatus provided in one embodiment of this specification is shown. Figure 5 As shown, the device includes: The sequence acquisition module 502 is configured to acquire the historical transaction behavior feature sequence of the target user and select the feature to be masked in the historical transaction behavior feature sequence. The masking module 504 is configured to perform masking processing on the feature to be masked, generate an intermediate transaction behavior feature sequence based on the masking processing result, and predict the predicted feature corresponding to the feature to be masked based on the intermediate transaction behavior feature sequence. The update sequence module 506 is configured to calculate the predicted feature score corresponding to the predicted feature, and update the historical transaction behavior feature sequence to the target transaction behavior feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence. The product recommendation module 508 is configured to input the target transaction behavior feature sequence into the product click prediction model for processing, obtain product click prediction information, and recommend target products to the target user based on the product click prediction information.

[0113] In summary, we can first obtain the historical transaction behavior feature sequence of the target user and select features to be masked from this sequence. Then, we can perform masking processing on these features, generate an intermediate transaction behavior feature sequence based on the masking results, and predict the corresponding predicted features based on this intermediate sequence. Based on this, we can calculate the predicted feature scores for the predicted features. Then, based on the predicted feature scores and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence, we update the historical transaction behavior feature sequence to the target transaction behavior feature sequence. At this point, we can input the target transaction behavior feature sequence into a product click prediction model for processing to obtain product click prediction information. Downstream services can then recommend target products to the target user based on this product click prediction information, thereby improving the user's shopping experience and ensuring that the recommended products better meet the user's shopping needs.

[0114] The above is an illustrative scheme of a feature processing device according to this embodiment. It should be noted that the technical solution of this feature processing device and the technical solution of the feature processing method described above belong to the same concept. For details not described in detail in the technical solution of the feature processing device, please refer to the description of the technical solution of the feature processing method described above.

[0115] Figure 6 A structural block diagram of a computing device 600 according to one embodiment of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0116] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0117] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0118] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0119] The processor 620 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described feature processing method.

[0120] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the feature processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the feature processing method described above.

[0121] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described feature processing method.

[0122] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the feature processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the feature processing method described above.

[0123] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described feature processing method.

[0124] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above-described feature processing method belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described feature processing method.

[0125] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0126] The computer program / instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0127] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0128] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0129] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A feature processing method, comprising: Obtain an initial feature sequence, and select features to be masked from the initial feature sequence; The feature to be masked is subjected to masking processing, an intermediate feature sequence is generated based on the masking processing result, and the predicted feature corresponding to the feature to be masked is predicted based on the intermediate feature sequence. Calculate the predicted feature score corresponding to the predicted feature, and update the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence. The target feature sequence is input into the object click prediction model for processing to obtain object click prediction information.

2. The feature processing method according to claim 1, wherein selecting the feature to be masked in the initial feature sequence includes: According to a preset feature masking strategy, features to be masked are selected from the initial feature sequence; Wherein, predicting the predicted feature corresponding to the feature to be masked based on the intermediate feature sequence includes: The intermediate feature sequence is input into the object click prediction model, wherein the object click prediction model includes a feature processing module; The feature processing module processes the intermediate feature sequence to obtain the predicted features corresponding to the features to be masked, wherein the feature processing module generates the predicted features based on the intermediate features in the intermediate feature sequence.

3. The feature processing method according to claim 1, wherein calculating the predicted feature score corresponding to the predicted feature includes: Determine the reference feature corresponding to the feature to be masked; Calculate the feature similarity between the reference feature and the predicted feature, and use the feature similarity as the predicted feature score corresponding to the predicted feature.

4. The feature processing method according to claim 1, further comprising, before the step of calculating the predicted feature score corresponding to the predicted feature, the method includes: The intermediate feature sequence is updated using the predicted features to obtain a candidate feature sequence; The step of updating the initial feature sequence to the target feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate feature sequence includes: Determine the feature scores corresponding to the candidate features in the candidate feature sequence, and sort the predicted feature scores and the feature scores; Based on the sorting results, a target candidate feature is selected from the candidate feature sequence, the target candidate feature is used as the feature to be masked, and a masking process is performed on the feature to be masked. Once a candidate feature sequence that meets the model's processing conditions is identified, it is taken as the target feature sequence.

5. The feature processing method according to claim 2, wherein training of the feature processing module includes: Obtain the sample feature sequence and determine the behavior label corresponding to the sample feature sequence; The sample feature sequence is input into the initial feature processing module, and the initial feature processing module performs feature forward processing and feature backward processing on the sample feature sequence. The model prediction features are determined based on the processing results. The initial feature processing module is optimized based on the behavior label and the model prediction features until a feature processing module that meets the training stopping condition is obtained.

6. The feature processing method according to claim 5, wherein the step of performing forward and backward feature processing on the sample features using the initial feature processing module, and determining the model prediction features based on the processing results, includes: The initial feature processing module is used to perform state transitions on the sample features contained in the sample feature sequence, and the intermediate sample feature sequence is determined based on the state transition results. The initial feature processing module is used to process the state transition features in the intermediate sample feature sequence, and the model prediction features are determined based on the processing results.

7. The feature processing method according to any one of claims 1 to 6, after the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information is performed, it further includes: Determine the candidate object set and target user corresponding to the initial feature sequence; Based on the object click prediction information, at least one target object is selected from the candidate object set, and target object recommendation information corresponding to each of the at least one target object is constructed. The target object recommendation information is sent to the target user according to the preset recommendation strategy.

8. The feature processing method according to any one of claims 1 to 6, wherein before the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information is performed, the method further includes: Obtain the original information set corresponding to the initial feature sequence, input the original information set into the feature embedding model for processing, and obtain the reference feature sequence; The reference feature sequence is compared with the target feature sequence; If the target feature sequence is determined to meet the prediction conditions based on the comparison results, the step of inputting the target feature sequence into the object click prediction model for processing to obtain object click prediction information is performed.

9. A feature processing method, comprising: Obtain the historical transaction behavior feature sequence of the target user, and select the feature to be masked from the historical transaction behavior feature sequence; Masking processing is performed on the features to be masked, an intermediate transaction behavior feature sequence is generated based on the masking processing result, and the predicted features corresponding to the features to be masked are predicted based on the intermediate transaction behavior feature sequence. Calculate the predicted feature score corresponding to the predicted feature, and update the historical transaction behavior feature sequence to the target transaction behavior feature sequence based on the predicted feature score and the feature scores corresponding to the features in the intermediate transaction behavior feature sequence. The target transaction behavior feature sequence is input into the product click prediction model for processing to obtain product click prediction information, and the target product is recommended to the target user based on the product click prediction information.

10. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

12. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

Images